Coaching Tools – The Assessment

NOTE: This article and the coaching tool it describes is an update to the older gap analysis tool described in Developing Strong Product Managers.

In my last several articles, I have been focusing on coaching tools for helping managers of product managers to raise the level of performance of the product managers that report to them.

I want every manager of product managers to feel considerable urgency and importance around this need.  Your cross-functional product teams depend on competent product managers, and if you don’t develop your people and provide growth opportunities, there are usually other companies that will.  I have always been a big believer in the old adage that “people join a company, but leave their manager.”

This article discusses the technique that I use and advocate for assessing a product manager. It is the foundation for then coaching the person to success.

This skills assessment is structured in the form of a gap analysis.  The purpose is to assess the product manager’s current level of competence along each of several necessary dimensions, and then compare that with the level of competence that’s required for this particular team and company.

This format acknowledges that not all skills are equally important, not all gaps are equally significant, and expectations change with the level of responsibility.  This tool is intended to help focus the attention where it is most needed.

People, Process and Product

As many of you know, the taxonomy I like to use when talking about product management is the three pillars: people, process and product.

For purposes of the assessment tool, I like to cover “product” first because product knowledge is the foundation for everything else.  Without competence in product knowledge, the rest doesn’t really matter.

Product Knowledge:

• User and Customer Knowledge – Is the product manager a company acknowledged expert on her target users/customers?
• Data Knowledge – Is the product manager skilled with the various data tools and considered by her product team and her stakeholders as an acknowledged expert in how the product is actually used by users?
• Industry and Domain Knowledge – Is the product manager knowledgeable about the industry and domain?  Does she understand the competitive landscape and the relevant industry trends?
• Business and Company Knowledge – Does the product manager understand the various dimensions of your company’s business – marketing, sales, finance (both revenue and costs), services, legal, privacy, etc.?  And do the stakeholders believe that the product manager understands their concerns and constraints?
• Product Operational Knowledge – Is the product manager considered an acknowledged expert on how her product actually works?  Would she be able to effectively demo to a prospective customer, train a new customer on how to successfully use, and handle live customer support inquiries?

Product Knowledge really is table stakes.  A new product manager typically requires 2-3 months to ramp up to speed on product knowledge assuming she dives in aggressively and spends several hours per day learning.

Process Skills and Techniques:

• Product Discovery Techniques – Does the product manager have a strong understanding of the product risks and how to address each of them?  Does she understand how to tackle risks up front, before engineers are asked to build?  Does she know how to solve problems collaboratively? Does she focus on outcome?  Does she understand and utilize both qualitative and quantitative techniques?
• Product Optimization Techniques – Once a product or new capability is live and in production, does the product manager know how to utilize optimization techniques to rapidly improve and refine her product?
• Product Delivery Techniques – While the product manager’s primary responsibility is discovery, she still has an important supporting role to play in delivery.  Does she understand her responsibilities to the engineers and to product marketing?
• Product Development Process – Does the product manager have a solid understanding of the broader product development process including discovery and delivery, as well as the product manager’s administrative responsibilities as the team’s product owner?

New product managers are expected to know the basic techniques, but good product managers are always developing their skills and learning new and more advanced techniques.  Much like a good surgeon is constantly following the latest learnings on surgical skills and techniques, a strong product manager always has more to learn in terms of process and techniques.

People Skills and Responsibilities:

• Team Collaboration Skills – How effectively does the product manager work with her developers and product designer?  Is it a collaborative relationship?  Is there mutual respect?  Is the product manager involving the developers and designer early enough and providing them direct access to customers?   Is the product manager fully leveraging her team’s skills and minds?
• Stakeholder Management Skills – How good is the product manager at managing her stakeholders across the company?  Do they feel like they have a true partner in product that is genuinely committed to their business success?  Has she established mutual respect and mutual trust with each stakeholder, including the senior leadership of the company?
• Evangelism Skills – Is the product manager able to effectively share the product’s vision and strategy, and motivate and inspire her product team, as well as the various stakeholders and others in the company that must contribute to the product in one way or another?
• Leadership Skills – While the product manager does not actually manage anyone, she does need to influence and inspire people, so leadership skills are important.  Is she an effective communicator and motivator?  Do her team and her stakeholders look to her for leadership especially in stressful situations?

People skills are similar to product knowledge in that if you don’t have a solid foundation, it is very hard to do the PM job at all; yet as with process skills, strong product managers are constantly working to improve and develop their people skills.

NOTE: The taxonomy above is the set of skills and techniques that I generally use, however, in certain situations I customize this list based on the company’s culture or the company’s industry.  As an example, in media companies, there is a special and critically important relationship between product and editorial, and I like to call this relationship out explicitly, and not leave it bundled in with other stakeholders.  Which is to say, if you as the manager of product managers think you should adjust this taxonomy of skills, then by all means you should do that.

The Gap Analysis

Now that we have the skills taxonomy, the core of this technique is a gap analysis.  The way this works is that the manager should review the set of criteria above, and assign two ratings to each skill.

The Importance vs. The Current Capability

The first rating is an assessment of where the employee needs to be in this skill (i.e. how important it is), and the second rating is an assessment of where the employee currently performs on this scale (i.e. her ability).  I typically rate these on a 1-10 scale, with 10 being a skill that is absolutely essential to the job.

So, for example, if you consider a skill such as “Product Discovery Techniques” as an 8 in importance, yet you assess your product manager’s current skill level as a 4, then there is a significant gap for a highly ranked skill, and you’ll need to get to work coaching the product manager on her knowledge or skills in this key area.

NOTE: Normally the difference between a product manager and senior product manager, for example, is captured in the expectations of where the level of skills should be at (the importance rating).  As just one example, I usually rate stakeholder management skills as a 7 in importance for a standard product manager level person, yet I consider this skill a 9 for a senior product manager.

NOTE: The level of importance is always set by the manager, if not the organization as a whole.  The more difficult rating is the capability rating.  Normally the assessment of the product manager’s capability level is done by the manager.  However, there is no reason why the product manager can’t also do a self-assessment, and in fact I encourage that, but be aware that it is not at all uncommon for the self-assessment to expose some significant differences with how the product manager assesses her own capabilities.  A manager that only relies on the self-assessment because they are uncomfortable confronting these differences in perception, is in my view abdicating their responsibilities as a manager.

NOTE: I am sharing here my default set of skills and criteria.  However, you should feel free to adjust this based on the type of products you provide, the culture of your company, and the level of expectations for this level product manager (normally product manager, senior product manager, group product manager or principal product manager).  For group product managers (GPM), there is another category in the taxonomy because a GPM is also responsible for developing a small number of product managers.

NOTE: With this tool you can hopefully see why I’m not a fan of the “associate product manager” level.  If a person is competent on the above dimensions, then to me they deserve the unqualified product manager title.  If they’re not yet competent, then this is a problem, and it’s urgent to get the PM to competence quickly (normally within 2-3 months), and during this time the manager is going to need to oversee and probably help with much of the work.  If the manager is not able to quickly get that person to competence, then the manager needs to help that person find a more suitable position, and recruit someone that is capable of doing the necessary job.

The Coaching Plan

Now that we’ve done a skills assessment and subsequent gap analysis, we are looking for the areas with the biggest gaps.   That’s the purpose of this assessment.

For the coaching plan, I like to limit the initial focus to the top 3 areas.  After progressing on these, the product manager can move on to the next most important areas.

As the manager, you can now provide this product manager with coaching, training, reading or exercises intended to develop her skills in each area.

In the next article, I will share what I typically recommend in a coaching plan for developing each skill in the taxonomy described above.  But many of you already know how to coach a product manager in a specific skill, so all you really need is the assessment and gap analysis described here.

Also, once an employee has successfully closed the gaps, it is the ideal time to show her how the importance ratings move for the next level position, and she can set about developing and demonstrating the skills necessary for a promotion.

Be sure to sit down with each of your product managers no less than once a week to discuss progress on the coaching plan.

Assessments vs. Performance Reviews

Finally, you might be wondering how this sort of skills assessment and coaching plan relates to annual performance reviews.  In general, I find the way most companies implement performance reviews to be of little use in terms of developing people.  Sadly, they are more about HR compliance and pay administration.

You may have to comply with your HR department’s requirements in terms of annual reviews, but just realize that these are in no way an adequate substitute for active, ongoing, engaged coaching and development of each team member’s skills.  The good news is that if you are actively managing the skills assessment and the coaching plan as I’m describing here, then the annual review fire-drill is much easier.