How to measure things? - AMA with Barak Witkowski
Barak is a seasoned entrepreneur who has successfully launched many products, both B2B and B2C, with tens of millions of users worldwide. Barak is passionate about translating a dream into a real product, solving pain points and delighting the users.
So let's jump straight into the AMA!
How to measure what matters the most?
In this AMA, Barak answered questions about Product Management, setting smart & measurable KPIs and using data to create sustainable product strategy . Find out more below!
For someone starting their career in Product Management, how should they decide what to do next?
To be honest, I think that a product manager is the classic case of “not having a cookbook”.
That’s the challenging part, and the great part in being a product manager. My general framework for product management is:
- What is the most important thing to do right now for the success of my products (which, btw, doesn’t necessarily have to do with improving the product. It might be training CSMs, generating collaterals etc).
- What can I do to make it happen?
In other words, you should see yourself as the person in charge of the success of your product, whatever it means, and the one in charge making it happen.If you will follow that specific flow, you can keep growing more and more as a product manager, and succeed in any organisation.
What would you say are the hard skills/soft skills the best PMs possess?
Skills of the best PM:
- Impact driven
- Biz driven
- Executor & enabler - making things happen
- Great collaborator, communicator and team player
- Passionate, LOVING your product
- Leading without formal authority
- Endless passion and curiosity
What are the top 3 skills a PM should work on early in their career?
Top 3 skills:
- How to work with data - being data driven, but don’t let data manage you.
- Identifying what’s the right thing you should do right now to move the needle of success of your product. It’s an art, a skill to develop.
Making things happen - there are many excuses. But you should have the skills to ensure the right things are performed. Usually it depends on other people, ones that you don’t manage, but you should develop the skills to get their buy-in and make it happen.
What differentiates a good product manager from an excellent product manager in your opinion?
So, I think the main thing which differentiates an excellent PM from a good PM relies in understanding what is their job…A PM is NOT:
- A ‘factory’ to deliver more and more features
- Manager of the scrum of engineers and project manager
- Spec writer
- Email replier
- Escalation handler
A PM is:
the person in charge of the success of their product! Building more features, changing pricing, changing marketing, training the team, EVERYTHING. So, an excellent PM knows to move the right needle of the success of their product, and make sure it happens.
What skills do you think one as a PM should have to effectively work with Developers?
Being an engineer certainly helps working with engineers, but it’s not mandatory.
As a PM, you must have soft skills to get the buy-in of the engineers and engage them with the big picture.Be honest and transparent with them. Explain to them the motivation of the decisions. Share with them business successes and failures. Once they will understand the impact of what they’re building, you will work as one united group.
As a PM, how can one deal with the unrealistic timelines given by developers?
It really depends on which industry you’re working in.
The best way to progress here is actually learning from experience, and knowing your counterparts from R&D. Are they usually over-estimating? Under-estimating?As a PM, you should make the people around you be accountable. Engineers need to understand the impact of their estimations. It changes your plans, you build a go-to-market plan according to it, sometimes it’s communicated as a commitment to the customers.So, it’s three things:
- Growing your instincts about their estimations, learning from each such case, so next time you could challenge their estimations based on experience.
- Growing their accountability to their estimations
- Define the right processes (how much buffer should be added before something is communicated outside? At which level the buffer should be added?) etc.
Data driven PM or insights driven PM? What do you prefer? What would be the correct balance in your opinion?
A great PM must be data driven, but can’t take decisions only based on ‘dry data’, simply because data (including big data) can easily lie. Always, in each organisation, there are blind spots to the data, and easily data can be presented in a biassed way. So, the right balance in my opinion is:
- Be data driven - know to collect insights from your data.
- Challenge the data! Be familiar with the market trends, generate good instincts in your market, learn from the A/B tests you’ve already done.
In other words - as a PM you should manage the data, don’t let the data manage you.
How do you decide on the key metrics to measure for the initial, say 6 months, when launching a new product in the market?
Launching a product is not the end of your role as a PM, actually it’s just the beginning!As product people, we’re not a factory of releasing new features, but we need to create an impact. So, I’m happy to see that you want to focus on measuring the impact of your product.
So, deciding the key metrics actually relates exactly to the IMPACT you want to generate, and then focus your measurement around that.Are you developing this feature to increase stickiness? measure that.
Are you developing this feature to cause the customer to upsell? measure that.If you can focus me a bit more on which type of product you refer (B2C? B2B? Mobile? Saas? Is it a standalone product or a part of a bigger suite?) - I can give a more specific answer.
What is the right way to set north star metrics?
North star metrics is something you will never fulfil. It’s an important character of a north star. If you will ever achieve it, it means it’s a wrong north star…A good north star is something which:
- Helps to take strategic decisions upon it. Something that helps your entire organisation to ask: “but how does it serve our north star?“.
- Something that defines the IMPACT your product wants to make in the world. Not the WHAT you are doing, but the WHY. Then, you ensure that all the products you deliver serve this impact.
How do we understand the metrics of a specific feature has increased by product or through marketing ?
Depends which feature.
Usually, the best is to measure it via the product, as usually that’s the IMPACT you want to generate.
You want customers to use your product, and get value out of this feature, so measure exactly that (via the product).
What could be some of the good product-led-growth strategies?
PLG is a hot topic these days!
I treat it as a sort of a magnifying glass, where the product itself unifies all the business KPIs (many different ray of lights), and makes one ray of light (the product experience) out of it. So, you should find the strategy where your product experience helps you not only to serve your customers, but also:
- To grow your customers number
- Reduce churn
- Generate feature visibility
- Drive upsells
You've been on both sides of the table doing B2B and B2C, what are some major differences when it comes to building products for these two industries?
The way I feel it, there are two main differences in the day-to-day:
- In B2C the ‘customers’ are much less obligated to you. They installed your app (I developed mobile apps, so I refer to that) without friction, but they can also uninstall it without thinking twice. On the other hand, in b2b the friction to start using the service is higher, but also the friction to leave is higher.
- In B2C, my testing cycles were much more lean. Actually, I let the users themselves be the testers (released the product gradually to some users, checked their behaviour, checked the stability, and then rolled out to all users). In B2B, we don’t do that…
But actually, there are many things which are also similar, including the importance of K factor and word-of-mouth in both.
How much time do you spend your day looking at data and uncovering insights? What would be a good habit to have to ensure you’re up to date about your customers from a quantitative POV?
I personally look at data just few minutes a day
But - there is a hard work by our data analysis teams to create to each person (including me) exactly the dashboards needed, in order for it to take just few minutes.But - it’s not only to save everyone’s (and mine) time. It’s also because as a product organisation you need to focus.
You can’t have dozens of KPIs. You should have a few.Then, when you find something disturbing in one of your major KPIs, you should dig deeper to understand why.
What would you say is more important in understanding your user - Qualitative feedback from Customers or data backed insights?
Qualitative vs. quantitative - I would roughly say: 80% quantitative, as long as you have enough data. It’s much more scalable.
Can you please talk more about how to use data to create a sustainable product strategy?
How to use data for a sustainable product strategy - define a few data KPIs you follow (5 as a ballpark), ensure the entire org is aligned around them, ensure they’re measureable easily, ensure they relate to your north star, and don’t change them often.
How can a PM in the B2B space work with data to arrive at decisions? In B2C you have the facility to do AB testing but in B2B our clients are so few and big ticketed. How do you prioritise features?
Working with data doesn’t necessarily mean A/B testing. It’s indeed harder when you don’t have a lot of customers, but you can still release gradually, interview customers after the release, send CSAT/NPS questionnaires etc.We certainly prioritise feature requests based on all parameters you’ve mentioned. The key is working very closely with the customer-facing teams (sales, CSM, support) - ensuring they document everything in SalesForce, and then running processes on top of that (such as: which deals are pending this feature and what are their size?).
How do you ensure feature adoption among high ticket B2B clients? Your top ways to do so?
We have a KPI for our customer success managers to drive adoption for their higher tiers customers, for specific features. Our CSMs have great relationships with their customers, so they can help to push adoption for the ones we think will get high value out of it.
How do you retain your clients at Appsflyer? Any strategies you can share here?
We retain them by optimising all the time the infinite loop of:
Customer feedback --> great relationship with them --> adjust the product accordingly --> listen to them again.This enables us to ensure that we always put our resources on the right things for our customers.Now, don’t get me wrong, it’s not that easy.
We have thousands of customers, different regions, different segments, different needs.So, it doesn’t mean that we do all what the customers ask.
But - we have a process to digest all these inputs, understand what they expect us to do, focus on their WHY and not WHAT (i.e.: they might ask for a specific feature, but we try to understand why they’ve asked for it, and maybe deliver it differently), and then prioritise accordingly. Because eventually - there is only one way to retain your customers - keep delivering to them value via the product.
Want to join the next conversation? We’ll be having another Product Chat soon, get your invite to our Slack community to get all the details. See you inside.