Sam Clemens, InsightSquared Co-Founder, on Product Management Processes

Sam Clemens is co-founder and Chief Product Officer of InsightSquared, a Boston-based startup. Sam has been a frequent guest instructor in the Product Management 101 elective at Harvard Business School. My students and I have learned a lot about product leadership over the years from Sam. His insights on product management processes are nicely captured in this interview with Mike Fishbein for the podcast, "This is Product Management." Mike and Sam have graciously allowed me to republish this lightly edited transcript of their original interview.

*  *  *

“This is Product Management” host Mike Fishbein: On this episode, I’ll be speaking with Samuel Clemens, co-founder and chief product officer at InsightSquared. A few weeks ago we had Drift founder/CEO David Cancel on the show to talk about customer-driven product teams and implementing a fluid product management environment. David previously was the head of product at Hubspot, but before him, Samuel Clemens was Hubspot’s product lead. The two share many beliefs, but have different views regarding the purpose and implementation of process in product management.

Clemens, 1:19: My topic is “Active Process is Good Product Management.” I’m picking something potentially controversial here, since I know a lot of your previous guests have been advocates against heavy process in product management. And truth be told, I’m a process-light guy. Most of the time, I believe that process should be introduced after having a bunch of smart, driven people figure out your first-ever iterations. Indeed, you should stay away from them with any kind of process. So the question becomes: What do you do after those early iterations?

If you think back over the years of building things at your company, how many customer visits have you done? How many mock-ups? How many times have you pushed out a release? How many times have you triaged a bug? The numbers are probably in the hundreds, if not thousands.

If you do something that many times, you will develop a process. The question is whether you’re an active or passive participant in developing that process. There are some advantages to passive — reasons you’d want to let process develop organically. But if you choose to be an active participant, you get the opportunity to reinforce attributes that you’re looking for.

2:35, Host: How did you get into tech in the first place and what has shaped your perspectives?

I’m the child of two photographers. One was an artistic, creative photographer; the other was very mathematical and logical. I think having two ways of thinking about a problem—creative and logical; ideation and testing—is useful to any product manager. Cycling back and forth is often key to problem solving.

As an undergrad, I was an applied math major, training to become what we now call a data scientist. This is really just a fancy label for ‘here’s a toolbox for problem-solving.’ I chose to apply the tools to product management.

After college, I got into management consulting, but found it unsatisfying. I’m a builder. So right away, I went into the first of five start-ups — a freelance marketplace for services, which eventually merged with O-Desk. My second startup was a word-of-mouth media firm that was acquired by a British retailer. The third was a 3D modeling company called Models for Mars. At my fourth startup, I ran product at Hubspot for a couple of years, and then left to co-found a company called InsightSquared with two very good friends. InsightSquared is now five-and-a-half years old.

We’re a B2B software company that does sales analytics and business intelligence for the non-Fortune 500. We are middle-growth stage with about 160 people. On the R&D side, including engineering, product and design, we probably have 40 or 45 employees. Besides leading product at InsightSquared, I’m an entrepreneur-in-residence at Harvard Business School, and I do frequent speaking engagements at HBS and MIT on product management, design, and product marketing.

5:10, Host: What have been the key lessons you learned about creating process?

I often speak with entrepreneurs and managers about how to implement agile development and related iterative product management practices. They don’t get hung up on the theory—they’re bought into that—but rather, they struggle with how to implement the theory to get a smooth-running machine.

Over time I’ve identified seven actions that help teams implement good product management processes.

The first action is at the core of good product management: You have to know your customer very well and in particular, gain this knowledge through on-premises customer visits.

I stress the on-premises part. You need to be at the customer location to see the animal in their native habitat. If you’re on a phone call with them showing a new mock-up via a screenshare, there are many things that you’re not seeing. Most obviously, you’re not seeing their facial expressions — whether they’re confused about something. If you’re careful, you may be able to pick up hesitation in their vocal tone over the phone. You’re also not seeing the white board in their office, covered with their brainstorming notes, which might give you a sense for what really bothers them. You’re not seeing the way that they relate to their peers. You’re missing all of that context, and when it comes to really building an insightful product, that context matters.

You can’t just do surveys. You can’t just do phone calls. You have to get out of the building and visit customers. So, one of the processes for my product team is mandatory, once-a-month, on-premises customer visits for all of my PMs.

The second item on my list is tuning whatever processes you’ve chosen for your engineering teams. Common choices include kanban and scrum, but whatever you select, the key is to really tune the process so that iterations flow easily and the process becomes an enabler — and not a burden for your teams.

A common question with scrum is cycle frequency: Should it be seven days, two weeks, or four weeks? The answer matters. We ran one-week cycles for the first year at InsightSquared. This eventually resulted in a reactive approach and burnt-out PMs.

One-week cycles resulted in a rushed product development experience. We often didn’t have enough time to plan out more complex features — the type you might build in your second year. So that’s why, starting in year two and continuing today, we’ve been running two-week cycles. They still yield a sense of urgency, but with less burn out. Four-week cycles are an option, too, but I find that them to be better suited to a more mature company where projects are longer and require more planning.

Other choices include how you set up estimation so that it’s easy, not painful. How do you prioritize and triage bugs? This makes a big difference to your engineering team and to your customer success team.

How do you demo software that you’ve built? We do a once-a-month internal demo to the entire company. We make sure we describe the business value of stuff that we’ve built. The audience includes our customer success teams and our sales teams; they’ve taken an hour from their busy days, so we had better show them something impressive.

To summarize, whatever process you choose will have many levers. It’s important to study those levers and really tune them, so when you’re doing two hundred reps of the process, it’s smooth and it's an enabler rather than a burden for your teams.

10:10 – The third thing on my list is specifications. Frequently, entrepreneurs will ask me: How do you spec out what you will build? I actually have a “kill on sight” order on specs in my company. I won’t allow them. I think the most dangerous thing about a spec is that someone might actually build it. The problem is that a spec represents a point-in-time view of what someone once believed about the product — maybe three, six, or nine months ago. Today, that view is almost certainly out of date.

You’ve met a bunch of customers since then. You’ve had a bunch of customer support cases. None of that is reflected in the spec. Also, the product has progressed. New features have been built, introducing lots of new interdependencies that the spec can’t possibly consider. And finally, no matter how detailed the spec is, it can never anticipate all the questions that might come up. I’ve seen such problems with specs over and over. My first two startups used waterfall processes, with 14-page Word docs and all that.

A spec can give you a false illusion of completeness. Rather than harbor that illusion, I say, ‘Kill the spec.’ Replace it with a set of conversations between the product manager and the engineers. When an engineer starts a story, I prefer that they turn to the product manager, and they say, “Hey, what are we doing with this feature?” The PM says “Ah, glad you asked. So here’s why we’re doing this, here’s what we’re trying to do, here’s the problem.” Now the engineer has context. Then engineer says, “So, there are a couple of ways we can do this –approach A or B,” and the PM says “Considering the pros and cons of each, I don’t think the pros for approach B would be relevant for our customer, so let’s go with A.”

The engineer comes back the next day, and says, “Hey, you’ll never guess what happened – when the count for variable X is zero, the whole thing breaks.” And the PM says, “Wow, I never thought of that. Of the ways you say we could handle that edge case, here’s the one that will pose the fewest problems for our customers.”  That kind of exchange that would never be in a traditional PRD spec, yet it’s critical. An ongoing conversation between the PM and engineer is the most reliable way I know of to build quality product.

12:40 – The fourth thing on my list is release frequency. How often do you release software into production? Four or five years ago, the answer might be once a month, or maybe every two weeks at the end of a sprint. Essentially, this means you are in batch mode.

Instead of batching, you can set up your engineering process as a flow in which you’re releasing new features and product improvements as they are ready, multiple times per day — dozens or even hundreds of times per day. The benefits from shifting from batch to flow are immense. Boosting your release frequency enables a more iterative approach to development. You can develop a module, have it gated and hidden, push it out quickly, get customer feedback on it, and go back and keep iterating to develop the next module.

For the engineering team, there are other benefits. Your most senior engineers don’t have to stay up until 2:00 am every two weeks, trying to push out a mass of software. For your customer success team, not pushing out a big chunk of code means that fewer bugs will crop up due to interdependencies. Fixing such bugs usually requires roll-backs and other complicated maneuvers. Instead, when each release involves a much smaller chunk of code, if a bug happens, it’s easier to identify where that bug came from.

14:37 – The fifth item on my list is to have everyone on the product team coding, including both product managers and designers. There are some steps that an engineering team needs to take to enable PMs and their designers to code, so it takes some effort to do this. I believe that this effort is worthwhile, because one of the highest impact things you can change in a product is customer-facing copy. A typical process for tweaking product copy might involve a product manager saying, “Every time I’m with a customer, they find this page confusing. I believe that once they read the headline, they have the wrong frame of mind for everything else on the screen. I want to change that headline.”

They might go to an engineer on their team, and say, “Could you change that headline?” The engineer will say, “Sure, no problem!” Then the PM hits reload, and the new, longer headline is wrapping in a weird way. She says, “Could you change it again, with this shorter headline?” The engineer changes it, but now it’s not clear again, and so they go back and forth. After a couple of rounds, the engineer feels like a typist, and the PM feels like an ass for making the engineer feel that way. Next time, the PM chooses to avoid that situation — to preserve team harmony, she’ll choose to live with an inferior product.

The friction to go from good to great is just too high. You can decrease that friction if you enable your PMs and your designers to code, giving them direct control over the last layer of detail. You get a productivity gain, a motivational gain, and a quality gain. For designers, this can be even more impactful. When you have designers who can code instead of handing off mocks, they are less likely to suggest UX that is not implementable, and more likely to suggest UX that is informed by the kind of cool things that you can do in code these days.

17:55 – The sixth thing on my list is testing. This overlaps with the question of how to run betas. I view testing as a many-layered onion. When you move out from the onion’s center, you get tests with increasing fidelity, but also with increasing cost. At the onion’s center, you have the cheapest possible test: simply asking a nearby engineering team member: “Hey, can you look at this screen and tell me what you think?” The cost for that is a few minutes. The fidelity may be okay, but it is potentially questionable if your colleague doesn't have a true customer perspective on the item in question.

Slightly more expensive is going to your customer success people, who are a proxy for the customer. This takes maybe 5 minutes, and gets you higher fidelity.
So then you can go outside to a beta customer, with whom you have close relationships, increasing cost and fidelity. The cost is say, 2 hours. The benefit is that these are real customers, but a drawback is they are beta customers, so they’re not truly representative of the entire set. They’re too bug-tolerant and they like new software. So, you can go out another layer, and do a gated release to a subset of one-tenth of your entire customer base. The cost to set up now is several hours, because you have to manage a release.
So on and on you go, until the improvement is released to your entire base. Even then, you are testing. You’re watching the improvement being used and then tweaking and iterating it. Whenever you’re releasing something, you should ask yourself: What uncertainty do we face with this software? And for that type of uncertainty, what is the appropriate level of testing?

20:56 – The last item on my list is creating a roadmap. I get lots of questions about this from entrepreneurs and product managers at other companies. At InsightSquared, we use a four-quarter roadmap, showing what we plan to build in each of the next four quarters. Once a month, I revise the roadmap. It’s fully transparent internally – available to the entire company. I ask salespeople to tape it on their desks. I do regular lunch conversations open to anyone in the company about items on the roadmap, and explain the trade-offs behind it.

Levels of certainty decline as the quarters progress. For the current quarter, maybe there’s an 80% chance that nothing will change by the time we finish the quarter. But if you go four quarters out, there might be a 20% chance. We may change how we’re selling things, how we’re marketing, our understanding of customer needs, and so forth. We make sure that consumers of the road map within the company know that it’s not all set in stone. They get a feel for which pieces are more fluid, and which are more reliable.

22:42, Host: In Samuel’s list of product management processes, he repeatedly emphasizes autonomous product teams and a customer-centric perspective. It seems like he and product leaders like David Cancel see eye-to-eye on this. Is that the case?

23:00 – I’m not against customer-driven as a concept, I’m against customer-driven as it’s interpreted and implemented by many product management organizations. Many product leaders say, ‘Let’s be customer driven; let’s have a team dedicated to improving engagement, to reduce the number of customer support tickets. We will look at all the tickets that have come over the last X months, then group them and force rank them according to which customers have found most urgent. Then we’ll burn down that list. Oh, and let’s put a metric on the team, so that they have to reduce the list from 20 down to 5. That’s our customer-driven approach.’

I have a major problem with that. When you interpret customer-driven that way, you often miss the bigger picture. You end up with incremental levels of product improvement. Fixing issues that your customers are surfacing for you will indeed give you an improvement on each individual issue, if you’re doing it right. But each individual issue is often part of a bigger problem with flow in the product, so you really you shouldn’t be fixing the 5 individual things. You should be rewriting the entire flow. 

Or, imagine you have five different and very big problems with that product; it’s a mess. In a ‘customer-driven’ process, you will never hear a customer say, “That product, you really need to kill it.” And yet that’s something that may need to do as a product manager: decide to not support a product, so that you can focus more resources on something new.

Instead, I advocate for product managers taking a much more active role in guiding the development of their product. Customer needs are indeed an input, in fact, they are probably the most important input, but they are not the only driver of what you’re developing. There are other inputs, like competition and corporate strategy, the cost of things and sequencing. All of these things need to get mixed into an active process of road mapping and planning, instead of just saying ‘Hey, we’re customer driven. We’re gonna let the customer drive this bus’. As a product manager, you need to drive the bus.

I do think the foundation is having a good understanding of the customer. I like hiring product managers from the customer success team, because they already understand customer needs at a very deep level

-->