Back to Basics

Normally I focus on the product definition aspects of creating successful products. My reasoning is simple: it doesn’t matter how great a job you do in building your product if it isn’t the right product. That’s really the role of the product manager; to define the right product at the right time. However, there is one little detail that too many product teams seem to miss, even when they define an otherwise excellent product. That is, the product has to actually work.

I wish that by now, after roughly 30 years of “modern” software development, we wouldn’t have to keep talking about this, but in truth, while there are many notable exceptions in well-run companies, the state of software quality in general is still awful. During the boom times of the nineties, when Internet sites were developed and launched in a just months, with little concern for quality (“just call it a beta!”), this was an especially severe problem. Things seemed to get better for a while, but now I fear we’re on a downswing again.

I’m pointed to new products and sites almost daily, and I get so frustrated when the quality issues are so bad that they completely obscure what great innovations or contributions may lay behind them.

Largely this is due to the general state of software QA (quality assurance) in our industry. There is a huge chasm between best practices and common practices. For engineering teams that know how to design and build for automated testing, and know how to measure quality and ensure against regressions, the tools and practices are well established and the results are real.

The problem is that far too many teams still are doing little better than was done a generation ago. Testing is all too often still manual, the priority of the testing is so low that it’s the first thing to get cut in the rush to launch, the QA organization is understaffed and especially undertrained. The logic goes like this: this site is for teenagers, so lets hire some teenagers to run through the site and find bugs. I’m all for giving teenagers jobs, but a) this is no substitute for the knowledge of how to systematically and rigorously ensure that a release of software meets the specifications; and b) manual testing quickly breaks down when you are turning out new builds faster than you can manually test the product (which today is almost always the case).

Typically, the product manager worries about the spec, and the engineering manager worries about delivering a quality implementation of that spec. However, the product manager is ultimately responsible for the success of the product, and if the quality is so poor that the user experience is severely impacted, then it’s essentially the same problem as if the features are the wrong ones.

This doesn’t mean that the product manager needs to micro-manage QA, but it does mean that the product manager needs to take a very active role in defining release criteria, including and especially software quality. There’s nothing wrong with the product manager asking his engineering or QA manager colleagues questions about the QA staffing, funding, and testing process.

The irony is that too many engineering teams still believe that modern software testing practices cost more money or take more time than doing it manually. If done properly, you’ll find significantly faster time to market, more productive QA staff (with greater job satisfaction), and most importantly, a much better user experience.

Most types of software don’t need to be perfect, but remember that it’s all about the user experience. If the product is a mess, with bugs constantly getting in the user’s way, then your product will suffer. If this is the situation you face with your product, you’ll need to argue for fixing the problem at its core, investing in QA staff, tools and process, and prepare to trade off some new features you want so that your engineering team can have the time to clean up the product.

(Can you tell that I just wasted several hours doing battle with a product from a company that clearly has no clue? 😉

Let’s hope that someday soon a note like this will no longer be necessary.