Wednesday, June 18, 2014

Measuring Product Development Productivity

I've recently been involved in a number of discussions about how to measure product development productivity, and I thought this was a great topic worth sharing. I've wrestled with this topic for much of my career, and have developed some pretty strong opinions on the subject.

First, I am not in favor of what I think of as "easily quantifiable but hollow metrics" such as lines of code, number of work items, number of drawings, number of ECOs, etc. Why? Because these measure effort and not results, and they may have little relationship to the success of the business. Remember, the whole purpose of product development is to help a business accomplish its goals. There are teams that can crank out tons of code that does not help the business in the least. Inversely, some teams may produce things that really matter to the business with a much smaller code base. Which team would you rather have?

Therefore, product development metrics need to be related to goals that are important to the business. These goals probably have something to do with increasing profits, revenue, or market share by introducing new products and features, but there are many other possibilities. Coming up with such metrics is admittedly easier said than done.
Here are two metrics that I have evolved into to over time:

1) Revenue from new products introduced in the past three years. This is pretty easy to measure and can be tailored to for any business. Perhaps it's 1 year instead of 3. Perhaps revenue is replaced by the percent of end-users who use new software features released in the past year. You get the idea.

2) ROI from development effort. This is much more complicated. Actually, the "I" part is pretty easy, but the "R" part can be tough; it will require debate and agreement among various stakeholders. This is also a long-term metric, as large projects may take years to produce any ROI at all.

I'm interested in your thoughts and comments. What metrics do you use?