Eric Ries is not very fond of what he calls "vanity metrics"-- numbers that show things going up or down (like hits on a website or number of new customers per month) because they don’t actually measure what was done to cause the change.
This leads to people making the following conclusions:
In my experience, when the numbers go up, people think the improvement was caused by their actions, by whatever they were working on at the time. That’s why it’s common to have a meeting in which marketing thinks the numbers went up because of a new PR or marketing effort and engineering thinks the better numbers are the result of the new features it added.
Unfortunately, when the numbers go down, it results in a very different reaction: now it’s somebody else’s fault. Thus, most team members or departments live in a world where their department is constantly making things better, only to have their hard work sabotaged by other departments that just don’t get it.
<p class='p2'>He suggests two solutions to this problem, both of which need to be implemented. First, people need to work in cross-functional teams, not traditional departments like marketing or engineering. Second, metrics need to actually give real information about what caused the change. Primarily, he suggests using A/B testing on the product (giving different versions with and without a new change to different groups of customers).
For the entrepreneur working with a small team of a few (or even one!) the first solution is irrelevant, but the second could prove invaluable.
In other news, I made a new comic. Go read it!
I’m reading Eric Ries’ The Lean Startup and I’m surprised how many good lessons there are inside, not just for entrepreneurs, but for basically every company.
Take this quote:
A few years ago, a team that sells products to large media companies invited me to help them as a consultant because they were concerned that their engineers were not working hard enough. However, the fault was not in the engineers, it was in process the whole company was using to make decisions. They had customers but did not know them very well. They were deluged by feature requests from customers, the internal sales team, and the business leadership. Every new insight became an emergency that had to be addressed immediately. As a result, long-term projects were hampered by constant interruptions. Even worse, the team had no clear sense of whether any of the changes they were making mattered to customers. Despite the constant tuning and tweaking, the business results were consistently mediocre.
<p class='p2'>Does that sound like your company? It sounds exactly like my former company! That’s basically all we did for the five years I was there. Management always blamed the engineers and kept meddling and changing procedures, seemingly at random, when the problems lay elsewhere.
Definitely something to think about.
I'm a writer and occasional programmer. I write science fiction stories and novels.
I also write technology articles for Ars Technica.
I'm the creator of newLISP on Rockets, a web development framework and blog application.