Eric Ries is not very fond of what he calls "vanity metrics"-- numbers that show things going up or down (like hits on a website or number of new customers per month) because they don’t actually measure what was done to cause the change.
This leads to people making the following conclusions:
In my experience, when the numbers go up, people think the improvement was caused by their actions, by whatever they were working on at the time. That’s why it’s common to have a meeting in which marketing thinks the numbers went up because of a new PR or marketing effort and engineering thinks the better numbers are the result of the new features it added.
Unfortunately, when the numbers go down, it results in a very different reaction: now it’s somebody else’s fault. Thus, most team members or departments live in a world where their department is constantly making things better, only to have their hard work sabotaged by other departments that just don’t get it.
<p class='p2'>He suggests two solutions to this problem, both of which need to be implemented. First, people need to work in cross-functional teams, not traditional departments like marketing or engineering. Second, metrics need to actually give real information about what caused the change. Primarily, he suggests using A/B testing on the product (giving different versions with and without a new change to different groups of customers).
For the entrepreneur working with a small team of a few (or even one!) the first solution is irrelevant, but the second could prove invaluable.
In other news, I made a new comic. Go read it!