An Introduction to Metrics Driven Development — build features that change real numbers.

How many times have developers picked up tickets because they were in the backlog? Delivered them and never asked why? Metrics driven development tries to put an end to this. It focuses the teams and business on building features that aim to have high impact on what matters i.e. logins, revenue, or sign ups. You can read more about this in a blog I posted back in February. I want to talk about where I used this methodology in a real world example, and how this could change the way your organisation works forever.

Recently at Roadie, a Dundee based startup helping learner drivers find the right driving instructor, we have been trying to increase the number of our booking requests. The process is lightweight and something other startups can pick up easily.

We started by looking what we were trying to achieve; more booking requests. Then we looked at our metrics for search results and seen we have 60% drop off. Right now our booking conversion is around 4% so in theory if we managed to lower this drop off we would see more booking requests. A leap of faith assumption but we went with it.

So, the objective was simple: Reduce bounce rate on search by 5%. This is our criteria for success

Next, we had a brainstorming session where came up with possible solutions, all varying degrees of work.

We discussed what we thought were worth taking on and that could have high impact. As a team we voted and began the work.

We released three changes on that list; higher resolution images, banner for popular instructors and a nicer heading.

Then we released those changes.

The process of gathering metrics, deciding what to do, then implementing the changes took less than 1 hour.

When the (cross-functional) team has a clear goal i.e. increase conversion and the team is left to their own devices to solve the problem, development becomes a lot faster. Ownership aids delivery.

View Driving Lessons in Dundee results.

The real test comes from looking back at the analytics. Did we succeed? Is the bounce rate lower?

Ten days after the change, I think the metrics speak for themselves. 60.26% to 46.38% is definite win. This is something we will continue to measure in the coming weeks. However, we did make an assumption that decreasing the search bounce rate would increase the number of requests overall. It is too early to tell if this change has a significant impact on booking requests, however by watching those metrics, and having a predefined criteria for success the team knows that the deliverables met the desired outcome; to lower the bounce rate by 5%.

By using Metrics Driven Development, the team were able to quickly deliver features as the nature of the problem was clear, and by writing out the criteria for success beforehand it is easy to judge wither they failed or not. Using these steps could drastically speed up delivery. Focussing on metrics daily will help startups prioritise it's backlog, give real clarity of success and failure. Try it, putting your analytics first will change the way you look at startups and software.

Michael Carr

Founder of Roadie

Twitter

Blog