Save Time And Money From The Coin Toss Accuracy Of Delivery Time Estimation Sessions

Photo by charlesdeluvio on Unsplash

Some Background

Team Setup

  • Four squads
  • Each squad had at least two projects running in parallel, ie. at least eight deliverables in any given time
  • The set up between each team was very similar. Where the majority of team members remained unchanged for about a year, so the velocities were in a reasonably well-calibrated range

The problems

  1. Between closing new deals with clients, new product ideas, new C-level executives, our 1-year roadmap got shuffled around on a monthly basis
  2. ROI played a big part in our roadmap prioritization. To get ROI, we needed to estimate the cost, i.e. (development time) x (avg. hourly rate of the squad)
  3. Between switching between deliverables and fixing the issues from the legacy systems, we ended up late on the delivery way too often (see the data below). The actual time spent on a feature turned out was a million miles away from the initial estimation.
  • In 50% of tickets the time spent was within 20% over or under the estimate
  • about 25% was between no work done (-100%) or significantly under the estimate (-100% → -80%)
  • The other 25% included that long-tail estimation

Cost Of Estimation Meetings

  1. Typically we spent two hours in an estimation meeting and sometimes it took more than two meetings to estimate one deliverable
  2. Members who were typically part of the estimation meetings: head of dev, head of product, team lead, product manager, program manager, CTO (sometimes). Where the total annual salary of the members mentioned above is based on the avg. IT salary survey in 2020: that’s £580k per year, that’s £285 per hour before tax
  3. We normally ended up changing half of the deliverables in the coming six months, some projects lasted longer, others shorter (one monthish). There were normally eight projects running in parallel. Let’s say we have a three month project and another one month project for all squads at all times. That’s 12 small projects and four big projects, per year, per squad

Task

  • Delivery velocity
  • Size of each ticket
  • Correlation between the ticket size and the actual time spent
  • An estimated amount of how many tickets for a project. It’d be an educated guess from the BAs and tech leads

Raw data from JIRA

  • Issue key: this was just the internal key we used, masked in the screenshot, as it is an index for me and not relevant to the result
  • Issue Type: story, bug, hotfix
  • Sum Original Estimate (unit min)
  • Sum Time Spent (unit min)

Clean up and categorize data

Generate Data

  • Time difference between estimation and actual time spent (per ticket) = Sum original estimate time — Sum time pent
  • Prediction accuracy (per ticket) = (Sum original estimate time — Sum time pent) / Sum original estimate time

Time estimated accuracy distribution- individual ticket level

  • 50% of tickets delivery time spent were within 20% over or under the original estimation
  • about 25% of the work done was under the estimate
  • And another 25% was part of that long-tail in the diagram

Time spent distribution of each ticket type

Result

Building model

Result Interpretation

Premise In This Propose Process

Conclusion

--

--

--

Ex-(might still am)-Entrepreneur/ Product (IT)/ Scientist/ Writer/ Gamer

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Calendar Reviews

Why do we exist? Opinion of a Product Manager!!

Do Product Owners Know What VALUE Really Is?

Why Are Most Backlog Refinement Sessions Dreadful

A painting of someone frustrated

Five things you need to know about the product manager in tech before making your career decision

The Biggest Red Flags to Watch Before Accepting a Job as a Product Owner

A man wearing a hat in front of a laptop

Building products for consumers, businesses and internal stakeholders

Product Management — Guesstimates | The number of pizzas Dominos sell in India in a day

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
J Hu

J Hu

Ex-(might still am)-Entrepreneur/ Product (IT)/ Scientist/ Writer/ Gamer

More from Medium

Why I decided to leave a management position

Two people working in a project with two computers, a paper pile and a pencil. Only their arms are visible.

Automation and big data. The clinical trials of tomorrow

Is your code too complicated?

Fauna: Embed vs Collection — Why you should choose Embed over Collection