Cohort Analysis for TTV: Tracking Speed Improvements Over Time

You shipped a new onboarding flow. The design is slick. The copy is punchy. The team is celebrating. But did it actually work?

The only way to know is Cohort Analysis. You cannot just look at "Global TTV" because that mixes old users (who saw the old UI) with new users. You need to isolate time.

Visualizing the Race

Imagine TTV as a race.

Cohort A: Users who signed up in January.
Cohort B: Users who signed up in February (after your redesign).

To compare them, we plot a Cumulative Distribution Function (CDF).

X-Axis: Hours/Days since signup (Time).
Y-Axis: % of Cohort that Activated.

This generates a curve that starts at 0% and climbs toward 100% (or whatever your max conversion rate is).

The "Up and Left" Shift

When you ship an improvement, you want the curve to move Up and to the Left.

1. The Shift Left (Speed)

The curve rises steeper and earlier.

Jan Cohort: Reached 50% activation in 4 days.
Feb Cohort: Reached 50% activation in 2 days. Conclusion: You removed friction. The product is faster.

2. The Shift Up (Conversion)

The curve reaches a higher ceiling.

Jan Cohort: Flattened out at 30% total activation.
Feb Cohort: Flattened out at 40% total activation. Conclusion: You rescued users who would have otherwise churned.

Warning: The "Slow Down" Trap

Sometimes, a Product Manager adds "Education" to the onboarding. A mandatory 5-step tutorial. This often causes a Shift Right (Slower Speed) but a Shift Up (Higher Conversion).

It takes longer to finish (bad TTV), but more people eventually understand the product (good Retention).

This is a trade-off.

If you shift Right by 10 minutes but Up by 20%, take the deal.
If you shift Right by 3 days but Up by 1%, revert the change. You are killing momentum for marginal gain.

Finding the "Golden Window"

Cohort analysis also reveals your "Time to Death." Look at where the curve goes flat. If your curve goes completely flat at Day 7, it means virtually nobody activates after their first week.

This is your Golden Window. If a user is on Day 6 and hasn't activated, you have 24 hours to save them. Send the email. Make the call. Once they cross the Day 7 threshold, they are statistically dead.