As we head to the end of our 4rd month of beta usage I’m more than a bit disappointed at how accurate the bike prediction feature has been working. If you recall, we added a second algorithm in January in an attempt to improve. While this helped a bit a new wrinkle has developed given how fast people’s fitness is changing and right now we’re averaging ~75% accuracy across all users. This isn’t enough to feel good about exiting beta and scaling up our user base.
To fix this we’re testing two new algorithms (Gear3 and Gear4) now that use more history than the previous approach where we tried to sample the past 8 rides on each bike. Early results looking decent for total accuracy sitting at about 95% after 3 days.
In addition to this test we’re now creating two new algorithms that will separate predictions for bikes with power from those without to avoid using averages to fill in missing data and to use IF, TSS and NP in the training and prediction. We’ll change the notification logic next week once we pin down what working best after a busy weekend of rides and races.
If both of these approaches don’t work we’ll shelve the notifications until such time as we can get the sensor device ids from Garmin making the prediction much easier and 100% accurate for bikes with a speed, cadence or power sensor.
Please stay tuned and take at look at the Gear3 and Gear4 predictions on any ride that gets flagged for a Gear Alert over the next few days. You can click on the activity name link in these notices to see a list of the predictions as well as thresholds and segment watts/kg, etc…