It tested 118 trackers using a treadmill to complete the distance of a marathon - 26.2 miles (42km).
Garmin Vivosmart 4, underestimated the distance by 10.8 miles – meaning the researcher actually ran 37 miles. Garmin blamed the poor showing on the tracker not having GPS.
It described the Vivosmart 4 as an “all-round smart fitness tracker” and suggested that marathon runners use its Forerunner range which is GPS-enabled.
The Apple Watch series 1 was the most accurate, over-estimating the distance by ond percent, while the series 3 overestimated by 13 percent - stating that the runner had completed the marathon distance after 22.8 miles.
Natalie Hitchins, head of home product and services at Which? said: “Our tests have found a number of models from big-name brands that can’t be trusted when it comes to measuring distance, so before you buy, make sure you do your research to find a model that you can rely on.”
Other results for the number of miles reached before the tracker recorded the official marathon distance included:
- Samsung Gear S2 – 36.2 miles
- Xiaomi Amazfit Bip – 34 miles
- Huawei Watch 2 Sport – 18.9 miles
A Huawei spokesman told the BBC “individual runner variances” could have affected the test results.
“With regards to running indoors, as this particular test was carried out on a treadmill. The algorithm of Huawei Watch 2 Sport calculates the user’s stride length from the acceleration sensor data while running at different speeds."
The tests were conducted in January 2019 and researchers at Aberystwyth University found that all the trackers they tested overestimated the number of calories burned during activity.
Gavin Taitt is a regular middle-distance runner from Earlston, in the Scottish Borders, who also coaches others. He said he and his group use a combination of Garmin Forerunner watches and the social fitness network Strava to measure and share results.
“The watches are quite expensive but have good feedback”, he said.
Another expert agreed that the calibrated treadmill test was not the best method because all the devices would have had to rely on step-counting algorithms rather than GPS (for those which had it) to calculate distance.
Dr Dale Esliger, senior lecturer in physical activity and health at Loughborough University, said that this was a real shame as a real world (on-road) test would have been more useful for consumers.