The effect of software and hardware version on Apple Watch activity measurement: A secondary analysis of the COVFIT retrospective cohort study

The objective of this study was to estimate the impact of software and hardware version on Apple Watch activity measurement using data from the COVFIT retrospective cohort study. We estimated the impact of software and hardware versions on activity measurement by comparing daily active calories and daily exercise minutes in the 7 days before and 7 days after upgrading from watchOS 5 to 6, 6 to 7, 7 to 8, 8 to 9 or between two hardware versions. For each transition, we fit mixed effect negative binomial regression models to estimate the effect of the upgrade on daily (a) exercise minutes and (b) active calories, overall and stratified by sex, with and without adjusting for weekday. We also calculated and plotted the mean person-level change in average activity levels between the two weeks. As a control, we repeated the entire analysis comparing activity data two weeks before vs. one week before each upgrade. 253 participants contributed data about at least one transition (software = 250, hardware = 74). Hardware upgrades were not associated with either outcome; however, some software upgrades were. Upgrading from watchOS 7 to 8 was associated with a large, statistically significant increase in daily exercise minutes (unadjusted rate ratio (RR) = 1.13, 95% CI: 1.06, 1.20). WatchOS 6 to 7 and 8 to 9 transitions were associated with statistically significant decreases in daily exercise minutes (6 to 7: unadjusted RR = 0.92, 95% CI: 0.86, 0.99; 8 to 9: unadjusted RR = 0.91, 95% CI: 0.86, 0.96) and active calories (6 to 7: RR = 0.96, 95% CI: 0.94, 0.99); 8 to 9: RR = 0.97, 95% CI: 0.94, 0.99). There was no significant change in either outcome during in the two-week control period for most transitions. Differences in software version over time or between people may confound physical activity analyses using Apple Watch data.Author summary: Researchers are increasingly using data from participant’s personal wearable devices, like Apple Watch, to study physical activity over time and between people. These people may all be using different hardware and/or software versions, which may impact how physical activity is measured. If this is the case, comparisons of physical activity levels over time, or between people, may represent the combined effects of differences in behaviour, as well as differences in how that behaviour is being measured. Using data collected for the COVFIT study, we compared daily active calories and daily exercise minutes in the week before and the week after our participants upgraded between two major software versions (e.g., watchOS 5 to watchOS 6), or switched to a different hardware version. We found large changes in daily exercise minutes and active calories in the week after compared to the week before some, but not all, major software upgrades. In future, researchers should measure and account for hardware and software version when either may differ between participants and/or over time.