“CATCH-IT Reports” are Critically Appraised Topics in Communication, Health Informatics, and Technology, discussing recently published ehealth research. We hope these reports will draw attention to important work published in journals, provide a platform for discussion around results and methodological issues in eHealth research, and help to develop a framework for evidence-based eHealth. CATCH-IT Reports arise from “journal club” - like sessions founded in February 2003 by Gunther Eysenbach.

Friday, December 4, 2009

(Final) CATCH-IT Report: Effectiveness of Active-Online, An Individually Tailored Physical Activity Intervention, in a Real-life Setting: RCT

Wanner M., Martin-Diener E., Braun-Fahrländer C., Bauer G., Martin B.W. (2009). Effectiveness of Active-Online, an Individually Tailored Physical Activity Intervention, in a Real-Life Setting: Randomized Controlled Trial. J Med Internet Res, 11 (3): e23.


Original Post - Abstract Only - Full Text - Slideshow - Draft Report

Introduction

This report is a summary and analysis of the study conducted by Wanner et al. (2009)entitled Effectiveness of Active-Online, An Individually Tailored Physical Activity Intervention, in a Real-life Setting: Randomized Controlled Trial. The focus of the study was a web-based physical activity intervention called Active-Online which provides users with customized advice on increasing their physical activity levels. The study compared the effectiveness of Active-Online to a non-tailored website in changing physical activity behaviour when delivered in a real-life setting.

The authors found that a tailored web-based intervention is not more effective than a non-tailored website when deployed in an uncontrolled setting.


Objective

The study aimed to answer the following three questions:

  1. What is the effectiveness of Active-Online, compared to a non-tailored website, in increasing self-reported and objectively measured physical activity levels in the general population when delivered in a real-life setting?
  2. Do participants of the randomized study differ from spontaneous users of Active-Online, and how does effectiveness differ among these groups?
  3. What is the impact of frequency and duration of use of Active-Online on changes in physical activity behaviour?


Methods

A randomized controlled trial (RCT) was used to answer the questions posed by the authors. Three groups of participants were observed during the trial—the control group (CG), the intervention group (IG), and the spontaneous users (SU) group. CG and IG participants were recruited through media advertisements and randomized into their respective groups using a computer-based random number generator. Participants in the SU group were recruited directly from the Active-Online website by redirecting them to the study website if they chose to participate in the study. A sub-group of participants volunteered to wear an accelerometer so that their physical activity levels could be objectively measured during the study.

Participants in IG and SU visited the Active-Online website and answered diagnostic questions about their physical activity behaviour to receive customized feedback on how to improve them. Those in CG visited a static website to receive generic tips on physical activity and health.

All groups were followed up via email at 6 weeks (FU1), 6 months (FU2) and 13 months (FU3) after the baseline assessment. There was no face-to-face component in the study.


Measures

Three types of data were collected in the study:

  1. Self-reported subjective measurements of physical activity levels obtained through follow-up questionnaires presented to all groups
  2. Objective measurements obtained from accelerometers worn by the subgroup
  3. Frequency and duration of visits to Active-Online obtained from the Active-Online user database which recorded each log-in to the website


Results

There was a significant increase in subjectively measured levels of physical activity among all groups from baseline to FU3, but no significant differences between randomized groups. However, the differences were more pronounced in the SU group. As for the objective measurements of physical activity obtained from accelerometer readings, there was no increase from baseline values in any of the groups. Measurements of frequency and duration of use of Active-Online showed an increase in self-reported total minutes of physical activity with increasing duration of use. However, this result was no longer significant when adjusted for stages of behaviour change (a concept based on the seven-stage behaviour change model as described by Martin-Deiner et al. (2004)).


Limitations

The inclusion of SU as an additional study arm may be seen by some readers as an interesting and exploratory endeavour. However, others may find that it takes away from the clarity of the study. The fact that the SU group is not randomized, not homogeneous with the two other groups, and only represents 7.4% of all visitors of Active-Online may cause some readers to wonder why it was included in the study at all. Moreover, the measurements obtained from this group were explicitly discounted in the “Discussion” section of the paper. It is suggested that the authors alert readers of the exploratory nature of the SU group early in the “Methods” section of the report when this group is first introduced. Readers will thus be made aware that the SU will not be counted towards the results of the study and has only been included to add another dimension of interest.

It is not clear in the report whether or not the authors had a set of eligibility criteria for participants, although this being a web-based study with no face-to-face component, it would have been difficult to enforce any eligibility criteria on participants at all. Furthermore, it is not known from the report how the authors ensured the uniqueness of participants. Participants were identified using unique email addresses, but it is quite likely that a single participant may have registered for the study multiple times using several different email addresses. This could seriously impact the study results if the same user was assigned to more than one user group as a result of using multiple email addresses.

In addition to the eligibility criteria, it is recommended that the authors provide a sample of the advertisement used to recruit participants and the questionnaire used at each follow-up, to comply with the CONSORT standards for reporting RCTs (CONSORT, 2009).

One other limitation of the study is that most of the participants already had high levels of physical activity at baseline, leading to a ceiling effect. As expected from such a large sample size, the results showed a regression towards the mean physical activity level in the population.


Discussion

Overall, the study was very well presented, with sufficient background information, clear writing, and appropriate use of tables and figures. The authors took a bold step in conducting a web-based intervention in an uncontrolled setting over a very long period of time. It is commendable that the authors frankly reported the limited effectiveness of their intervention when some researchers may have hesitated to do so. They did not go beyond their evidence to draw conclusions.

The study clearly answered the three questions set forth in the objective:

  1. The study found significant increase in physical activity levels between baseline and last follow-up (FU3) in all groups; however, there was no difference in the results between the randomized groups.
  2. Spontaneous users differed from randomized users in baseline characteristics, and also showed a significant increase in physical activity levels after using the intervention, compared to the randomized groups.
  3. The impact of frequency and duration of use of Active-Online on changes in physical activity levels of participants is not clear after the study.

The results from this study resonate with those of similar studies investigating web-based physical activity interventions (Spittaels et al., 2007). It adds to the existing evidence that effectiveness of a web-based physical activity intervention may be difficult to demonstrate when delivered in an uncontrolled setting. The study highlights some of the key issues pertaining to web-based studies in real-life settings, including attrition and contamination of the control group. High attrition rates have been recognized as a common problem in Internet-based studies (Eysenbach, 2005) and this was evident in the present study as well. It was also acknowledged by the authors that some members of the control group were familiar with and had used Active-Online at least once during the course of the study. This may have caused a bias towards the null.

Results from this study will be particularly useful for researchers in the field of healthcare and sports medicine. Further research could include the delivery of web-based physical activity interventions within wider health promotion contexts such as primary care or workplace settings.


Questions to the Authors

  1. How do you account for the contamination of CG in Internet-based studies such as this one?
  2. Could members of CG have accessed Active-Online as SU (using a different email address)?
  3. What were the technical difficulties causing 38 participants to be omitted from the study?
  4. How did you validate the uniqueness of the participants?
  5. What was the reason for not measuring the usage of the non-tailored website?


References

CONSORT. (2009). The CONSORT Group. Retrieved November 14, 2009, from The CONSORT Group: http://www.consort-statement.org/

Eysenbach, G. (2005). The Law of Attrition. Journal of Medical Internet Research , 7 (1), e11.

Martin-Diener, E., Thuring, N., Melges, T., & Martin, B. (2004). The Stages of Change in three stage concepts and two modes of physical activity: a comparison of stage distributions and practical implications. Health Education Research , 19 (4), 406-417.

Spittaels, H., De Bourdeaudhuij, I., & Vandelanotte, C. (2007). Evaluation of a website-delivered computer-tailored intervention for increasing physical activity in the general population. Prev Med, 44 (3), 209-217.

Spittaels, H., De Bourdeaudhuij, I., Brug, J., & Vandelanotte, C. (2007). Effectiveness of an online computer-tailored physical activity intervention in a real-life setting. Health Education Research, 22 (3), 385-396.

Wanner, M., Martin-Diener, E., Braun-Fahrlander, C., Bauer, G., & Martin, B. (2009). Effectiveness of Active-Online: An Individually Tailored Physical Activity Intervention, in a Real-Life Setting: Randomized Controlled Trial. Journal of Medical Internet Research , 11 (3), e23.

No comments:

Post a Comment