Participants, Interventions, and Outcomes
Study Setting
Overview
The study setting to recruit participants is the geographic boundaries of a large academic integrated health care system spanning several contiguous counties in a Midwestern state. The study management site is housed within the academic medical center (a university with a medical school and a teaching hospital or health care system) [18]. As an mHealth study, this study is conducted in a fully remote fashion, including partnerships with local community organizations.
Eligibility Criteria
There are two groups of participants for this study: (1) individuals who will receive the intervention and (2) health care professionals practicing in a broader health care environment.
Inclusion and Exclusion Criteria for Participants Receiving the mHealth Intervention
Participants will self-refer to the study; members of the study team will determine eligibility based on self-reported data (Textbox 1) collected via a secure, web-based screening survey.
Other substance use disorders or misuse of other substances will not preclude participants’ enrollment in the study. If the following scenarios occur during a participant’s study period, the participant will remain enrolled in the study and their circumstances will be documented by the study coordinator:
• The participant is unreachable for follow-up surveys.
• The participant becomes incarcerated. In the case of incarceration, no study data will be collected during the time they are in custody.
Inclusion and Exclusion Criteria for Health Care Professionals
The selected group of health care professionals will include (1) clinicians who provided information about the study to at least one patient; (2) health care professionals who participated in interviews during the development year preceding the clinical trial; or (3) health care professionals who were involved in the implementation and could provide a health system perspective.
Intervention
Overview
Similar to previous and concurrent versions of A-CHESS, Tula operates on smartphones. Tula maintains the core components of A-CHESS but is modified to address the broad spectrum of issues related to alcohol misuse likely to be found in any community in the United States. For example, while the original version of A-CHESS [8] encouraged participants to abstain from using alcohol as a part of their AUD recovery, Tula encourages users to reduce their use of alcohol. One key adaptation is that Tula adopts the principles of the Whole Health model [18], placing alcohol use reduction within the context of integrative medicine, an adaptation made based on input from partnering health care stakeholders.
Tools and services in Tula (Textbox 2) include self-assessments; goal-setting tools and strategies; techniques for maintaining motivation and building healthy habits; information about drinking and wellness; strategies for reducing drinking and meeting other health goals; audio recordings; and other tools for stress reduction, relaxation, and personal health management.
mHealth Implementation Models
Table 1 outlines the key characteristics of the study arms during the 3-month active intervention period for each patient. All participants had access to the same basic content in Tula. The key differences relate to the level of human touch available by study group assignment.
Self-monitored Group (Low Touch)
Participants will use the app on their own, just as they would any commercially available health app downloaded to their smartphone. There is no access to a discussion forum and private messaging, or the aid of an external care team or social support. Participants may reach out to the study team by email, phone, or from within the app via the “Messages from the Researchers” tool. Communication is only intended for participants to ask questions about the app, about the study, and receive technological support. The study team conducts routine safety monitoring based on participant use data and any communication initiated by the participants to the study team.
Peer-Supported Group (Medium Touch)
Participants have access to social support and access to certified peer support specialists. Peer support specialists are staff members from a community partner outside of the health care system. Newly randomized participants receive a welcome message via the private messaging feature in Tula from peer support specialists. To maintain anonymity, all participants identify themselves using their username. The participants’ main interactions are with other members in the same group through discussion forums where they share posts. Peer support specialists moderate and participate in the discussion forums while encouraging the use of Tula (eg, by posting topics in the discussion forum, pointing Tula users to a potentially helpful tip). They have no access to study data and only know the participants through their usernames.
Clinically Integrated Group (High Touch)
Participants have access to a discussion forum specific to this group and have access to a certified health coach. Health coaches, as employees of the health care system, help individuals make lifestyle changes to achieve their goals for health and wellness. Participants can have 3 one-on-one personal health coaching sessions (via phone) during the active 90-day implementation period. Health coaches will provide a structure for participants to achieve their self-identified goals by helping them envision a healthier lifestyle as they reduce their alcohol consumption. Participants can opt to share selected Tula data with the health coach through a dashboard—data on drinking days or drinks per day and other data reported via weekly surveys. These data allow health coaches to provide more responsive support to participants in meeting their goals for alcohol use.
Outcomes
Overview
Table 2 shows the outcomes and the mediators and their measurement. The definition and psychometric properties of each measure are also listed.
Primary Outcomes
The primary outcomes include patient-reported risky drinking days and quality of life. Risky drinking days are the number of days on which a participant’s drinking in a 2-hour period exceeded 4 standard drinks for men and 3 standard drinks for women, defined using the National Institute on Alcohol Abuse and Alcoholism’s definition of a standard drink as 1 containing 14 g of 1 alcohol (12 oz of regular beer, 5 oz of wine, or 1.5 oz of distilled spirits) [2]. Risky drinking days are measured using the patient-reported timeline follow back survey. The timeline follow back survey consistently demonstrates reliability for assessments in recall periods as long as 6 months, test-retest reliability of ≥0.80, and convergent and discriminant validity with other measures; it also correlates with collateral reports and urine toxicology tests [28]. Quality of life will be measured using the Patient-Reported Outcomes Measurement Information System (PROMIS) Global Health scale, a 10-item subjective measure of general health [29]. It includes a 4-item global physical health scale (Cronbach α=.81), a 4-item global mental health scale (Cronbach α=.86), and two additional items—general health and satisfaction with social roles—that can each be scored as a single item. The PROMIS scales were developed using item response theory and capture a greater range of the trait being measured with greater precision than other instruments. At the beginning of the COVID-19 pandemic, we added two 5-point Likert scale items asking about the impact of the COVID-19 pandemic on participants’physical and mental health.
Cost Outcomes
Cost measures include health care use costs and intervention and implementation costs. Health care use will be collected from the self-reported medical services utilization form [24] (6 items). Intervention costs will be determined using Tula time-stamped log files and billing codes logged by the health coach. Implementation costs will be determined through clinic staff interviews at the end of the intervention period using the Cost of Implementing New Strategies model [30]. The cost per participant will be determined for each study arm (self-monitored, peer-supported, and clinically integrated) with intervention and implementation costs separated within each group.
Mediators
Several mediators include the 3 fundamental psychological needs based on the self-determination theory [15] and the Tula use data. Among the 3 core needs specified by self-determination theory, relatedness will be measured using the McTavish 5-item Bonding scale [26], which is highly correlated with other social support scales, has a high reliability of 0.9, and has been found to mediate effects between patients’ use of eHealth systems and their coping behaviors [26,31]. Competence will be measured by the perceived competence scale, a 4-item alcohol-addiction focused scale rating confidence in the ability to cope in high-risk situations; reported reliability was above 0.9 [32]. Autonomous motivation will be measured by the modified Treatment Self-Regulation Questionnaire, which uses a 7-point Likert scale to respond to 5 items assessing the degree to which a person’s motivation for healthy behavior is autonomous. Previous tests found good reliability (Cronbach α=.88) and predicted changes in health-related behavior (P<.001). Participant use of Tula will be measured by the number of days using Tula and the number of pages viewed. Days of use and pages were found to be associated with a reduction in risky drinking days among AUD participants [8]. Clinician use of Tula will be measured by counting the number of days per month in which clinicians log into the system during the intervention period and the pages viewed.
Participant Timeline
Participants’ involvement in the study can be divided into four distinct phases: (1) a pre-enrollment period encompassing screening completion, evaluation by the study team, and downloading the app; (2) an enrollment period consisting of a 2-stage consent process framing a 72-hour run-in period, a phone call with a member of the research team, randomization, and the final confirmation of enrollment; (3) a 3-month active intervention period; and (4) a 9-month follow-up period (Figure 2). The pre-enrollment and enrollment phases, together, may range from approximately 48 hours to 3 weeks, depending on the participant’s responsiveness and availability. Once enrolled, a participant will be engaged in the study for 12 months.
Sample Size
This study was designed to detect the differences in the two primary outcomes, Risky Drinking Days and Quality of Life, among the 3 groups. In a previous study [33], after a 6-month web-based intervention study, alcohol users in the internet-based, therapist-led group reported having fewer drinks (Cohen d=0.38) and a better quality of life (Cohen d=0.44) than those in the internet-based, self-help group. In this protocol, sufficient power (1-β=.80, multiple comparison adjusted Cronbach α=.00833, 2-tailed) to detect a more conservative effect size of Cohen d=0.25 in a design with 4 repeated measurements with a first-order autoregressive covariance structure (correlation ρ=0.3) would require approximately 182 participants per group (or a total of 546), assuming 28% attrition. Using prior research [14] to estimate the SD of risky drinking days, the effect of Cohen d=0.25 would equate to a difference of approximately 0.24 risky drinking days per week and 1.62 overall quality of life value measured by the PROMIS Global Health instrument.
Recruitment
Participant Recruitment
Recruitment for this study relies on raising its visibility among key groups of stakeholders and tailoring our outreach efforts to be relevant and sensitive to each group’s needs and interests. The study’s recruitment strategy has centered on three key areas: clinical settings, community-based organizations, and the public media marketplace. By promoting the study to health care professionals, primary care providers, social workers, emergency departments, and behavioral health specialists operating within the local health systems, we enlisted the help of clinical study champions and aim to provide information and resources they can share with patients who may be eligible for the study. Engaging with local leaders from underrepresented and marginalized communities and working to build or fortify relationships with community-based organizations and partners is critical for promoting the study in a way that invites participation and inclusion of diverse voices, perspectives, and experiences. Finally, the use of targeted digital and print media enables the study team to promote the study broadly but strategically while providing information and materials that enable potential participants to learn about the study in a way that maintains discretion, privacy, and autonomy.
All participants who complete the eligibility screen receive a US $10 incentive, delivered in the form of a digital gift card when the study team notifies them of their eligibility.
Recruitment of Health Care Professionals (After the Intervention)
Upon completion of the study’s active intervention and follow-up period, the study team will invite a select group of health care professionals to participate in the interviews. Email invitations will ask if they would like to share their experiences. Details of the study and their participation will be explained, and if they agree to participate in the study, signed informed consent will be collected in person, at the time of the interview, by a member of the study team.
Assignment of Interventions
Randomization
The randomization list was generated using the block randomization procedure in the Power Analysis and Statistical Software (PASS 2020) and stratified for participant-reported biological sex (male or female) and alcohol use severity (mild or moderate) based on Diagnostic and Statistical Manual of Mental Disorders, fifth edition (DSM-5) scores (eg, mild=DSM-5 score 2-3 and moderate=DSM-5 score 4-6) calculated from participants’ responses in the pretest survey (Table 2).
To prepare the randomization tool, the program manager of the study team entered the sequence of randomization assignments for each stratification group into a protected spreadsheet and masked the data. During randomization, a researcher from the study team conducts an active call with the participant, enters the participant’s study ID into the next available placement in the randomization sequence, and reveals or unmasks the participant’s group assignment. Although on call with the participant, the researcher configures the participant’s permissions in the app based on their group assignment and confirms that the participant has access to the version of the app that corresponds to their placement.
Before the randomization call, to limit or avoid interaction between the participants and the study team members responsible for conducting randomization, each stage of the pre-enrollment and enrollment process (screening, eligibility communication, scheduling, and randomization) is led by a different member of the team.
Enrollment
Potential participants will complete a web-based eligibility survey on the research team’s website. Eligible participants will receive an invitation to participate with instructions to begin the enrollment process and a link to download the app. They may receive up to 2 email reminders during the 2 weeks following their completed screening.
When participants opened the app, the first of 2 digital informed consent documents appeared. Participants may contact the study team by email or phone with questions or concerns before electronically signing the consent form. If a participant declines to participate (eg, do not consent), reasons for refusal (if provided) will be documented, in keeping with CONSORT (Consolidated Standards for Reporting Trials) standards [32]. The study team will not initiate further contact with the participant.
Once participants consent and submit the digital consent form, they can create a user account initiating a 72-hour run-in period where they complete the baseline survey bundle (10 surveys, 64 questions in total). Participants who do not complete the 72-hour run-in are not randomized; their account is deactivated; they are removed from the study; and the reason for exclusion (“did not complete run-in”) is documented accordingly.
Participants completing the 72-hour run-in are invited to schedule a call for the final step in the enrollment process. During the call, a member of the study team reviews the study and first consent form, answers any questions, and completes the randomization process. Once the app permissions have been reconfigured to the appropriate study group, the participant receives a second informed consent to review and sign. The second consent document describes the study activities specific to the participant’s group assignment. Both informed consents are stored in the Study Information page in Tula for the participants’ records with an option to print a paper copy.
Participants may withdraw their consent at any time throughout the 12-month period after their enrollment, at which time the study team will document their request, deactivate their account, and send a final confirmation. The participant will not receive further communication from the study.
Data Collection, Management, and Analysis
Quantitative Data Collection
The data collected by the Tula system come from multiple sources (Table 2). All surveys are conducted in the Tula app. Participants receive survey notifications from the Tula app on their phones and then complete the survey in the Tula app accordingly.
Baseline and Quarterly Surveys
Weekly Surveys
Tula prompts participants to take weekly surveys administered via the app to track their drinking and quality of life. Participants report their drinking using the 7-day timeline follow back [21,22] survey. A modified version of the Brief Alcohol Monitor [27] survey (10 items) allows participants to track individual risk and protection factors that may influence problematic alcohol use.
Tula Use
Tula use data are collected in time-stamped log files and include when a participant accessed Tula, the services used, duration of service use, pages viewed, messages posted and received, and content of messages. Tula use measures the dose of the intervention received for dose and response analyses.
Health Care Use
Health care use will be collected from the Self-reported Medical Services Utilization Form 61 (6 items; see Table 2 for frequency).
Intervention and Implementation Costs
Intervention costs will be determined using Tula time-stamped log files and billing codes logged by the health coach and peer mentors. Implementation costs will be estimated through health care professional interviews at the end of the intervention period and organized according to the Cost of Implementing New Strategies model [25]. The cost per participant will be determined for each study arm (self-monitored, peer-supported, and clinically integrated) with intervention and implementation costs separated within each group.
Qualitative Data Collection
Interviews will take place once the intervention phase of the study is complete and last 30-60 minutes. Open-ended questions will allow health care professionals to assess their experiences with Tula and provide feedback on implementation as well as their thoughts on participants’ experiences using Tula. Interviews will be conducted in a private location convenient for participants, either in clinician offices or via a secure videoconference call. All interviews will be audio-recorded and transcribed.
Retention or Adherence
Participants who complete their enrollment in the study are eligible for renumeration. Study incentives are built into the first 12 weekly surveys and 4 quarterly follow-up surveys at months 3, 6, 9, and 12. How to earn the incentives (gift cards) and how the incentives will be distributed monthly to participants by digital gift code sent via the message feature in Tula are explained to participants during the final step of enrollment. At the end of the 12-month study period, participants could potentially earn up to US $250 in gift cards.
Data Analysis
Primary Analysis
The analysis assesses the direct treatment effects on participant outcomes over time. We will construct a longitudinal model of the outcome measures at 3, 6, 9, and 12 months after randomization. Variables, stratified by sex and severity of alcohol use, will be included as factors in the model. The baseline values of the outcome will be included as covariates, with a separate model for each primary outcome (risky drinking days and quality of life). This longitudinal analysis is complicated by the dependence on successive observations made on the same individual. Furthermore, as complete control of measurement is not possible, there may be incomplete data from individual participants. Therefore, we will conduct a mixed-model analysis of repeated measures based on the general linear model with the assessment of various covariance structures (compound symmetrical, autoregressive order one, and unstructured). Covariance structure selection is based on the Akaike information criterion and Schwarz Bayesian criterion [23,24]. Pairwise comparisons between treatment groups and specific treatment time contrasts in the mixed model will be conducted to respond to between-group effects and time-based effects.
Descriptive Analyses
The research team will use descriptive statistics for all demographic and clinical variables across all 3 arms. To assess the impact of chance baseline imbalances between arms on intervention effect estimates, variables with noticeable differences will be included as covariates in a sensitivity analysis.
Mediation and Moderation Analysis
To augment the intervention analysis, we will estimate the direct and indirect effects that groups have on the outcomes by mediating variables using the structural equation model method. To test moderation effects, the interaction of the moderators (gender and severity of alcohol use severity) and randomization will be added to the model and estimated separately. We will examine the magnitude and direction of differential intervention effects between levels in these moderators (eg, male vs female).
Cost-effectiveness Analysis
Operational cost will be calculated based on the tenets of engineering economics. This study will use incremental cost-effectiveness ratios to compare the clinically integrated, peer-supported, and self-monitored groups. The incremental cost associated with reductions in drinking days and improvements in quality of life will be calculated over the 12-month intervention period.
Qualitative Data Analysis
Content analysis will describe how Tula use can improve patient outcomes, identify potential improvements in Tula, and identify how qualitative data can supplement the quantitative analysis. A qualitative researcher will construct a coding scheme to assess the ideas of the study to capture references to a concept. The analyses will help the research team refine Tula for future dissemination by determining the individual and organizational conditions necessary to promote effectiveness.
Data Monitoring
All study data collected by the app will be stored in secure password-protected servers. No patient health information will be collected from their electronic medical records, and no data collected by the app will be entered into patients’ electronic medical records or affect the legal medical record. All participants are assigned an ID number, and all data will be deidentified before exporting for statistical analyses. Any hard copy–identifying information will be stored in a locked cabinet. When all study activities are complete, audio recordings, participant ID, and other identifiable information will be destroyed; only the deidentified code will remain.
With consultation from the funding agency (National Institute on Alcoholism and Alcohol Abuse) and the institutional review board, this protocol was exempted from requiring a formal data monitoring committee review.
Ethics Approval
This protocol was initially approved by the Health Sciences Minimal Risk Institutional Review Board (2019-0337) with subsequent annual reviews.