As technology continues to shape the way we conduct studies, researchers are increasingly opting for Bring Your Own Device (BYOD) for experience sampling. In this post, we will explore some of the advantages and important considerations when implementing a BYOD approach in experience sampling, focusing on how each aspect will impact participant motivation and burden. For more information on daily burden, check out our previous post.
The Advantages of BYOD in ESM Studies
As we’ve discussed throughout this series of articles, balancing participant burden and participant motivation is vital for a successful experience sampling study. Using the BYOD model can help increase motivation and decrease burden. There are a number of related advantages to the BYOD method.
1. Device Familiarity & Ecological Validity
A 2018 randomized equivalence trial found that most participants are willing to download an app onto a personal device in order to participate in a study. This study also found that for data collection, only 15% of their sample preferred a provisioned device, whereas 40% of their sample had no preference and the other 45% preferred to use a personal device (Byrom et al., 2018).
Leveraging participants’ personal devices fosters a sense of familiarity and comfort. BYOD also eliminates participants having to learn to use a provisioned device that may be different from their own. Participants experience burden when they have to carry multiple devices. Familiarity with the collection device may also contribute marginally to the ecological validity of the data collection as participants will be using a device that is already a part of their daily routine.
2. Potential to Increase Enrollment and Adherence
Allowing participants to bring their own device opens study design up for remote onboarding and fewer visits to on-site labs or clinics. This, in turn, can reduce the participant burden of having to come to a physical location to be onboarded or to check in, thereby increasing motivation to participate in and complete an ESM study (Demanuele et al. , 2022).
Ultimately, these impacts on burden and motivation are reflected in the compliance rates of BYOD participants versus non-BYOD participants. Participants in a 2016 feasibility study found that those who used their own device showed significantly greater engagement than participants using provisioned devices (Pugliese et al., 2016). In the same study this higher engagement translated to better medication adherence, indicating that opting for BYOD may improve clinical outcomes as well.
3. Cost Savings for Researchers
Providing devices to users can be costly to research sponsors due to the cost of the device and the cost of their corresponding data plans. According to one study, when participants bring their own devices, it can significantly reduce research costs. In a 2018 survey of attitudes toward BYOD, 74% of participants reported that receiving reimbursement for data charges was important, very important, or essential (Byrom et al., 2018). It is unclear whether data charges are as important now, given the number of people who now have unlimited data plans (see Statista Global Consumer Survey reports here and here . However, for populations that have limited data plans, structuring your study around locations that offer wifi can help to reduce or prevent the need for such reimbursement.
Considerations for Implementing BYOD in Experience Sampling Studies
The statistics seem to clearly support the use of BYOD in an experience sampling study. However, as with every aspect of ESM protocol, there are several key considerations to think through before inviting participants to bring their own device.
1. Privacy and Security Concerns
As smartphones continue to gain prevalence, privacy and security concerns are paramount. Often, the BYOD method of data collection requires the use of a third-party data collection service, which may concern some participants. When polling attitudes related to BYOD, the earlier cited randomized equivalence trial found that 78% of participants polled rated data privacy to be essential, very important, or important (Byrom et al., 2018). This concern can be addressed by minimizing sensitive or personally identifiable data collected from participants. Many intensive longitudinal platforms allow for anonymous data collection to reduce or eliminate security concerns.
Clear communication with participants that the data collection vendor complies with robust security measures can also help participants to feel more confident. Disclosing what, if any, additional data the vendor will be collecting from the device, in addition to the participant’s self-reported data, is also an important trust-building measure that can increase participant motivation.
Device Security Concerns
The BYOD method can introduce concerns around device security. Personal devices may not be secured with a passcode or biometric login, and if they have some device-level security, the access may be shared with friends or family members. Researchers can work to ensure that the authenticity of participant self-reports is protected with questionnaire-level security measures, such as PIN numbers or biometrics.
Typically, mobile apps developed with ESM in mind tend to have an advantage over SMS messages containing assessments because they are able to implement and provide app-level security for increased data validity. If you are working with an ESM mobile app, ask your provider about questionnaire-level security options to eliminate concern around device security.
Device Variability
A BYOD approach means that participants will be using devices that differ in age, screen size and resolution, available RAM, bloatware included on the device, battery optimization, participant data plan or lack thereof, and many other aspects. This variability can contribute to the variability of participant experience, which is why piloting with live devices of various makes, models, and generations of device is important to identify and address potential device-specific problems.
When devices are provided to participants, some of these issues will still arise, but may be limited to the device chosen for the study. One earlier cited feasibility study found that while initial adherence rates were higher for the group using a study-provided device, the decline among provisioned device users was greater (Pugliese et al., 2016 ). This is likely due to the burden associated with remembering to bring and interact with an unknown device compared to their own.
Technical Support and Compatibility
When participants become frustrated by unique, device-specific issues, the added burden can impact the outcome of an ESM study. To combat this potential problem, researchers must find an adequate platform that:
- Supports a reasonable number of device makes, models, and generations, including older models
- Functions over wifi, on roaming data, in low-data mode, and offline as needed, since not all participants will have an unlimited data plan
- Prioritizes uniformity across many device types while also working with the native controls that participants are accustomed to using, such as the “back” button on Android devices or gestures on Apple devices
- Offers user and researcher support for technical errors and troubleshooting
- Retains and reports device information, such as make and model, to allow researchers to note variability between device types and patterns in function
Researchers should also develop troubleshooting documentation and clear onboarding processes for study staff to help prevent and remedy frustrating device issues, such as missing an assessment due to notifications being deactivated by a device’s battery optimization mode.
Potential for Bias and Impact on Validity
Some researchers have argued that BYOD data collection may introduce systematic bias because participants with technological literacy, access to compatible devices, and sufficient data plans may not be representative of the target population (Demanuele et al. , 2022; Cho et al., 2022). However, other studies have contended that while smartphone usage may never reach 100% of the population, rates of usage are roughly equivalent across certain demographic categories such as race (Coons et al., 2015). Recent estimates place smartphone ownership at approximately 90% of the population in the United States. Additionally, the European Commission estimates smartphone prevalence ranging from 78%–98% in constituent countries with a mean of 87%.
ESM researchers should review the literature relevant to their research question to assess whether there is a reason to expect significant differences between those who own smart devices and those who do not. In many cases, a hybrid BYOD/provisioned device model may be appropriate (Demanuele et al. , 2022). When running a hybrid study, researchers should expect a larger participant burden to be placed on those individuals who may be using unfamiliar technology. Additional care should be taken to train and support those participants.
When the goal of research is to encounter participants in their daily life, few options get closer to a participant’s daily experience than integrating with their personal device. When conducting an ESM study, allowing participants to provide their own device can limit participant burden, and collecting data using an excellent ESM-focused app can help researchers to prevent technical difficulty, while also increasing ecological validity.