Experience sampling is a valuable tool for researchers to gather data on various aspects of people’s lives. However, finding the right balance between participant burden and motivation is crucial for successful studies. In this article, we outline four keys to striking the optimal balance between burden and motivation.
1. Protocol Design in Experience Sampling
Having a research-based protocol is a critical element of a well-designed ESM study. The nature and number of questions which should be used varies depending on what is being studied. In some cases, a question or two will provide sufficient information, while in others a validated instrument may be needed. In studies involving multidimensional constructs, researchers must identify a sufficient number of questions to assess each dimension while being mindful of the additional burden. According to a recent study, considerations such as the characteristics of the group being studied, time of day, and pre-existing stressors should also be considered as they may affect participant burden. In any case, several considerations are at play when developing a research protocol, and each consideration requires diligent planning to craft an effective study. Giving a protocol the proper amount of time and consideration is the first key to effectively gathering accurate experience sampling data.
2. Participant Incentives as Motivation
Are incentives necessary to procure engagement? Absolutely. Incentives and motivation go hand-in-hand. The more motivated a participant is, the more likely they are to engage, and the more tolerant they will be of longer surveys, more detailed responses, and more frequent requests for information. An ill-motivated participant may get into the habit of swiping away notifications, which can culminate in participant drop out and can ultimately affect the reliability of participant data. Here are some common types of incentives:
- Altruistic: Raising participant awareness that what they are doing is important or will help other people.
- Financial: Rewarding participants for completing a certain percentage of responses or for each response given.
- Achievement: Adding percentage completed or other fun user interfaces.
Whichever approach is taken, incentivizing responses is certainly the most direct way to increase motivation for a balanced ESM study.
3. Determining Daily Burden
The length of sessions and the frequency of prompting combined add up to the daily burden associated with an experience sampling study. Finding an appropriate daily burden comes with a number of unique considerations. These include:
- How many questions are too many?
- How often is too often to request a response?
- How much will users be willing to read?
- What length of responses are they willing to contribute before getting tired or frustrated?
- How long should the study run?
There is no golden answer to these questions; each study will be unique. Consider how often users will be prompted to answer these questions, or if they’ll be responding to particular events as they occur. Consider the time-of-day users will receive prompts, and what other factors may be vying for their attention. One study indicates that including longer questions within assessment sessions significantly increased participant burden, whereas sampling frequency (3, 6, or 9 times daily) had no significant effect. Researchers whose aims require longer questions will need to balance the added burden with increased motivation for a successful study. The main goal is to minimize the daily burden to ask just enough questions in just the right length to achieve the study’s aims.
4. The “Bring Your Own Device” Strategy
In many studies, researchers will benefit from encouraging participants to use their own devices. While this strategy may not be applicable to every type of ESM study, those studies that can reasonably allow users to bring their own device will reap benefits, such as:
- Reduced participant burden: Users will not need to adapt to carrying a different device and will benefit from interacting with a familiar interface.
- Less confusion-related intervention required: Fewer responses will be affected by user error or technical difficulty.
- Low-to-no learning curve: Users are already familiar with the device, and only need to be introduced to the data collection app.
- Increased rate of engagement: Participants already carry and use their device during key times such as before and after work and during their morning and evening routine.
Though other strategies may be more fitting for particular research questions, allowing participants to use their own technology can significantly decrease burden and can lead to higher engagement rate overall. When researchers are considering whether or not the BYOD strategy is a good fit, they may consult studies like this one that explore the critical benefits and pitfalls of BYOD in clinical trials.
By applying these four keys, researchers in any space can find the balance needed to achieve the participant engagement they seek. Whether you’re studying processes, actions, emotions, or even workplace satisfaction, you can get the answers you need from a well-designed experience sampling study.