This is Part 2 of a two-blog post. Final take-aways from Dr. Conner’s Experience Sampling are discussed. Last month, I attended the Annual Convention of the Association for Psychological Sciences in New York City. One of the conference highlights was a workshop by Dr. Tamlin Conner of the University of Otago in New Zealand. During the two-hour session, Dr. Conner offered a great overview of Experience Sampling Method—what we refer to as Ecological Momentary Assessment (EMA)—and provided practical advice for those new to this methodology. While the session covered a lot of information, here are a few takeaways for those interested in learning more about EMA strategies:
5. Understand the different kinds of EMA schedules. Dr. Conner identified three kinds of EMA schedules.
- Signal contingent schedules involve the use of an alarm or other notification method to alert users to respond to a session. These notifications can be randomly distributed or may be stratified within a particular block of time.
- Interval contingent schedules gather data at specified intervals such as at wake time and bedtime, or at the top of every hour.
- Event contingent requires participants to initiate responding under certain conditions. For example, a smoker might be asked to respond whenever they feel an urge to smoke.
6. Length of EMA protocols. Researchers who are familiar with traditional survey designs can struggle to create effective EMA protocols. Gathering data in the moment and over time without careful consideration of the response burden on participants can diminish response rates and increase attrition. Dr. Conner offered a simple formula to help conceptualize the issue:
Response burden= (#Items/report) x (Reports/day) x (#Days).
She then suggested that 20 minutes/day was a rough estimate of the highest response burden one should place on study participants. Importantly, she qualified this estimate, stating that sampling participants multiple times per day, and/or over many days requires fewer and shorter items within each assessment session. Experience from our lab suggests that sampling more than 4 times per day tends to burn out participants quickly. An excellent strategy to manage this, as Dr. Conner noted, is to add an end-of-day diary or use a measurement burst design. The latter involves intensive periods of assessment interspersed with little or no assessment. A few years ago, we used this approach to study first semester university students. Participants in that study received daily notifications for three 1-week periods, separated by a few weeks of no assessment. We found that this lightened the burden on participants, though response rates still dropped by the third assessment week.
7. Analyses can be tricky—plan ahead and consult good resources. Because EMA involves the repeated collection of data within participants and across time, it can generate a vast amount of data. In one of our first EMA studies, we followed 81 freshman university students to understand how they spent their time. We randomly sampled them 4 times per day and once each evening for three different weeks during their first semester. By the third week of data collection, we had over 60,000 data points! Figuring out how to manage the data was not easy, especially on our old system. Since then, we have learned to gather only the data we need and we are continuing to learn how to analyze these complex data. To take full advantage of intensive longitudinal data gathered by EMA, within-person analyses are required. Dr. Conner suggested a number of resources to help with these more complex analyses: Hayes (2006) article on multilevel modeling, Garson’s (2013) book on HLM, and Raudenbush & Byrk’s (2002) book on HLM. Bolger & Laurenceau’s (2013) book, Intensive Longitudinal Methods: An Introduction to diary and experience sampling research, was also on her list of key readings. Those authors have created a helpful companion website to the book: intensivelongitudinal.com.
8. Great resources are available. While Dr. Conner listed several excellent resources in her talk, she humbly downplayed what I believe is the most comprehensive source available: the book she co-edited with Dr. Matthias Mehl of the University of Arizona. The Handbook of Research Methods for Studying Daily Life (2012) is packed full of useful content for EMA novices and experts alike. Another great resource for those just getting started is a one-hour webinar on EMA by Dr. Conner and Dr. Smyth of Penn State. Slides for her talk, as well as additional resources are available here.
9. Go with a well-established company. Dr. Conner highlighted LifeData and three other companies that provide apps specifically devoted to EMA/ESM. LifeData is committed to rolling out additional features and system improvements that will help us deliver top-notch services to our customers. Beware of settling for companies that don’t have a dedicated development team available to keep up with changes related to every release of a new operating system. Over the past couple of years, I’ve come to appreciate just how much goes into not only building, but also maintaining a reliable EMA system.
It’s an exciting time to be conducting psychological research. At no other point in history have we been able to reach so many people and gather so much ecologically valid data on human life. I anticipate that we’ll see a lot more talks from Dr. Conner and others, as researchers increasingly recognize the value of real-time, real-world research strategies.
For more information about LifeData’s system, click here.