8 lessons learnt from working on an Academic Start-up:
The ESM Item Repository
Yoram Kunkels, January 2025
Start-ups have been storming the limelight in the last few years. Mostly are active in the broader technology sector, such as Whatsapp, Netflix, and Airbnb. Though most of these start-ups are often mostly commercial in nature, one could easily forget that Academia is also more than capable to spawn groundbreaking startup projects. However, where more business-oriented start-ups might attract large investments and can thus quickly scale up and professionalise, academic startups are often more limited in that sense. So how to start, grow, and maintain an academic startup? Here I will give some pointers from my experience from the ESM Item Repository.
1. Identify a need / goal
With the ESM Item Repository we tried to solve a problem that we identified as a broader need within our academic field, namely; addressing the need for a central database containing commonly used ESM items. Having such a clearly defined need to solve will help constrain your efforts and planning. Beware of adding or expanding to much in this stage; keep your goals simple, concise, and in touch with the practicalities of the issue you are trying to solve.
2. Get a capable team together
A capable team is one of the most important ingredients to make your academic startup a success. Though academic startups often do not immediately have access to large sums of capital to attract the best and brightest, they do have an important advantage. That is, many of the best and brightest start their career in academia, through study or work, so your ideal starting team will often be right at your doorstep. Moreover, a team is a long-term investment, so skills that are not yet completely filled in the team, can be trained or found in future team members.
3. Start thinking and planning
With your starting team complete, let's start thinking, talking, and brainstorming together. For example, an initial hack-a-thon session was the successful start of the ESM Item Repository. It helps make plans more concrete, and allow team members to give feedback on the plans and help develop them. It can also help to get your team structure clear early. Who is leading the project? How to match each team members best skills to the work? However, as the project is in its early stage it might be beneficial not to plan to much, as a lot can still change. That is, you might want to start focussing now on the next point.
4. Get a prototype or minimum viable product (MVP) soon
While it might be tempting to try and completely plan out your product and it's development beforehand, it might be more efficient to start working on a prototype or minimum viable product (MVP) soon after the initial brainstorm session. This will allow you to build practical knowledge about the product you are creating and will highlight potential challenges early on. For example, with the ESM Item Repository we had a prototype running relatively quickly after our first brainstorm session. This first prototype was flawed and thus we decided on another option. However, without this first prototype we would not have this valuable practical experience, and allows others to give feedback. So if it is possible to fail cheaply and frequently, as with our software, it might actually be wise to do so. Also because it trains skills and experience in your team, which brings us to the next point.
5. Build, manage, and maintain the team
While you might already be happy with your initial team, it is important to recognize that it is subject to change, as for example, people can start other jobs. Also, as an academic startup, many team members might only be available part-time, requiring extra management. I think the natural academic team process lends itself well to an academic startup, as it is naturally focussed on developing people's skills.
6. Create awareness / sell your product
Now hopefully you have a working team and product, thus making it time to start pushing your product into the market. If for example, your product is an online platform, it might be as easy a putting your website online. However, even then making the right people find your website can still be difficult. A whole business field has developed around this, called Search Engine Optimalization (SEO).
Also be ready to engage your audience personally, and invite them to participate. As the ESM Item Repository depends on researcher collaboration (in contributing their ESM items), this already created positive engagement which motivates people to use and revisit our site.
7. Ask / listen to user feedback
Just as your team grows, your product might also have to grow and develop to keep up with customer requirements. To do this properly, user feedback is necessary. Think about what kind of feedback you want, and how to acquire it. While a short questionnaire or text field on your website can yield some basic information, a more thorough look at user experience might require some more in-depth focus groups. Adress any negative feedback you receive, and ask users for confirmation on new features.
8. Professionalise your workflow and infrastructure
Early on during the academic startup lifecycle you might be dependent on free-to-use alternative to professional options. For example, an online app might be hosted for very low costs or even for free. However, these free plans are often throttled on important features. Look what would offer the most benefit for the lowest cost in the beginning. For example, for the ESM Item Repository we have free app hosting via shinyapps.io. The downside here is that it comes with an unsightly and very forgettable URL, e.g., “myname.shinyapps.io/My_New_App_v18”. This made it difficult for people to remember and visit organically. Hence a cheap and simple solution we used is just to rent a proper domain name (“www.esmitemrepository.com”) and redirect visitors from there to our app.
The same goes for (digital) infrastructure. There is often a plethora of free or inexpensive digital tools. For example, sites like GitHub allow not only for software version control, but also project planning tools like milestones.
The ESM Item Repository: Review of 2024 and looking ahead to 2025
Olivia Kirtley, January 2025
New Year is always a contemplative period and so it seems like a good moment to reflect on how 2024 was for the ESM Item Repository, and look ahead to what’s in store for 2025.
Last year marked a major milestone for the Repository: in March, we entered our 1000th item into the Repository. Then, with thanks to our newly established Data Team (Benjamin Kunc, Steffie Schoefs, and Nian Kemme, as well as former interns Ben Símsa and Johan Le Grange), no sooner had we reached 1000 items, but by June, we had more than 3000 items in the Repository. Currently, there are 3,419 items in the ESM Item Repository, from researchers across 11 countries, and we have many more items being cleaned and processed ready for entry into the main Repository database.
In the last two years, the Repository team has been working hard to gather ESM items from “the wild”, i.e., harvesting items listed in published empirical articles and systematic reviews of ESM research. Whilst this may sound pretty simple — look at the methods section and find the ESM item details — this is painstaking, almost forensic work, as many empirical articles and reviews do not provide all the information that we are looking for about the ESM items, and this means we often have to contact authors or follow a citation paper trail to track down information about ESM items. For some, older, ESM studies, the exact wording of the items and their response options may be lost forever. This challenge only underscores the value of the ESM Item Repository and our mission to increase measurement transparency and methodological rigor in ESM research. To this end, alongside our work building an open bank of ESM items for researchers to use, we also conduct research and build tools to advance ESM measurement.
Another major achievement for the ESM Item Repository in 2024 was the release of the first quality assessment tool for ESM items, the ESM-Q. The development of the ESM-Q was led by Research Foundation Flanders Junior Postdoctoral Research Fellow, Dr Gudrun Eisele (Center for Contextual Psychiatry, KU Leuven) and Dr Anu Hiekkaranta (Center for Contextual Psychiatry, KU Leuven; University of Gothenburg, Sweden), and involved a Delphi study with 42 international ESM experts. The idea for the tool was born several years ago in a truly “Hal and the lightbulb” moment, common to ESM research: we need X to do Y, but as X doesn’t exist, I guess we better make X first. The result is a quality assessment tool with 10 core criteria and 15 additional criteria to help researchers judge “what makes a good ESM item?”. We hope that researchers will be able to use the tool to aid them in selecting and designing good ESM items, and potentially also as a tool for assessing item quality in systematic reviews of ESM studies.
Last year also saw the restructuring of the ESM Item Repository team into additional subteams: our existing coding team was joined by the scientific team and the data team. This enables us to better divide the different tasks that keep the Repository going and to work on building specialist expertise within these subteams. Especially important here is our new data team, led by Steffie Schoefs, as a constant stream of high-quality data (ESM items) is central to the continued growth of the Repository. We welcomed several new team members, Nieke Vermaelen and Martien Wampers, including masters student interns Beatriz Diaz (University of Algarve, Portugal), and Christina Stefani and Wikor Januszewski (KU Leuven, Belgium), and we also said goodbye to Nian Kemme (KU Leuven, Belgium), who left her Research Assistant position with us to pursue at PhD at the University of Nijmegen (Netherlands).
So what’s next for the Repository in 2025? For a little while now, we have been working on a pilot for a new workstream at the Repository called “Special Collections”. This builds on the idea of the Repository as a living museum of ESM items. In the same way that museums have special collections on a particular topic, the ESM Item Repository will soon start to display special collections of ESM items on different topics, to aid discoverability of items and facilitate ESM measurement work in specific domains. Watch this space for the first Repository Special Collection coming out in the first quarter of the year. Another development that we are very excited about is being led by CCP PhD student, Laura Van Heck, in collaboration with Martien Wampers and Steffie Schoefs, and that is a brand new item submission system for the Repository. We hope that this new system will make it easier for researchers to submit items to the Repository, and also reduce the potential for errors in the data that can cause back-end coding issues for the Repository search portal. You can expect to see this new system coming out later this year and some ESM Item Repository supporters may even get a sneak peek of the new system during testing.
As ever, we rely on researchers submitting their items to grow the Repository’s collection, so please consider making it your new year’s resolution to contribute to the ESM Item Repository and to help us build a better and more transparent science of daily life.
Using the repository: Emotion regulation items
Anu Hiekkaranta, November 2021
Many emotion regulation researchers are now expanding their investigation outside the laboratory to the study of naturally occurring emotion regulation. Ideally suited for such an endeavor of course, is the Experience Sampling Method (ESM). An ESM novice aiming to study emotion regulation in the wild may ask: Where do I start? We suggest you start by browsing the repository! You can explore existing items in the repository by using the search tools at https://www.esmitemrepository.com/. In this post, I will walk you through the search results for two emotion regulation strategies, rumination and acceptance, as well as how you might best make use of what you find in the repository.
Example case 1: Rumination
Let’s say you would like to study rumination in daily life. Rumination items are one of the most often included items in new contributions to the repository. One way to use the repository to find relevant items with a concept in mind is to look up items with a certain description. At the time of writing this blog post searching the repository for items with “rumination” in the description results in 14 hits. You will immediately be able to view the results and some details about them. For more information about each item, you can download the search results. Browsing the rumination items will give you an idea of how different researchers have adapted items from traditional self-report questionnaires to ESM items (e.g., I am thinking about my feelings, I am thinking about my problems (Moberly & Watkins, 2008), and I am ruminating (Snippe et al., 2019)). Moreover, your search results will also illustrate how others have developed ways to investigate ruminating by anchoring to different time frames, specific events, and specific thoughts. For example, some researchers first ask participants what they are thinking about at the moment they got the notification, followed by questions about the characteristics of their current thoughts, such as rating their agreement with statements like: This is a repetitive thought (ESM Item Repository, Kirtley et al., 2019, item 152). Alternatively, participants in a study may be asked to identify a specific event, such as the most important event in the past two hours. Then, participants can be asked questions related the event, such as I have thought about it a lot (most important event in the past 2 hours), (Kirtley et al., 2021). You may also note that researchers have anchored ruminating to different types of events: positive, negative, and important.
Per each search results with the description “rumination”, you can also see, among other details, what population the item has been used in, the response scale, and the number of times the item was presented per day. In addition, browsing for emotion regulation items in the repository reveals differences in how many items different researchers have used to study emotion regulation constructs within their experience sampling questionnaires. You will see for instance, that when studying rumination, some have opted for one item, while others use several.
Now, let’s consider what similarities we find when we search for another emotion regulation strategy in the repository, and take note some of the practical challenges in choosing the items for your study.
Example case 2: Acceptance
As with rumination, browsing the repository for acceptance-related items demonstrates the diversity in operationalization of emotion regulation constructs. For example, the description “acceptance” can be found in the repository for all of the following items:
-I let my thoughts be there without reacting to them (ESM Item Repository, Kirtley et al., 2019: item 307)
-Think about the most negative event today. I let it happen (Kirtley et al., 2021)
-Today, I could let go of my negative feelings without acting upon them (ESM Item Repository, Kirtley et al., 2019, item 341)
-(Most negative event) I just accepted my emotions (Kirtley et al., 2021)
Careful examination of these items reveals subtle but important differences. Acceptance, like rumination, can be anchored to e.g., thoughts, events, emotions, or emotions related to specific events. When deciding which items to use for any emotion regulation construct, it will be useful to consider not only one’s theoretical understanding of the construct of acceptance, but also practical concerns. Which of the items are easiest to respond to and how many times per day, given the population in which you want to study acceptance, and the timing of your study? For example, reflecting on one’s thoughts and reactivity to them several times a day requires a level of insight you may not expect from children or teenagers (and indeed, this poses a challenge to adults as well!). To tackle the issue, anchoring acceptance to events may help participants grasp the concept of acceptance better in the moment. However, persons living in quarantine or locked down cities as many currently are, may have few events to report. These concerns raise new questions. What will you consider an event? Do participants need to be briefed with specific examples of events that may take place for them, in their current circumstances? You may also conclude that existing items (gathered from the repository or elsewhere) do not sufficiently capture the type of emotion regulation you are interested in. In that case, we hope you return to the drawing board wiser and even a little more inspired!
Discover and develop
The field of emotion regulation in daily life is still in its infancy. Indeed, many fields in psychology are still new to experience sampling. Therefore, we invite you to use the repository to discover the existing items relevant to your research, be it emotion regulation or something else. We hope that what you learn may save you time and help you make decisions about which items to use. Moreover, we hope to inspire you to think about how to validate existing items and how to adapt and develop items to your research needs. As a final point, once you have collected some data, we warmly encourage you to make your own submission to the ESM Item Repository!
References:
Kirtley, O. J., Hiekkaranta, A. P., Kunkels, Y. K., Verhoeven, D., Van Nierop, M., & Myin-Germeys, I. (2019). The Experience Sampling Method (ESM) Item Repository. https://doi.org/10.17605/OSF.IO/KG376
Kirtley, O. J., Achterhof, R., Hagemann, N., Hermans, K. S. F. M., Hiekkaranta, A. P., Lecei, A., … Myin-Germeys, I. (2021, April 2). Initial cohort characteristics and protocol for SIGMA: An accelerated longitudinal study of environmental factors, inter- and intrapersonal processes, and mental health in adolescence. https://doi.org/10.31234/osf.io/jp2fk
Moberly, N. J., & Watkins, E, R. (2008). Ruminative self-focus and negative affect: An experience sampling study. Journal of Abnormal Psychology, 117(2), 314.
Snippe, E., Helmich, M., Kunkels, Y. K., Riese, H., Smit, A., & Wichers, M. (2019). Esm Item Documentation. Retrieved from https://osf.io/a8572/.
The struggle of selecting good experience sampling items
Gudrun Eisele, November 2020
I clearly remember the feeling when setting up my first experience sampling method (ESM) study. It felt like only few days had passed since the start of my PhD and the first meeting with my supervisors who said to “go ahead and have it ready by May”. So there I was, going ahead, kind of. The truth is, I was rather overwhelmed by the seemingly infinite number of decisions to make.
In this state of trying to figure out what I was doing and with May coming closer, I remember finding the item selection a particularly daunting task. Just quickly adding some items, “it can’t be that hard”, I kept telling myself. I spent most of my days scanning through ESM articles, which often did not report the exact items. When they did, these were usually not presented in the original language, and most of the time no information on item quality was provided whatsoever. How should we choose?
The broader problem
Almost 3 years later, after running this study (https://psyarxiv.com/zf4nm/), certainly making numerous mistakes, and seeing others set up studies, I know now that this is not an isolated experience. Choosing good ESM items is just very hard. There are many reasons for this: Frequently, exact ESM items are still not reported in journal articles and their quality remains extremely difficult to judge. Guidance on how to develop them is practically nonexistent within the ESM literature (there are exceptions, for example: Horstman & Ziegler, 2020; Palmier-Claus et al., 2010) and researchers selecting the items have often not received sufficient training in good measurement practices. Some labs may have elaborate procedures for the development or selection of ESM items, ranging from meetings, to pilot studies, to focus groups. However, the results of this work are rarely shared publicly, and all of this valuable information remains hidden away behind closed doors.
The ESM item repository
The ESM item repository is a first step in solving this issue. It provides a platform to collect and find existing ESM items. However, the repository does not currently help users choose from the large pool of items that are in it. You might find yourself scrolling through the increasingly long list of items (we now reached 756 item submissions!) and still have no idea how to select items for your study.
A Delphi study on ESM item quality criteria
This is why we decided to set up a Delphi study on item quality with ESM experts from different countries and fields of study. In our preregistered Delphi study, over the course of three rounds, we are hoping to develop an item quality assessment tool, specific to ESM, that can be used to evaluate existing items and develop new ones.
At the moment, 50 ESM experts who signed up or were nominated to participate are completing the first round of the study. During this round, we ask experts to name quality criteria that they use to evaluate the quality of ESM items. In the second round, experts will be asked to evaluate the resulting criteria. Finally, quality criteria that are agreed upon in the third round of the Delphi study will be used to develop the item quality assessment tool, which will be made publicly available.
We will keep you posted about the progress of the study here and on Twitter!
Horstmann, K. T., & Ziegler, M. (2020). Assessing Personality States: What to Consider when Constructing Personality State Measures. European Journal of Personality. https://doi.org/10.1002/per.2266
Palmier‐Claus, J. E., Myin‐Germeys, I., Barkus, E., Bentley, L., Udachina, A., Delespaul, P. A. E. G., ... & Dunn, G. (2011). Experience sampling research in individuals with mental illness: reflections and guidance. Acta Psychiatrica Scandinavica, 123(1), 12-20. https://doi.org/10.1111/j.1600-0447.2010.01596.x
The Experience Sampling Method Item Repository: What are we doing, why are we doing it and where are we going next?
Olivia Kirtley, October 2020
The Experience Sampling Method Item Repository is one year old! To celebrate our first birthday, we have launched a new blog where the rest of the repository team (Anu Hiekkaranta, Yoram Kunkels, Gudrun Eisele, Davinia Verhoeven, Martine van Nierop and Inez Myin-Germeys) and I will keep you up-to-date on the latest developments in the repository project, including highlighting the collection of items, and discussing ESM research more broadly. So let me start us off with our very first blog post, to tell you a bit about what we are doing, why we’re doing it and what lies ahead for the repository.
The Experience Sampling Method (ESM; Csikszentmihalyi & Larson, 1987), sometimes referred to as Ecological Momentary Assessment (EMA; Stone & Shiffman, 1994) brings myriad possibilities for gaining rich, valuable data about the context in which individuals’ thoughts, feelings and behaviours arise: their normal day-to-day lives (Myin-Germeys et al., 2018). Capturing these daily-life experiences, outside of the lab and the clinic brings us many new opportunities for increasing our understanding of phenomena, including individuals’ social experiences, emotion regulation, psychopathology symptoms and self-harm thoughts and behaviours. Over the years, ESM research has grown a lot and is a field full of exciting, cutting-edge statistical and technological developments. Perhaps surprisingly, however, some key questions around how we conduct ESM research, including about the measures that we use, remain unanswered.
We do not have a suite of validated ESM questionnaires, which have undergone a rigorous psychometric validation process, in the same way that we have validated measures for assessing depression or anxiety*. Consequently, many ESM items are constructed by researchers “on the fly”, and passed along from one researcher to another. Sometimes the same ESM items are used to assess different or related constructs and there is currently very little psychometric validation of ESM items (Horstmann & Ziegler, 2020; Wright & Zimmermann, 2019), with only around 30% of papers in a recent review reporting the psychometric properties of the ESM items used (Trull & Ebner-Priemer, 2020). Yet, measurement matters (Fried & Flake, 2018) and lack of transparency around measures, as well as the absence of solid psychometric validation, can threaten the transparency and methodological reproducibility of ESM research more broadly. Further, it may also limit the conclusions we can draw from our studies. Whilst conversations around the transparency, reproducibility and replicability of research have mainly focused on cognitive and social psychology, this does not mean that these issues do not apply to ESM research (Kirtley et al., 2020). So how do we go about addressing these issues?
A good first step is to get an overview of the items currently “out in the wild” of the ESM research landscape. That’s where we come in. The Experience Sampling Method (ESM) Item Repository (Kirtley et al., 2020) is an ongoing three-phase project in which we are building an open (publicly available) bank of ESM items, with the help of contributions from ESM researchers from around the world. Phase I of the project involves populating the repository and collecting as many items as possible. This first version of the repository is available to discover and search at www.esmitemrepository.com. At this stage, the repository has only undergone minor quality screening for formatting issues, spelling and language errors etc, so is very much still a “warts and all” list of ESM items. Items cover topics including mood, emotion regulation, psychotic experiences, quantity and quality of online and offline social experiences and self-harm thoughts and behaviours. This allows us and other researchers to gain insights regarding the range and content of ESM items being used, and to begin to identify patterns among items. Phase II of the project is just about to be launched and involves a quality assessment of the items in the repository. To accomplish this task, we need a quality assessment tool for ESM items, but because research is rather like Hal changing a lightbulb (https://youtu.be/AbSehcT19u0), we first need to develop a quality assessment tool. Gudrun Eisele, PhD student in the Center for Contextual Psychiatry (CCP), is leading this work with fellow CCP PhD student Anu Hiekkaranta and they’ll be telling you more about that in the next blog post. In Phase III of the project, we will begin the psychometric validation of items within the repository, but we don’t just want to keep all the psychometric fun to ourselves; a key goal of the repository is also to facilitate other scientists to conduct ESM research and to psychometrically validate available items. We’re busy with plans for this phase of the project and will share more details soon.
Our main goals in establishing the repository are to increase the discoverability of ESM items, facilitate ESM research and to improve the transparency and quality of ESM research. Given that ESM is increasingly used in clinical psychology and psychiatry research, we also hope to make a contribution to transparency and reproducibility within those areas, which have often not been part of conversations around open science practices. For more on that topic, please read the brilliant work of Jennifer Tackett and her colleagues (Tackett, Brandes, & Reardon, 2019; Tackett et al., 2019). The rest of the fantastic ESM Item Repository team and I are thrilled that our efforts to increase the openness of ESM research have been recognised by two awards this year: a commendation from the Society for the Improvement of Psychological Science (https://twitter.com/improvingpsych/status/1247246158432526336?s=20) and an Open Research Award from the University of Groningen (https://twitter.com/Bibliothecaris/status/1311299779591385089?s=20). We’re not done yet though! There are lots more exciting things to come. A vital and ongoing task is continuing to build the collection of items. After all, the success of the ESM Item Repository depends on researchers contributing their items, so please consider sending us your items to include in the repository! To find out more information about the project and how to contribute items, check out the contributors pack on our OSF page (https://osf.io/kg376/).
Notes:
* Unfortunately, in some cases, the “rigorous psychometric validation process” for traditional self-report questionnaires has also been treated as more of a serving suggestion. I could write a whole other blog post about this (and maybe I will!), but let’s stick to ESM for now. Interested readers should definitely check out all the wonderful things that Eiko Fried and Jessica Flake have written about these issues (see references below).
References:
Csikszentmihalyi, M., & Larson, R. (1987). Validity and reliability of the Experience Sampling Method. Journal of Nervous and Mental Disease, 175(9), 526–536.
Flake, J. K., & Fried, E. I. (In press). Measurement Schmeasurement: Questionable Measurement Practices and How to Avoid Them. Advances in Methods and Practices in Psychological Science. Preprint: https://doi.org/10.31234/osf.io/hs7wm
Fried & Flake (2018). Measurement Matters. The Observer (Association for Psychological Science).
Horstmann, K. T., and Ziegler, M. (2020) Assessing Personality States: What to Consider when Constructing Personality State Measures. European Journal of Personality.
Kirtley, O. J., Hiekkaranta, A. P., Kunkels, Y. K., Eisele, G., Verhoeven, D., Van Nierop, M., & Myin-Germeys, I. (2020, October 1). The Experience Sampling Method (ESM) Item Repository. https://doi.org/10.17605/OSF.IO/KG376
Kirtley, O. J., Lafit, G., Achterhof, R., Hiekkaranta, A. P., & Myin-Germeys, I. (In press). Making the black box transparent: A template and tutorial for (pre-)registration of studies using Experience Sampling Methods (ESM). Advances in Methods and Practices in Psychological Science. Preprint: https://doi.org/10.31234/osf.io/seyq7
Myin‐Germeys, I., Kasanova, Z., Vaessen, T., Vachon, H., Kirtley, O., Viechtbauer, W., & Reininghaus, U. (2018). Experience sampling methodology in mental health research: new insights and technical developments. World Psychiatry, 17(2), 123-132.
Stone, A.A., & Shiffman, S. (1994). Ecological momentary assessment (EMA) in behavioural medicine. Annals of Behavioral Medicine, 16, 199-202.
Tackett, J. L., Brandes, C. M., & Reardon, K. W. (2019). Leveraging the Open Science Framework in clinical psychological assessment research. Psychological Assessment. doi:10.1037/pas0000583
Tackett, J. L., Lilienfeld, S. O., Patrick, C. J., Johnson, S. L., Krueger, R. F., Miller, J. D., . . . Shrout, P. E. (2017). It's Time to Broaden the Replicability Conversation: Thoughts for and From Clinical Psychological Science. Perspectives in Psychological Science, 12(5), 742-756. doi:10.1177/1745691617690042
Trull, T. J., & Ebner-Priemer, U. W. (2020). Ambulatory assessment in psychopathology research: A review of recommended reporting guidelines and current practices. Journal of Abnormal Psychology, 129(1), 56–63.
Wright, A. G. C., & Zimmermann, J. (2019). Applied ambulatory assessment: Integrating idiographic and nomothetic principles of measurement. Psychological Assessment, 31(12), 1467–1480. https://doi.org/10.1037/pas0000685
About the author: Olivia Kirtley is an FWO Senior Postdoctoral Research Fellow within the Center for Contextual Psychiatry at KU Leuven in Belgium, where she also leads “SIGMA”, a large-scale longitudinal study of adolescent mental health and development using experience sampling methods (ESM). Her current research uses ESM to investigate dynamic processes involved in ideation-to-action transitions in adolescents who self-harm. Olivia leads several projects aimed at increasing transparency and reproducibility in the ESM field, including designing a pre-registration template for ESM research and leading the ESM Item Repository.
