Naslov
Piloting longitudinal cohort surveys: Best practices and challenges

Autori
Babarović, Toni ; Blažev, Mirta ; Serracant, Pau

Vrsta, podvrsta i kategorija rada
Sažeci sa skupova, ostalo, znanstveni

Skup
European Survey Research Association (ESRA) 2021

Mjesto i datum
Zagreb, Hrvatska; online, 02-23.07.2021

Vrsta sudjelovanja
Predavanje

Vrsta recenzije
Međunarodna recenzija

Ključne riječi
piloting, pilot study, longitudinal, dress rehearsal

Sažetak
Pilot studies are a valuable tool to pinpoint the areas in which the main study could fail and should be improved. This is especially important in comprehensive large-scale longitudinal studies where piloting is considered as prerequisite and necessity. Despite their noted importance pilot studies are still under-utilised in large scale studies and reporting of pilots is often partial and incomplete. There is a lot of confusion among researchers what are good practices and what to avoid in piloting. Because of that, we wanted to emphasize good piloting practices so that researchers utilise piloting as a tool in research more often and in the best way. We identified ten large longitudinal birth cohort studies to provide a comprehensive overview of best practices and challenges in piloting approaches. The focus was put on birth cohort studies that follow many individuals born at a particular time through the course of their lives because of their complexity and essential need for piloting multiple waves due to developmental changes of children over time. Through our analyses, we identified regular occurrence of three types of pilots in such studies – pre-pilots, instruments pilots and dress rehearsals. All three types differ regarding sample size, sampling procedures, aim and desired outcomes. Thus, we recommend incorporating all three types in every wave before the main study. This is especially important in the first decade of the study where participants go through extensive developmental changes that require a lot of modifications in instruments, data collection methods etc. There is a lot of variability regarding sample sizes from study to study, but one characteristic is consistent in all studies: pre-pilots always have relatively smaller sample size than instruments pilots, and instruments pilots have relatively smaller sample size than dress rehearsals. Also, pre-pilots and instruments pilots most often incorporate convenient sampling procedures, while dress rehearsals sampling procedure always matches the one that will be implemented in the main study. We found that few studies did merge the pilot study data with the main study data, however, we would recommend keeping pilot study participants and data separate from the main studies because the two differ in many characteristics. One serious challenge that needs to be addressed in future studies is a lack of longitudinal samples in pilots. Although it is more challenging, incorporating longitudinal samples would be beneficial because it enables calculation of attrition rates from wave to wave that can indicate potential attrition difficulties in the main study. Also, there are no clear criteria based on which the researcher would conclude if the pilot was a success or failure. Thus, we recommend that before conducting the pilot, researchers state the desired objectives and outcomes of a pilot to properly evaluate its success. Finally, we urge researchers to report all pilot study information straightforwardly through the separate chapter in technical reports or user guides or even better, separate technical report. To have such information publicly available is a good way of identifying and implementing better piloting practices.

Izvorni jezik
Engleski

POVEZANOST RADA