Discussion
generalised, annoyance, NMS
NMS, EVENT
In our 2020 NMS, an RCT of a paper survey and a push-to-web survey and a concurrent social media survey were conducted to compare different combinations of sampling and data collection methods with respect to response rate, respondent representativeness, external validity of prevalence estimates of selected maternity indicators, and financial costs.The response rate was 3.5% higher in the standard paper survey compared to the push-to-web survey, in which women were encouraged to complete the questionnaire online. The difference was small but statistically significant and, with the current downward trend in response rates to surveys, even small improvements are important. The current findings, therefore, do not provide support for the use of push-to-web methods over standard paper methods based on response rates alone. The findings are in line with those of a recent US study which evaluated the effectiveness of push-to-web methods in a survey of new mothers and found that a traditional mailed survey yielded the highest response rate [Non-response bias in the surveys recruiting through ONS and self-selection bias in the social media survey meant that the respondents were not representative of all women giving birth in England during 2020 on key sociodemographic characteristics, such as age, ethnicity and level of deprivation. The women in the social media survey were even less representative of the target population on these sociodemographic characteristics compared to the women in the paper or push-to-web surveys. Therefore, these findings suggest that self-selection in surveys which recruit through social media may pose a greater threat to sample representativeness than high levels of non-response in surveys which recruit through defined sampling frames. This is consistent with systematic review findings that non-probability samples are less likely to be representative of the target population [Weights were calculated and applied to the data in each survey to reduce the effect of non-response and self-selection bias, yet there was still bias in the survey estimates for some maternity indicators, when compared with population-based estimates available from national routine data. This may be due to the survey weights being based on limited sociodemographic variables, which do not fully explain the complex array of reasons for why women may have not responded to the survey [The cost of recruiting through ONS was significantly higher than recruiting through social media. Compared to the paper survey, the standard method used in previous NMS, the social media survey offered a cost saving of more than twenty pounds (UK£) per respondent, which amounts to a substantial saving in large surveys. Consistent with previous findings [In addition to the potential cost saving, other advantages of social media surveys include the avoidance of coercion and intrusion, which can be inherent in survey invitations received via post. The self-selection sampling approach is potentially advantageous at a time when people may be feeling increasingly bombarded with requests for personal information. Related to this is the feasibility of conducting social media surveys without the requirement for any personal information, which avoids the potential annoyance caused when invitation recipients are unclear about how their details have been accessed. An additional advantage of surveys carried out solely online is the lower environmental impact, which is an increasingly recognised outcome.There are also disadvantages of social media surveys. Although there are examples of surveys which have recruited very large samples through social media, such as a recent Spanish study on the impact of Covid-19 on the entire adult population, which achieved almost 142,000 responses within one week [Broader eligibility criteria may also increase the likelihood of obtaining heterogeneous samples. For example, in the current study the women recruited through social media gave birth over a six-month period whereas the women recruited though ONS gave birth during a two-week period. In a survey exploring maternity experiences during a fast evolving event like the Covid-19 pandemic, timing is critical. The rapid changes in guidelines and practice as more information became available are likely to have had an impact on women’s care [On balance, despite some clear advantages of social media surveys, there are also numerous limitations that should be taken into account when considering social media surveys as an alternative to traditional survey methods. The current findings have implications for other UK and international organisations who conduct large population-based surveys, and who may be considering reviewing traditional survey methods in light of the availability of more contemporary and seemingly cost-effective methods.The main strength of this analysis is that the three surveys were carried out concurrently with many consistent methodological components, which enables the sampling and data collection methods to be reliably compared. The inclusion of the RCT, which used ONS birth registration data to compare the standard paper and push-to-web surveys, is a particular strength and the large sample size in the RCT allows precise estimates to be calculated and the results to be generalised to other similar surveys. Furthermore, all surveys were sufficiently large to allow the representativeness of the respondents to be assessed. The main limitation of the analysis is that the variables used to construct the survey weights did not explain all of the non-response and self-selection bias [