Should We Worry About Problematic Response Behaviour in Social Media Surveys?

Social media advertising has become a common tool for participant recruitment, especially when targeting hard-to-reach or underrepresented populations. A common strategy to boost engagement is the use of social group cues – textual or visual references to religion, gender identity, or ethnicity. While these cues can enhance recruitment efficiency, their impact on response data quality remains poorly understood. This study investigates how social group cues affect problematic response behaviour in a Facebook-recruited survey on labour market experiences in Germany. Respondents were recruited via four distinct ad conditions: two referencing Muslim identity, and two with neutral framing. Seven indicators were used to assess response quality, capturing satisficing (e.g., speeding, non-differentiation) and potential misrepresentation (e.g., implausible entries, inconsistent metadata). Results show that social group cues, particularly when targeting Muslim men, are associated with elevated rates of problematic response behaviour. Latent class analysis reveals three behavioural profiles: attentive respondents, likely misrepresentation, and likely satisficing. These groups differ in both sociodemographic patterns and attitudinal responses. Although problematic cases distort sample composition, sensitivity analyses confirm that core associations remain robust. The findings contribute to ongoing methodological debates on risks and opportunities of social media recruitment and offer practical insights for quality-conscious recruitment strategies.

doi: 10.13094/SMIF-2026-00015 Open Access