In today’s fast-paced world, the promise of AI-driven efficiency is intoxicating. From previously waiting for months to cobble together the opinion of a handful of healthcare professionals (HCPs) to being able to tap into thousands of “opinions” instantly, without the complexities of traditional research; synthetic panels, powered by Large Language Models (LLMs), offer just this allure. A seemingly endless wellspring of insights at your fingertips.

But what if this apparent shortcut to truth is leading us down a dangerous path? What if some synthetic insights, are far from revealing true HCP needs, and are merely creating a sophisticated echo chamber that amplifies existing biases and distorts reality?

How Synthetic Data Can Lead You Astray

At the heart of our research ambition is to build accurate pictures of the real world. Real human opinions are diverse, nuanced and sometimes contradictory. There is an inherent messiness in real human data, and it is exactly this set of disparate opinions that we love to tap into it, to understand the true spread of opinions in a society. When synthetic outputs are treated as genuine reflections of human sentiment, without rigorous validation we can step into realm of unreliable insights.

One of the most critical issues with synthetic data is that conventional techniques can inadequately capture the rich complexity and diversity of real-world scenarios (Giuffrè and Shung, 2023). Creating a world where LLMs provide:

• Generalised approximations
• Lack nuanced and variable responses
• Miss the fringes of human thought and experience

This is only further exacerbated when considering potential biases in datasets that LLMs are being trained on.

LLMs learn from the vast oceans of data they are trained on. Unfortunately, much of this readily accessible data carries historical biases, incompleteness, or even misinformation. If models are trained on such flawed data sets their synthetic outputs will not only replicate these flaws but can amplify them, creating a sophisticated echo chamber that confirms existing biases rather than revealing true sentiment.

Within a health environment many of us are already aware of gaps within historical health data. From disproportional focuses on certain demographics, or biases within specific conditions (Chen et al,2023).

For example, in one study covering over 8,500 tumour samples across 33 cancer types they found that 82% of all cases in the dataset were from white patients, 10.1% from black people, 7.5% were from Asians and 0.4% were from highly under reported minorities. (Gao, 2013). This is just one example out of many that shines a light on the challenges in historic health data. Due to limitations in historical data sets we have an increased risk in underrepresenting or completely missing critical issues for marginalised groups.

To add another layer to this problem, it’s not just the data sets itself where bias can exist, but we also encounter challenges based on what data AI models prefer to showcase. LLMs have tended to show a preference in showcasing data generated by Ai over human-authored content. When this happens at scale it has the potential to create a “model collapse” a degenerative process where generated data pollutes the training set for the next generations of models. Leading to a situation where probable events are over-estimated and improbable events underestimated. (Shumailov et al, 2023). This AI-AI bias raises a question: if our research becomes primarily guided by AI-Generated insights and Ai systems inherently prefer Ai-generated content, are we creating a self-reinforcing loop that inadvertently devalues genuine human inputs? For brand managers this means decisions made on synthetic insight alone could reinforce historic bias, or simply be built on Ai created generalisations, leading to communications that fail to resonate with real people.

Beyond the Echo Chamber: The Indispensable Value of Authentic Human Insight

So, what’s the answer? There is clearly a need, a desire for fast actionable insights that is driving uptake and use cases of synthetic data panels. The core problem that sits at heart of commonly adopted approaches is a limited data foundation and lack of methodological rigour. We as an industry need to apply an added layer of care and be aware of the limitations of public LLM models being applied to the healthcare sector.

Beyond this awareness there are steps we can do to help improve synthetic outputs.

• Start with real-world data: Use verified HCP/ patient datasets as the foundation for synthetic modelling.
• Run bias audits: Identify gaps in representation and adjust inputs accordingly
• Combine verified data sets: Utilise multiple sets of data to help offset singular data source limitations – while continuing to apply checks for bias
• Measure divergence and verify with humans: Calculate the delta between synthetic and human responses. By setting specific thresholds across outputs, we can better identify synthetic viewpoints that require a re-work or should be discarded
• Educate stakeholders: Ensure everyone understands the risks as well as the benefits of using AI in research.

AI can be a powerful assistant, but it must be used responsible, complementing the irreplaceable insights derived from authentic human voices. To truly understand our audiences and address complex health challenges, we must prioritize methodologies that continue to connect us with the real world, not merely its synthetic reflection.

References

Chen RJ, Wang JJ, Williamson DFK, Chen TY, Lipkova J, Lu MY, Sahai S, Mahmood F. Algorithmic fairness in artificial intelligence for medicine and healthcare. Nat Biomed Eng. 2023 Jun;7(6):719-742. doi: 10.1038/s41551-023-01056-8. Epub 2023 Jun 28. PMID: 37380750; PMCID: PMC10632090.

Gao J et al. Integrative analysis of complex cancer genomics and clinical profiles using the cBioPortal. Sci. Signal 6, pl1 (2013)

Giuffrè, M., Shung, D.L. Harnessing the power of synthetic data in healthcare: innovation, application, and privacy. npj Digit. Med. 6, 186 (2023). https://doi.org/10.1038/s41746-023-00927-3

Shumailov, E., Shumaylov, Z., Zhao, Y., Belliveau, N., Priest, B., Kazhdan, D., Mahoney, M. W., Kandylas, V., & Kuratov, Y. (2023). The Curse of Recursion: Training on Generated Data Makes Models Forget. arXiv preprint arXiv:2305.17493.