A collective autopsy of chart crimes, via the scientific method

We've all seen it: there is a propensity for graphs to be perceived as showing accurate data, yet somehow, lead us to wrong conclusions. For example, bar charts with truncated axes that emphasise a 2% difference in a measure, or a correlation that vanishes when one breaks down the data by subgroups, or a trend line that hides more than it reveals.

I'd like to bring together a group of participants through this dialogue to examine "chart crimes", which visualisations mislead (whether unintentionally or by virtue of their design), and counter them by using the scientific method applied to the practice of plotting. We'll build a shared vocabulary of common patterns that can aid plots in deceiving us as humans, i.e., as poor logicians who tend to over-rely on heuristics.

Some specific themes I have in mind are: the ecological fallacy, Simpson's paradox, base rate neglect, dredging, cherry-picking, the premise of survivorship, and more. Particularly, plots in reports, newspapers, research pieces, and academia appeal to authority, making it easier for people to trust them. (I'd appreciate help refining these further!).

Why does this matter? Charts are persuasive precisely because they appear objective. When we see data visualised, we assume rigour – that someone has done the work and checked the numbers. This makes misleading visualisations dangerous, especially in contexts like public health, climate reporting, or policy debates where stakes are high, and most readers lack the time, training, or will to scrutinise every plot they encounter.

The problem isn't of malice (of course), as most misleading charts come from well-intentioned people making common mistakes. However, intent doesn't change impact. A poorly chosen axis scale or an overlooked subgroup can shift understanding, influence decisions, and spread disinformation that we must not be as easily prone to. Much of the discussion right now, in my view, has centred on how to make better charts with clearer hierarchies, better colour choices, and stronger narratives. However, as a contrarian, I'd like to focus on the inverse question: what makes charts fail? What are the systematic ways visualisations mislead, and how do we spot them? I think of this dialogue as the other half of that conversation.

Structure of the session should be roughly an hour split into three parts. I can start us with some striking examples of plots that look fine in their guise. The next segment will walk through the vocabulary: for each concept, I'll show what it looks like visually, why it happens, and what cues to to use to help build our pattern recognition.

My idea so far is to get participants to examine scientific visualisations from news, research, and reports (all pre-decided out of a jumbo box of examples I'll bring) and perform collective work, after which findings and diagnostics can be shared.

I'd like to keep this open to anyone who creates or consumes data visualisations: journalists, analysts, designers, researchers, students, policy professionals and will spare requiring a statistics background.

My proposed (and hoped-for) learning outcome is that participants will ask sharper questions about plots using the scientific method and will be able to distinguish between visualisations that deceive and those that don't. They should be able to consider these pitfalls in their own work. My goal is not of cynicism but of informed scepticism.