❌Pitfalls of Data-Driven Decision Making (Series): Data Integrity

The question is simple: At what speed did the car accident happen?

The method is simple: Using the data sourced through surveys.

𝑩𝒖𝒕 𝒊𝒔 𝒕𝒉𝒂𝒕 𝒓𝒆𝒂𝒍𝒍𝒚 𝒆𝒗𝒆𝒓𝒚𝒕𝒉𝒊𝒏𝒈?

In this groundbreaking 𝑳𝒐𝒇𝒕𝒖𝒔 𝒂𝒏𝒅 𝑷𝒂𝒍𝒎𝒆𝒓 𝒔𝒕𝒖𝒅𝒚, they delve into the power of question wording on eyewitness accounts.

The study exposed participants to the same event, a car crash, but with varied questions.

7 films of traffic accidents, ranging in duration from 5 to 30 seconds, were presented in a random order to each group.

After watching the film participants were asked the question About how fast were the cars going when they (smashed / collided / bumped / hit / contacted) each other?”

𝐓𝐡𝐞 𝐩𝐚𝐫𝐭𝐢𝐜𝐢𝐩𝐚𝐧𝐭𝐬 𝐰𝐡𝐨 𝐰𝐞𝐫𝐞 𝐚𝐬𝐤𝐞𝐝 𝐭𝐡𝐞 “𝐬𝐦𝐚𝐬𝐡𝐞𝐝” 𝐪𝐮𝐞𝐬𝐭𝐢𝐨𝐧 𝐫𝐞𝐩𝐨𝐫𝐭𝐞𝐝 𝐭𝐡𝐞 𝐡𝐢𝐠𝐡𝐞𝐬𝐭 𝐬𝐩𝐞𝐞𝐝 𝐞𝐬𝐭𝐢𝐦𝐚𝐭𝐞 (40.8 𝐦𝐩𝐡), 𝐟𝐨𝐥𝐥𝐨𝐰𝐞𝐝 𝐛𝐲 “𝐜𝐨𝐥𝐥𝐢𝐝𝐞𝐝” (39.3 𝐦𝐩𝐡), “𝐛𝐮𝐦𝐩𝐞𝐝” (38.1 𝐦𝐩𝐡), “𝐡𝐢𝐭” (34 𝐦𝐩𝐡), 𝐚𝐧𝐝 “𝐜𝐨𝐧𝐭𝐚𝐜𝐭𝐞𝐝” (31.8 𝐦𝐩𝐡).

This seemingly subtle shift in question framing caused distinct data outcomes.

It spotlighted how external influences, like phrasing, can shape our perception and thus, the data we collect.

𝐈𝐭’𝐬 𝐚 𝐫𝐞𝐦𝐢𝐧𝐝𝐞𝐫 𝐭𝐡𝐚𝐭 𝐰𝐡𝐢𝐥𝐞 𝐝𝐚𝐭𝐚-𝐝𝐫𝐢𝐯𝐞𝐧 𝐝𝐞𝐜𝐢𝐬𝐢𝐨𝐧-𝐦𝐚𝐤𝐢𝐧𝐠 𝐢𝐬 𝐯𝐚𝐥𝐮𝐚𝐛𝐥𝐞, 𝐞𝐦𝐛𝐫𝐚𝐜𝐢𝐧𝐠 𝐬𝐜𝐢𝐞𝐧𝐭𝐢𝐟𝐢𝐜 𝐭𝐡𝐢𝐧𝐤𝐢𝐧𝐠 𝐭𝐡𝐚𝐭 𝐩𝐚𝐢𝐫𝐬 𝐛𝐨𝐭𝐡 𝐝𝐚𝐭𝐚 𝐚𝐧𝐝 𝐜𝐫𝐢𝐭𝐢𝐜𝐚𝐥 𝐭𝐡𝐢𝐧𝐤𝐢𝐧𝐠 𝐢𝐬 𝐭𝐡𝐞 𝐫𝐞𝐚𝐥 𝐬𝐨𝐥𝐮𝐭𝐢𝐨𝐧 𝐭𝐨 𝐭𝐨𝐝𝐚𝐲’𝐬 𝐫𝐞𝐚𝐥 𝐰𝐨𝐫𝐥𝐝 𝐜𝐨𝐦𝐩𝐥𝐞𝐱 𝐩𝐫𝐨𝐛𝐥𝐞𝐦𝐬.

Source: https://webfiles.uci.edu/eloftus/LoftusPalmer74.pdf