Thanks, Human Factors
Written by Marc Egeth
September 25, 2016.
Sitting in the Philadelphia airport waiting for a flight through London to Berlin. I checked a manikin. A “m-a-n-i-k-i-n” is a humanoid medical simulation, a “m-a-n-n-e-q-u-i-n” is a dress form. Core was hired to study a medical device in development for marketing in the U.S. and E.U., and the best way to make sure real-life users in the E.U. will understand and be able to use the device is to test it there with study participants carefully selected to be representative of real-life users. The best way to conduct such a test is with the careful attention of an experimenter – in this case, yours truly – who has been developing the study protocols and testing the device in similar ways in the U.S. I don’t speak German, so we have hired a local “moderator” to conduct study sessions and also a live translator. Study sessions will consist of study participants being asked to approach this new medical device and use it, as if for real – but instead of “actually” using it to deliver an injection to someone who needs it, instead deliver it to Bert, our manikin.
Traveling by air to conduct human factors testing of medical devices carries a certain irony, or at least resonance, as the field of human factors is widely credited as beginning in aviation. Understanding the precise capabilities – and limits – of human action, perception, attention, and endurance was necessary to design cockpits that were usable by pilots to hurl hundreds of tons of glass, steel, and cargo (bombs, troops, what have you) thousands of feet into the air and stay aloft avoiding obstacles for thousands of miles, ultimately landing safely 1.
The fact that it is far safer to fly (.5 deaths/billion miles) than to walk (41 deaths/billion miles) is due to the decades-long rigorous applied scientific study of human performance to design realistically usable flight interfaces. If adequately invested, the same principles that make flying safe, efficient, and useful will make medical devices safe, effective, and as much of a joy to use as they can be. However, the catastrophe and image of an airliner crash draws attention to the dangers of flying that must be mitigated in vivid manner unlike the day to day seepage of medical errors. “Flight life insurance” highlights the discrepancy between our fears and the reality of safety statistics – if anything, there should be a way to temporarily suspend your life insurance and get a small refund for the time you spend on a plane rather than walking or driving. That would be the safe bet in this case.
Will the study participants in Germany use the medical devices we are testing in a manner consistent with the special and safe flight statistics, or with the more pedestrian walking statistics? We are not employing a very large sample of participants – for one thing, the disease it treats is very rare – and for another, large samples with seemingly statistically “safe” patterns of use is not the best way to develop safe medical devices. Our research for this project is qualitative, meaning we are paying close attention to what happens, not how frequently it happens. We will experimentally set up a hazard-laden, rare scenario, and test if – make sure that – people are able to use the medical device safely. The standard the FDA holds by is that if even one study participant comes close to making a critical error, the manufacturer should take measures to redesign the device to help users avoid those errors. It does not matter if 50%, 90%, or fewer than 1 out of 20 test participants make such an error – the design process should still work to fix it.
September 29, 2016.
Sitting on a plane from Germany to Ireland, where I will have a night-long layover on the way back home. How did the device do? Because in this blog, we do not discuss client-confidential information, I will answer how well the study logistics did. We let about 20 people representative of potential real-life users of the medical device pretend to use a prototype of the device. They did this while we watched them from behind a 1-way mirror. We impressed upon them as best we could to act as if they were in a real-life situation in which they had been prescribed this new device. We simulated the training they would receive in real life. Because people have time to forget what their doctor trained them on by the time they actually use their prescribed device, we had participants wait for a similar delay before they pretend to use the device. We hired a local interviewer, and we practiced the scene before seeing the first study participant.
Study sessions typically combine elements of bench testing a product, a psychology experiment, improvisational acting, neutral and unbiased focus-group-type interviews, and producing a documentary. It’s fun! And boring, and tedious.
And the potential real-world effect of this odd amalgam of efforts?
Well, I feel strangely safe and secure hurtling through the air a mile high and 600 miles per hour in the dark, typing on this super-sleek on-fleek MacBook Air like some sort of nonchalant tech-loving superfast pigeon. So next time maybe I’ll do it sporting a brand-new life-enhancing safe and effective medical device that I just love to use. Thanks, Human Factors.
1I just got a slight thrill of transgression typing the word “bomb” in an airport. And now doubly so making a joke of it, the forbidden thing itself.