Sense about sensors
28 September 2016 by Tom Marshall
Commercial pollution monitors are attracting plenty of hype, but scientists say buyers should beware. Tom Marshall explains.
New kinds of smart sensor are hitting the market, and they could transform how we cope with health risks from polluted air by putting information at our fingertips. Asthmatics could stay indoors when sensors show lots of particle pollution; urbanites might skip their evening jog if nitrogen dioxide levels are high.
There's just one problem - most of these devices are unreliable and some are practically useless, according to the scientists who tested them. Until manufacturers are more open about the accuracy and limitations of sensors, they're little more than a curiosity and should not be used to make decisions about health.
"Like many atmospheric scientists I'd heard about these sensors and was rather dismissive," says Professor Ally Lewis of the University of York, deputy director of NERC's National Centre for Atmospheric Science. "But we were unsure how well they'd perform, so we started testing them in the lab. The results weren't very impressive."
Measuring atmospheric pollution is hard. You often need to find just one molecule of gas in every one billion in air, while managing other factors that could distort the result, like weather or other pollutants. There's a reason atmospheric chemists and those responsible for meeting legal obligations on pollution use big, high-powered lab equipment and not convenient handheld metres.
Most of the air quality sensors on the market rely on old technology re-purposed from products like fire alarms or car exhaust sensors. Few were designed to monitor air pollution to protect health. "Often it's not clear what performance people should expect from their sensor," Lewis says. "Sometimes the marketing does suggest they're just for fun, but manufacturers can't control how the data will be used. Ultimately most people are interested in air quality for health reasons - and data from these devices could well be for used for health decision-making and medication."
But sensor data isn't very reliable. Many sensors reacted inconsistently to a given input. They can respond as much to humidity, temperature or other gases like hydrogen or carbon dioxide (CO2) as to the pollutants. The team put 20 identical ozone sensors on a roof and tested them; the highest and lowest readings varied by a factor of six.
This wouldn't be so troubling if sensors made this uncertainty clear. If users knew they were accurate to within a factor of 50, they'd know not to trust them. Failing to provide this information is a problem because dealing with invisible pollution means we have no clear sense of what the right result might look like, making it hard to know when it is wildly inaccurate.
Lewis says some of the sensors' technology could be improved, but trying to turn scientific instruments into hand-held devices is hard. Many of the instruments he uses in the lab can't shrink beyond a certain point because of the basic physics of how they work. "Our instruments get better, but they don't tend to get any smaller," he notes.
Lewis argues scientists should do more to test these devices and tell the public about their strengths and limitations. They should report findings even when they find no evidence a device works - failure to do this is a big part of why there's so little objective information to counterbalance the hype. Sensor labelling should be clearer about reliability and what they should be used for, and there should be a scheme to ensure they meet minimum standards of accuracy.
The test results appear in Faraday Discussions DOI: 10.1039/C5FD00201J.
They are discussed in a comment piece in Nature DOI:10.1038/535029a.