Measurement Uncertainty and Error Rates
Any scientific measurement has some error associated with it. The concept of "measurement uncertainty" means that for even the most carefully performed measurement, the value of the thing being measured can never be known exactly - only an estimated value can be given. Measurement uncertainty and error rates can come into play in forensic science whenever a numerical measurement is made, for example with blood or breath alcohol levels or even in measurements such as IQ scores.
This figure can be used to understand the concept of measurement uncertainty:
Here, a BAC was reported as 0.080. The shaded blue area represents all of the possible actual BAC levels and the probability that that measurement is the actual value.
An "error rate" is often used to address systematic error and insure the results reported account for uncertainty in measurement. Without an error rate, a lab leaves the inference that the test result is an absolute or true result.
Standards and Recommendations
ISO 17025 is an international standard for testing and calibration of laboratories and is the accreditation standard that the NCSCL lab is currently working to achieve. ISO standard 5.4.6 requires labs to estimate uncertainty of measurements.
Additional support for requiring measurement uncertainty to be calculated comes from the 2009 Strengthening Forensic Science in the United States: A Path Forward which states "[f]ew forensic science methods have developed adequate measures of the accuracy of inferences made by forensic scientists. All results for every forensic science method should indicate the uncertainty in the measurements that are made, and studies must be conducted that enable the estimation of those values." (See p. 184.)
The Scientific Working Group for the Analysis of Seized Drugs (SWGDRG) states that "an understanding of uncertainty [is] fundamental to the interpretation and reporting of results." See additional SWGDRG recommendations on Quality Assurance and Uncertainty here.
Articles and Presentations
- Recent news articles - this page contains links to recent press coverage of local and national cases involving measurement uncertainty or error rates and is updated regularly.
Attorney Ted Vosk has authored articles and presents nationally on this topic. Below are links to several of his articles and presentations:
- DWI: Trial by Numbers - Nov 2010 Champion Magazine article by Ted Vosk - provides an explanation of measurement uncertainty that is geared toward lawyers, but explains the math and statistics behind the concept.
- Introduction to Uncertainty - a 2011 manuscript prepared by Ted Vosk which explains to attorneys the concept of measurement uncertainty.
- Forensic Metrology: A Primer for Lawyers, Judges, and Forensic Scientists - This 153-page book is based on one of Vosk's textbooks. Teaches the concept of measurement uncertainty to an audience of lawyers and scientists.
- Metrology: A Knowledge Base for Communication and Understanding - 2010 PowerPoint by Ted Vosk which explains several concepts related to measurement including validation, calibration, uncertainty. Includes helpful diagrams and sample reports that demonstrate the concepts presented.
- Two additional articles by Ted Vosk on related topics are, DUI Defense which focuses on whether standardized field sobriety tests are within the realm of lay understanding and the significance of not administering them according to NHSTA standards and City of Fircrest v. Jensen: Due process and science by legislative decree which discusses due process challenges to the accuracy and reliability of breath alcohol testing.
Motions and Orders
People v. Jabrocki - 79th District Court for the County of Mason, Michigan - May 6, 2011 opinion holding that "calculation of an uncertainty budget or error rate and the reporting of the same is an essential element of the scientific methodology for analyzing blood alcohol content using gas chromatography" and denying the prosecution's Motion to Admit the blood test results.
State v. Fausto - Sept. 21, 2010 Order Suppressing Defendant's Breath-Alcohol Measurements in the Absence of a Measurement for Uncertainty, District Court of King County, WA
State v. Weimer - March 23, 2010 Memorandum Decision on Motion to Suppress, Snohomish County District Court, Washington holding that to allow the numerical value of blood alcohol tests without stating a confidence level violates Rule of Evidence 403 because the probative value of the evidence is substantially outweighed by its prejudicial value.
City of Kent v. McDaniel - May 4, 2011 Order Suppressing Defendant's Breath-Alcohol Measurements in the Absence of a Measurement for Uncertainty