The National Highway Traffic Safety Administration’s efforts to identify and probe vehicle safety concerns have been persistently hampered by deficiencies in data analysis, staff size and training, use of regulatory authority over automakers, and a host of other problems, according to a newly released government audit.

The audit report, released by U.S. Department of Transportation Inspector General Calvin L. Scovel III, is unsparing in its criticism of NHTSA’s Office of Defects Investigation (ODI). U.S. Transportation Secretary Anthony Foxx requested the report in the wake of the General Motors ignition switch recalls in 2014. NHTSA is an agency within the DOT.

The report notes that ODI first looked at GM air bag non-deployments as a potential safety issue in 2007. Also, the office received data on the ignition switch defect as early as 2003. But in 2007 staff members decided against investigating the problem and failed to identify the ignition switch defect as the root cause of the air bag failures. More than 110 fatalities and 220 injuries have been linked to the defective ignition switches.

“ODI lacks the procedures needed to effectively identify safety defects that warrant an investigation,” the report states. “Specifically, ODI has not developed guidance for applying the factors it established for opening an investigation. In addition, the factors that influence ODI’s decisions on whether to open an investigation are not transparent, and it is unclear who is accountable for these decisions. This was the case with ODI’s decision not to investigate the GM air bag non-deployment defect.”

According to the report, ODI staff members who screen consumer complaints and analyze crash data often lack adequate technical training to conduct proper research and testing before opening investigations. This can result in potential safety defects being overlooked.

The ODI neither follows standard statistical practices when analyzing crash data from manufacturers, nor thoroughly screens consumer complaints, the report noted.

“For example, ODI’s initial screening of the roughly 330 complaints received daily is not thorough, and about 90 percent of complaints are set aside,” the report said. “While screeners are encouraged to query all complaints for similar issues in their area of concentration, half of them told us that they do not consistently do this.”

Such systemic weaknesses led ODI to overlook a GM-provided state trooper report suggesting that a faulty ignition switch in a Chevrolet Cobalt might have caused air bag failure in a 2007 fatal collision, the report asserts.

The audit report is particularly critical of ODI’s lack of transparency in its decision-making.

“Of the 56 investigation proposals for light vehicle safety defects in 2013, 32 were not investigated – 18 of which lacked documented justifications for not investigating,” the report said. Because staff screeners fail to learn what management deems worthy of investigation, they rely on precedents. As a result, past investigative mistakes aren’t likely to be revisited and corrected.

Additionally, according to the report, ODI fails to verify that manufacturer-provided crash data – including injury and death reports – are complete and accurate. Even when the office suspects non-compliance, officials don’t take prompt enforcement action. Variation in how manufacturers code and categorize data also contributes to investigatory lapses.

Given the volume of consumer complaints and crash data received, the ODI staff’s limited size also hinders its effectiveness. ODI consists of eight defect screeners, four early-warning data analysts and 16 investigators. The Obama administration has pushed for raising ODI’s budget to $31.3 million – approximately triple the current level.

The report concludes with 17 recommended actions to improve ODI’s collection, screening and analysis of vehicle safety data. NHTSA Administrator Mark Rosekind, a relative newcomer to the agency who took over the position last December, has agreed to implement all 17. Under Rosekind’s leadership, NHTSA earlier this month released its own report proposing sweeping reforms within the agency.

The 17 recommendations included in the DOT inspector general report are:

1. Develop and implement a method for assessing and improving the quality of early warning reporting data (crash data from manufacturers, including injury and death reports).

2. Issue guidance or best practices on the format and information that should be included in non-dealer field reports to improve consistency and usefulness.

3. Require manufacturers to develop and adhere to procedures for complying with early warning reporting requirements, and require ODI to review these procedures periodically.

4. Expand current data verification processes to assess manufacturers’ compliance with regulations to submit complete and accurate early warning reporting data. At minimum, this process should assess how manufacturers assign vehicle codes to specific incidents and how they determine which incidents are reportable.

5. Develop and implement internal guidance that identifies when and how to use oversight tools to enforce manufacturers’ compliance with early warning reporting data requirements.

6. Provide detailed and specific guidance to consumers on the information they should include in their complaints, as well as the records they should retain (such as police reports and photographs) in the event that ODI contacts them for more information.

7. Develop an approach that will determine which early warning reporting test scores provide statistically significant indications of potential safety defects.

8. Periodically assess the performance of the early warning reporting data tests using out-of-sample testing.

9. Institute periodic external expert reviews of the statistical tests used to analyze early warning reporting data to ensure that these methods are up-to-date and in keeping with best practices.

10. Implement a supervisory review process to ensure that all early warning reporting data are analyzed according to ODI policies and procedures.

11. Develop and implement a quality control process to help ensure complaints are reviewed thoroughly and within a specified timeframe.

12. Update standardized procedures for identifying, researching and documenting safety defect trends that consider additional sources of information beyond consumer complaints, such as special crash investigation reports and early warning data.

13. Document supervisory review throughout the pre-investigative process, including data screening.

14. Evaluate the training needed by pre-investigative staff to identify safety defect trends. Also, develop and implement a plan for meeting identified needs.

15. Develop and implement guidance on the amount and type of information needed to determine whether a potential safety defect warrants an investigation proposal and investigation.

16. Develop a process for prioritizing, assigning responsibility, and establishing periodic reviews of potential safety defects that ODI determines should be monitored.

17. Document and establish procedures for enforcing timeframes for deciding whether to open investigations. Also, establish a process for documenting justifications for these decisions.

 _________________________________________________________________________

Related: Safety Team to Help NHTSA Improve Defect Probes

_________________________________________________________________________