This reluctance to display any effectiveness to the TSA programs seems to run through the entire program. When I had a chance to question Paul Leyh, the TSA director of the Secure Flight program, he would not even offer a single instance of a terrorist arrest during the period that TSA has been screening passenger names.
Readers can download the unclassified GAO report. Read it and weep.
Here are some tidbits from the GAO report. Note: These are official U.S. government report statements, not my own opinions.
We don’t know of the behavior detection system works. (Page 2)
A scientific consensus does not exist on whether behavior detection principles can be reliably used for counterterrorism purposes, according to the National Research Council of the National Academy of Sciences. According to TSA, no other large-scale security screening program based on behavioral indicators has ever been rigorously scientifically validated. DHS plans to review aspects of SPOT, such as whether the program is more effective at identifying threats than random screening. Nonetheless, DHS’s current plan to assess SPOT is not designed to fully validate whether behavior detection can be used to reliably identify individuals in an airport environment who pose a security risk.
TSA is throwing money into a program that has displayed no cost-benefit for the program, however continues to expand the program. (Page 30)
TSA has not conducted a cost-benefit analysis, which could help the agency establish the value of the program relative to other layers of aviation security. Moreover, a cost-benefit analysis could also be useful considering recent program growth.
TSA has no way to asses the effectiveness of the program. They have not reported a single terrorism arrest since implementation of the program. (Page 43)
TSA … lacks the measures needed to evaluate the effectiveness of the SPOT program and, as a result, has not been able to fully assess SPOT’s contribution to improving aviation security. … The SPOT program uses teams to assess BDO proficiency, provide individual and team guidance, and address issues related to the interaction of BDOs with TSA checkpoint personnel. However, TSA does not systematically track the teams’ recommendations or the frequency of the teams’ airport visits.
Almost all domestic airlines now have a direct electronic link to the the Terrorist Screening Database through Secure Flight, but TSA still uses the phone. (Page 42)
… the Transportation Security Operations Center is not using all the resources at its disposal to support BDOs in verifying potential risks to the aviation system. This reduces the opportunities to “connect the dots” that would increase the chances of detecting terrorist attacks in their planning stage, which the SPOT Privacy Impact Assessment states is when they are the most vulnerable.
… TSA told us that the Operations Center does not have direct electronic access to the Terrorist Screening Database and must call the FBI’s Terrorist Screening Center to provide it with a name to verify. TSA stated that this is done if a passenger’s identity could not be verified using the Operations Center databases.
TSA sends out standardization teams (inspectors) but has no system for tracking suggestions. (Page 52)
… standardization teams provide individual and team guidance to the BDOs, offer assistance in program management, and cover issues related to the interaction of BDOs with other TSA checkpoint personnel.
TSA reported to us that it does not systematically track the standardization teams’ recommendations or the frequency of the teams’ airport visits.
Title photo from Centers for Disease Control and Prevention’s Public Health Image Library. All charts from GAO report GAO-10-763.