Where's The Data?

“In God We Trust. All Others Must Bring Data” - W. Edwards Deming

In November of 2023, the Pentagon failed its’ sixth-straight audit via the Government Accountability Office (GAO)¹. The results, according the the DOD, were “not unexpected”². In the audit, GAO found that the department had insufficiently documented 63%–2.39 trillion–of its 3.8 trillion in assets. The big takeaway of the audit? “Military contractors possess many of the government’s assets, but to an extent unbeknownst to the Pentagon”³. What that means–in effect–is that the government is consistently using taxpayer dollars to pay military contractors for goods that they already have.

The U.S government operates at a scale that can be difficult to comprehend. In 2024, total government spending amounted to 7.0 trillion dollars. With systems at this scale, small problems in underlying system architecture, scaled up, can have enormous impacts on the people who depend on that system. 

Right now, the government is having problems efficiently tracking how it spends its money. In 2016, the Office Of Management And Budget (OMB) implemented a new buying protocol for procured goods and services⁴. They called it, “Category Management”. An approach borrowed and vetted from the United Kingdom, Category Management was designed to “enable the federal government to buy smarter and more like a single enterprise”.

Category Management attempts to achieve two main objectives:

The first is to rate all contracts according to a system of 4 tiers. Tier 0 contracts are those that are very case-specific, and Tier 3 (the highest tier) are contracts that can be standardized and normalized across agencies. 

The second goal of Category Management, is to transition agencies away from using lower tier type contracts, and encourage them to use higher tier contracts. 

While up to this point, Category Management has been effective to some degree⁵, the rollout has been frictional and slow. For instance, most agencies' record-keeping systems “do not currently allow them to analyze the bulk of their contract obligations”. The reason?  “They do not reliably or easily capture prices-paid data”⁶. If you don’t track what you are spending on something, it may be very difficult to determine whether you are getting a good deal or not.

Data collection errors can often be mundane and technical in nature. They often creep into gaps in a product design process when relevant stakeholders fail to communicate with each other. One salient example from GAO’s 2020 report was particularly striking, “the data input field where contracting officers enter product and service codes allows the contracting officers to enter only one code, but agency officials told us that agencies often buy multiple products and services in a given transaction”⁷. Clearly, the relevant stakeholders in this case (contracting officers), were not consulted by whoever created that form. 

These types of errors are fairly endemic across government agencies. In a report on the progress of the GREAT act–a government effort to normalize data collection efforts across all federal agencies–GAO found that the government agencies had “partially fulfilled 1 of 3 deadlines related to data standards”⁸. The project had asked OMB to define a set of data elements that could be applied to the evaluation of grants, which would allow procurement officials to standardize their data collection and analysis efforts. When GAO took a random sample of the elements OMB submitted, the results were less than glamorous. Nearly half of the elements did not meet even one of the 13 requirements for effective data points, and  “501 of the 540 data elements were not machine-readable”⁹. And that is all the progress OMB made in half a decade of trying. 

While Federal Agencies clearly have room left for improvement in the collection and management of data, the intent to improve is certainly there, and some progress has been made, in spite of setbacks. With a focus on innovation and Human Centered Design gaining relevance among public sector circles, we can expect improvement to accelerate over the coming years.