What Waterfall Development Taught The FBI Post 9/11

In March 2010, The Federal bureau of Investigation dropped its most significant modernisation project, a project designed to prevent another attack like those on 9/11. The FBI had been trying to update their computer systems for a decade and once again the most recent initiative had failed. Jeff Johnson arrived at the FBI in 2009, encouraged by the CIO, Chad Fulgham, a previous colleague at Lehman Brothers, and now he found himself in charge of clearing up this mess.

Jeff had been Assistant Director of the IT Engineering Division, with a big office in the J. Edgar Hoover Building overlooking the Washington Monument. Despite this lofty position Jeff spent much of his time in the basement attempting to salvage a project everyone saw as already sunk.

After 10 years and hundreds of millions of dollars the plug was regrettably pulled with the decision being to bring the project in-house to be “done and done well”.

The project was a computer system to replace the filing of paper reports in the modern era of Facebook, Twitter, Amazon, and Google. The typical special agent trying to get anything done, from paying an informant to filing a report on a bank robber, continued using the process their peers had been using 30 years earlier. Writing up a document, printing 3 copies; one sent up the approval chain, a second stored in case the previous copy got lost, and a third used to manually index keywords into the database.

Approval was sent in an equally slow and dated process that was regularly blamed for a systemic failure to maintain a clear picture. Analyst’s for the 9/11 Commission couldn’t even get access to the information due to the poor state of the FBI systems, which could not be relied on to scale beyond small groups of individuals. Many factors were identified as to why there had never been a terrorism threat assessment carried out but the report singled out “woefully inadequate” technological sophistication.

The FBI lacked the ability to know what it knew: there was no effective mechanism for capturing or sharing its institutional knowledge.

The FBI responded with Virtual Case File (VCF), intended to ensure the failings of the past would remain there. They had already budgeted $100 million dollars and just needed another $70 million to see it through.

The FBI spent $170 million dollars on a computer system program that was dropped 3 years later before anyone could ever use it. Not to worry, in 2005 the FBI announced Sentinel, a $451 million dollar replacement designed to go live by 2009. All the budget controls and procedures missing years earlier had been taken into account to ensure successful delivery.

By 2010 Lockheed Martin had delivered half the project at a spend of $405 million dollars. Independent analysis proposed the project would require a further 8 year and $350 million dollars to complete.

What Went Wrong?

It is all too easy to point the finger at possible causes like the wrong people or technology, but the reason these programmes failed is in the way people work.

The people at Lockheed Martin spent months preparing to bid on the contract, looking over the requirements and planning the system architecture. They spent months planning how to do the work from beginning to end, producing beautiful Gantt charts to reassure everything would go smoothly.

The Gantt chart laid out every milestone and  delivery date, a complete picture of the future designed to assure the FBI that time, budget, and scope would be met. The problem with Gantt charts it they are always wrong.

No person can predict the future, and with so many possibilities to derail a prediction the odds are that the prediction will be wrong. In the event a Gantt chart is proven right, chance is a wonderful thing.

Gantt charts appeared prior to WW1 and were used in 1910 by General William Crozier, they have since been carried through to modern day project management and unfortunately the stubborn nature of trench warfare and “over the top” tactics remain popular.

Lies and fabrications hidden behind colourful charts may win contracts, but responding to change over following a plan is realistic and will successfully deliver desired outcomes on time and in budget.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s