Whither United States Air Force Academy?

From TaoSecurity

Thomas Ricks' post Does the Air Force Academy have ‘the least educated faculty’ in the country? inspired me to write this post. Mr. Ricks cited a story by Jeff Dyche, a former USAFA professor who cited a litany of concerns with the USAFA experience. I graduated from the Air Force Academy in 1994, ranked third in my class of 1024 cadets, and proceeded to complete a master's degree at Harvard in 1996. In my experience, at least in the early 1990s, USAFA faculty were as good, or better, than Harvard faculty. I considered the nature and volume of my graduate courses to be simple compared to my USAFA classes. When several fellow graduate students broke into tears after learning what the Harvard faculty expected of them, I couldn't believe how much easier the classes were going to be!

Rather than address points made by Ricks and Dyche, I prefer to focus on a theme that appears every few years: "why does the nation need service academies?" To provide one answer to this question, I'm going to draw on some lessons from a biography I'm reading called Grant by Jean Edward Smith. As you probably know, US Grant was a West Point graduate. According to Smith, during Grant's time as a cadet, West Point was one of only two schools in the nation that trained graduates as civil engineers. According to West Point in the Making of America,"

Following the example of the famous French engineering and artillery schools that the army had sent him to study, [Superintendent] Thayer made West Point America's national engineering school. West Point combined officer training with a highly technical undergraduate education...

Engineering itself became the army’s elite branch of service, the first choice by those who ranked highest in a graduating class. Lower-ranking cadets went to the cavalry, infantry, and other branches. West Point also became the nation's major source of civil engineers and of engineering educators. In the three decades before the Civil War, West Pointers as teachers, writers, and practitioners fostered science and engineering at Cornell, Harvard, Yale, and other colleges.

So, beyond just producing professional military officers, West Point met the country's exploding demand for civil engineers. As Smith says:

[A]s the nation moved westward, the demand for engineers grew steadily. For that reason, few of the young men who went to West Point did so with the intention of making the Army a career. It was no disgrace to resign from the service to take a better civilian position, and of the 1058 cadets who graduated from the academy between its inception in 1802 and 1839, only 395 remained on active duty.

I believe West Point's experience in the early 1800's could serve as an example for USAFA in this century. Few would advocate closing USAFA if it produced graduates with skills seldom found elsewhere, meeting another exploding demand. It seems to me that the skill most needed to help grow the nation is digital defense. This requirement takes many forms, including secure coding, infrastructure design/construction/operation, incident detection and response, forensics, threat intelligence and adversary characterization, malware analysis and reverse engineering, counter-threat operations, and related fields. With the proper leadership, faculty, and determination, USAFA could differentiate itself as the nation's premiere "cyber school," with an integrated curriculum focusing on securing cyberspace -- both in the military and government or private sectors.

This change doesn't require shifting all Academy resources to the cyber mission, but I would admit far greater numbers of cyber-affiliated candidates and radically beef up the Academy's cyber program. There is a precedent: in 1990 when I was admitted, I was one of the approximately 150 freshmen (out of about 1,500 total freshmen) who lacked 20/20 vision. 90% of my class was "pilot qualified" because the demand for pilot training was projected to be high by 1994.

If USAFA became known as "the" school for cyber, accepting that not all graduates intend to serve a twenty year Air Force career, I doubt the school would suffer so many questions of relevance and cost.

Tweet

Read More

Minneapolis Bridge Lessons for Digital Security

The Minneapolis bridge collapse is a tragedy. I had two thoughts that related to security.

1. If the bridge collapsed due to structural or design flaws, the proper response is to investigate the designers, contractors, inspectors, and maintenance personnel from a safety and negligence perspective. Based on the findings architectural and construction changes plus new safety operations might be applied in the future. This is a technical and operational response.


2. If the bridge collapsed due to attack, the proper response is to investigate, apprehend, proseceute, and incarcerate the criminals. Redesigning bridges to withstand bomb attack is unlikely. This is a threat reduction and deterrence response.

Do you agree with that assessment? If yes, why do you think response 1 (try to improve the "bridge" and similar operations) is the response to every digital security attack (i.e., case 2)? My short answer: everyone blames the victim, not the criminal.

The NTSB is on scene in Minneapolis with law enforcement to figure out if the bridge collapse was caused by scenario 1 or 2. Why don't we have a National Digital Security Board investigating breaches? My short answer: it's easier to hide a massive security breach than the destruction of any bridge, building, plane, or train.

Read More

More Engineering Disasters

I've written several times about engineering disasters here and elsewhere.

Watching more man-made failures on The History Channel's "Engineering Disasters," I realized lessons learned the hard way by safety, mechanical, and structural engineers and operators can be applied to those practicing digital security. >In 1983, en route from Virginia to Massachusetts, the World War II-era bulk carrier SS Marine Electric sank in high seas. The almost forty year-old ship was ferrying 24,000 tons of coal and 34 Merchant Mariners, none of whom had survival suits to resist the February chill of the Atlantic. All but three died.

The owner of the ship, Marine Transport Lines (MTL), blamed the crew and one of the survivors, Chief Mate Bob Cusick, for the disaster. Investigations of the wreck and a trial revealed the Marine Electric's coal hatch covers were in disrepair, as reported by Cusick prior to the disaster. Apparently the American Bureau of Shipping (ABS), an inspection organization upon which the Coast Guard relied, but funded by ship operators like MTL, had faked reports on the Marine Electric's status. With gaping holes in the coal hatches, the ship's coal containers filled with water in high seas and doomed the crew.

In the wake of the disaster, the Coast Guard recognized that ABS could not be an impartial investigator because ship owners could essentially pay to have their vessels judged seaworthy. Widespread analysis of ship inspections revealed many similar ships and others were unsound, and they were removed from service. Unreliable Coast Guard inspectors were removed. Finally, the Coast Guard created its rescue swimmer team (dramatized by the recent movie "The Guardian") to act as a rapid response unit.

The lessons from the Marine Electric disaster are numerous.

1. First, be prepared for incidents and have an incident response team equipped and trained for rapid and effective "rescue."


2. Second, be suspicious of reports done by parties with conflicts of interest. Stories abound of vulnerability assessment companies who find all of their clients "above average." To rate them otherwise would be to potentially lose future business.


3. Third, understand how to perform forensics to discover root causes of security incidents, and be willing to act decisively if those findings demonstrate problems applicable to other business assets.

In 1931, a Fokker F-10 Trimotor carrying eight passengers and crew crashed near Kansas City, Kansas. All aboard died, including Notre Dame football coach Knute Rockne. At the time of the disaster, plane crashes were fairly common. Because commercial passenger service had only become popular in the late 1920's, the public did not have much experience with flying. The death of Knute Rockne caused shock and outrage.

Despite the crude state of crash forensics in 1931, the Civil Aeronautics Authority (CAA) determined the plane crashed because its wood wing separated from its steel body during bad weather. TWA, operator of the doomed flight, removed all F-10s from service and burned them. Public pressure forced the CAA, forerunner of today's Federal Aviation Administration, to remove the veil of secrecy applied to its investigation and reporting processes. TWA turned to Donald Douglas for a replacement aircraft, and the very successful DC-3 was born.

The crash of TWA flight 599 provides several sad lessons for digital security.

1. First, few seem to care about disasters involving new technologies until a celebrity dies. While no one would like to see such an event occur, it's possible real change of opinion and technology will not happen until a modern Knute Rockne suffers at the hands of a security incident.


2. Second, authorities often do not have a real incentive to fix processes and methods until a tragedy like this occurs. Out of this incident came pressure to deploy flight data recorders and more robust aviation organizations.


3. Third, real inspection regulations and technological innovation followed the crash, so such momentum may appear after digital wrecks.

The final engineering disaster involves the Walt Disney Concert Hall in Los Angeles. This amazing, innovative structure, with a polished stainless steel skin, was completed in October 2003. When finished, visitors immediately realized a problem with its construction. The sweeping curves of its roof acting like a parabolic mirror, focusing the sun's ray like laser on nearby buildings, intersections, and sections of the sidewalk. Temperatures exceeded 140 degrees Fahrenheit in some places, while drivers and passersby were temporarily blinded by the glare.

Investigators decided to model the entire facility in a computer simulation, then monitor for the highest levels of sunlight over the course of a year. Using this data, 2% of the building's skin was discovered to be causing the reflection problems. The remediation plan, implemented in March 2005, resulted in sanding problematic panels to remove their sheen. The six-week, $60,000 effort fixed the glare.

The lessons from the concert hall involve complexity and unexpected consequences. Architect Frank Geary wanted to push the envelope of architecture with his design. His innovation caused a building that no one, prior to its construction, really understood. Had the system been modeled before being built, it's possible problems could have been avoided. This situation is similar to those involving enterprise network and software architects who design systems that no single person truly understands. Worse, the system may expose services or functionality never expected by its creators. Explicitly taking steps to simulate and test a new design prior to deployment is critical.

Digital security engineers should not ignore the lessons their analog counterparts have to offer. A commitment to learn from the past is the best way to avoid disasters in the future.

Read More

Nisley on Failure Analysis

Since I'm not a professional software developer, the only reason I pay attention to Dr. Dobb's Journal is Ed Nisley. I cited him earlier in Ed Nisley on Professional Engineering and Insights from Dr. Dobb's. The latest issue features Failure Analysis, Ed's look at NASA's documentation on mission failures. Ed writes:

[R]eviewing your projects to discover what you do worst can pay off, if only by discouraging dumb stunts.

What works for you also works for organizations, although few such reviews make it to the outside world. NASA, however, has done a remarkable job of analyzing its failures in public documents that can help the rest of us improve our techniques.

Documenting digital disasters has been a theme of this blog, although my request for readers to share their stories went largely unheeded. This is why I would like to see (and maybe create/lead) a National Digital Security Board.

Here are a few excerpts from Ed's article. I'm not going to summarize it; it takes about 5 minutes to read. These are the concepts I want to remember.

NASA defines the "root" cause of mishap as [a]long a chain of events leading to a mishap, the first causal action or failure to act that could have been controlled systematically either by policy/practice/procedure or individual adherence to policy/practice/procedure.

The root causes of these mishaps (incorrect units, invalid inputs, inverted G-switches) seem obvious in retrospect. How could anyone have possibly made those mistakes?

In addition to the root cause, the MIB Reports also identify a "contributing" cause as [a] factor, event or circumstance which led directly or indirectly to the dominant root cause, or which contributed to the severity of the mishap.

The "chain of events" is symptomatic of disasters. A break in that chain prevents the disaster.

However, the MIB [Mishap Investigation Board] discovered that [t]he Software Interface Specification (SIS) was developed but not properly used in the small forces ground software development and testing. End-to-end testing ... did not appear to be accomplished. (emphasis added)

Lack of end-to-end testing appears to be a common theme with disasters.

Mars, the Death Planet for spacecraft, might not have been the right venue for NASA's then-new "Faster, Better, Cheaper" mission-planning process...

The Mars Program Independent Assessment Team (MPIAT) Report pointed out that overall project management decisions caused the cascading series of failed verifications and tests. One slide of their report showed the MCO and MPL project constraints: Schedule, cost, science requirements, and launch vehicle were established constraints and margins were inadequate. The only remaining variable was risk.

In this context, "Faster" means flying more missions, getting rid of "non-value-added" work, and reducing the cycle time by working smarter rather than harder. "Cheaper" has the obvious meaning: spending less to get the same result. The MCO [Mars Climate Orbiter] and MPL [Mars Polar Lander] missions together cost less than the previous (successful) Mars Pathfinder mission.

The term "Better" has an amorphous definition, which I believe is the fundamental problem. In general, management gets what it measures and, if something cannot be measured, management simply won't insist on getting it.

You can easily demonstrate that you're doing things faster, that you've eliminated "non-value-added" operations, and that you're spending less money than ever before. You cannot show that those decisions are better (or worse), because the only result that really matters is whether the mission actually returns science data. Regrettably, you can measure that aspect of "better" after the fact and, in space, there are no do-overs. (emphasis added)

The last part is crucial. For digital security, the only result that really matters is whether you preserve confidentiality, integrity, and availability, usually by preventing and/or mitigating compromise. All the other stuff -- "percentage of systems certified and accredited," "percentage of systems with anti-virus applied," "percentage of systems with current patch levels" -- is absolutely secondary. In the Mars mission context, who cares if you build the spacecraft quicker, launch on time, and spend less money, if the vehicle crashes and the mission fails?

Thankfully NASA is taking steps to learn from its mistakes by investigating and documenting these disasters. It's time the digital security world learned something from these rocket scientists.

Read More

Insights from Dr. Dobbs

I've been flying a fair amount recently, so that means I've been reading various articles and the like. I want to make note of those I found interesting.

The March 2006 issue of Dr, Dobb's Journal featured a cool article on Performance Analysis and Multicore Processors. I found the first section the most helpful, since it differentiates between multithreading and hyperthreading. I remember when the FreeBSD development team was criticized for devoting so many resources to SMP. Now it seems SMP will be everywhere.

In the same issue Ed Nisley writes about Crash Handling. I call out this article for this quote:

Mechanical and civil engineers must consider how their projects can fail, right at the start of the design process, by calculating the stress applied to each component and the strength required to withstand it. Electrical engineers apply similar calculations to their circuits by considering voltage, current, and thermal ratings. In each case, engineers determine the project's expected behavior based on well-known properties of the bulk material and the finished components.

Software design isn't engineering simply because it does not deal with physical materials that have known properties. Software components don't exhibit a graded response to increasing stress: A single, trivial, data-dependent error can cause complete, instantaneous failure with no prior warning. Programs simply don't have a well-defined safe operating area.

If that's true, we're finished. I'm afraid it may be, but also in the same DDJ issue we read these comments on Project Management by Gregory V. Wilson:

[W]hile Stepanek [Bejtlich: author of a book reviewed by Wilson] refers to many of the classic works in software engineering, he seems to have missed most of what's appeared in the primary literature in the last 10 years. New journals such as Empirical Software Engineering have both reflected, and encouraged, new studies of what actually works and doesn't—studies that are methodologically sounder than many of their predecessors—and I think they ought to be required reading for anyone writing about software engineering today. I share Stepanek's sense of frustration, but think that software engineering isn't as special as it is often convenient for software engineers to believe.

I agree with that statement. I find too many people use the term "art" to make themselves feel special. If they took the time to document their approaches (analysis, administration, coding, pen testing, reverse engineering, etc.) they would find their processes repeatable and far less "special." Science is boring; art is cool. Science can also be automated, which could mean replacing humans. While I always believe humans are the best operators, hiding behind "art" is cowardly.

Read More

Ed Nisley on Professional Engineering

I get a free subscription to Dr. Dobb's Journal. The March 2006 issue features an article by Ed Nisley titled "Professionalism." Ed is a software developer with a degree in Electrical Engineering. After working at a computer manufacturer for ten years in New York state, he decided to become a "consulting engineer." Following the state's advice, Ed pursued a license to be a Professional Engineer. Now, 20 years after first earning his PE license, Ed declined to renew it. He says "the existing PE license structure has little relevance and poses considerable trouble for software developers." You have to register with DDJ to read the whole article, but the process is free and the article is worthwhile.

Here are a few of Ed's reasons to no longer be a PE:

• "[T]o maintain my Professional Engineering license, I must travel to inconvenient places, take largely irrelevant courses, and pay a few kilobucks. As nearly as I can tell from the course descriptions, the net benefit would be close to zero."


• ["T]here's no generally applicable Software Engineering Body of Knowledge (SWEBOK) upon which to base a Software Engineering examination, so (as I understand it) a Texas engineer seeking a PE license for software activities must demonstrate a suitable amount of experience, as attested by letters of recommendation." (He was discussing efforts in Texas to make software engineers be PEs.)


• "A 2001 ACM task force report on Licensing of Software Engineers Working on Safety-Critical Software concluded that professional licensing as it stands today simply wouldn't work in that field. They observe that very few 'software engineers' have an engineering degree accredited by the Accreditation Board for Engineering and Technology, which all state PE licensing boards require. Most programmers, it seems, don't have the opportunity to forget Thermo and Chem, having not studied them in the first place."


• "Software development also moves much faster than the NCEES testing process. Mechanical and electrical engineering questions dating back three decades remain perfectly useful, but most recent graduates have little knowledge of Fortran and GOTOs."


• "If you produce work as a PE, you must follow established design practices or risk a malpractice lawsuit when your design fails. Software engineering, even in the embedded field, simply doesn't have any known-good design practices: Most projects fail despite applying the current crop of Best Practices."


• "Worse, without a good self-imposed technical solution, we're definitely going to get legislative requirements that won't solve the problem."

If you think that creating a test designed for "software engineers" is a good idea, check out the rest of the article to see Ed's experience taking the exams. They sound like nothing more than a check to ensure the ability to answer a smattering of science and math questions.

The process reminded me of an exam we took at the Air Force Academy for what was then called (and may still be) Engineering 410. This was supposed to be a "capstone course" that all seniors took to demonstrate their engineering prowess. Yes, even your local history/political science double major took chemistry, physics (two courses), math (Cal III and Diff Eq), thermodynamics, and the five pure engineering courses (electrical, mechanical, civil, aeronautical, astronautical) prior to this capstone course. (That's why I have Bachelor of Science degrees and not BAs. At a normal college I would also have a minor in Engineering.)

To enter the capstone course, all students had to pass a cross-subject exam, where anything studied up to that point was fair game. I should add that non-engineering subjects like biology or the "soft sciences" were also included. If you failed the exam (with a possibility of one retake) you failed the course. If you took the course in the fall semester, you could return in the spring. If you took the course in the last semester of senior year (like me), and you failed the test, you were coming back for a special "fifth year" (USAFA has no real "fifth year" of study!) just to take Engineering 410.

In the dreamworld of the academic faculty, I'm sure they believed this exam would test the quality of the "engineers" they were producing. In reality all they tested was our ability to cram as much as we could fit into our brains prior to the test. By the time I was a senior I had no clue what I had studied in chemistry or biology three years earlier. After reading Ed's story, it sounds like his PE exams were exactly the same. They test the candidate's ability to remember information (itself no mean feat, granted) and then apply that to a test. They say nothing about whether the candidate is a good or even qualified engineer.

If the test is worthless, what might really drive PEs to do good work? I think the fact that PEs can lose their license to practice is a big factor. That happened in the 1981 Hyatt collapse I blogged about earlier. If you're a PE and you lose your license because your project fails, you've lost your ability to make a living. If you're a software developer and your project fails, you continue working or you get a job elsewhere.

Incidentally, the skies over USAFA looked exactly like the photo posted above. Every day. Ok, I'm kidding, but it felt like that. That is a real photo taken 10 August 2004.

Read More

Another Engineering Disaster

Does the following sound like any security project you may have worked?

1. Executives decide to pursue a project with a timetable that is too aggressive, given the nature of the task.


2. They appoint a manager with no technical or engineering experience to "lead" the project. He is a finance major who can neither create nor understand design documents. (This sounds like the news of MBAs being in vogue, as I reported earlier.)


3. The project is hastily implemented using shoddy techniques and lowest-cost components.


4. No serious testing is done. The only "testing" even tried does not stress the solution in any meaningful way -- it only "checks a box."


5. Shortly after implementation, the solution shows signs of trouble. The project manager literally patches the holes and misdirects attention without addressing the underlying flaws.


6. Catastrophe eventually ensues.

What I've just described is the Boston Molasses Flood of 1919, best described by the Boston Society of Civil Engineers in their newsletter (.pdf). I learned about this event by watching another episode of Engineering Disasters on Modern Marvels. Here's what happened.

1. In 1915, United States Industrial Alcohol needed to build a tank in Boston to support World War I munitions production. They decide to place it in an immigrant-dominated portion of the city; Italians live there.


2. USIA puts Arthur Jell in charge. He is a finance major with no technical or engineering experience or training. He can't even read blueprints, yet he designs a five-story, 90' diameter tank capable of holding over 2 million gallons of molasses, in the middle of a populated area.


3. The tank is built by contractors who use thin steel and too few rivets. No one supervises their work. They hurry to complete the tank 2 days before it is filled.


4. Prior to being filled, the tank is "tested" by holding between 4 and 8 inches of water!


5. The tank stands three years, although apparently it was never filled to capacity until shortly before its collapse. During those three years, molasses leak from the tank on a daily basis. Jell orders the leaks plugged and has the tank painted brown to divert attention from the leaks.


6. In 1918, with WWI ending and prohibition approaching, USIA decides to switch production from industrial alcohol to drinking alcohol. They want to cash out as fast as possible by supporting customers who want to "stock up" before prohibition begins. They accept a shipment of molasses from Cuba in January 1919, which fills the tank to capacity. Three days later, on January 15, the tank ruptures, killing 21 people and injuring 150.

USIA claimed Italian anarchists had destroyed the tank, but the evidence showed otherwise. USIA was subjected to the first ever class action lawsuit in the US, which the company lost. Safety regulations were enacted which required supervision of construction, real testing, and stamps of approval of blueprints by architects and engineers.

I foresee a similar event, with similar consequences, for the digital security industry. Hopefully not as much death and destruction will occur, but the remedies will be the same.

Read More

Engineering Disasters in Information Security Magazine

The December 2005 issue of Information Security magazine features an article I wrote titled History Lessons with the subtitle "Digital security could learn a lot from engineering's great disasters." It is based on this blog entry describing analog engineering disasters like the 1931 Yangze River damn failure, the 1944 Cleveland LNG tank fire, the 1981 Kansas City Hyatt Regency hotel walkway collapse, and the 1933 Atlanta Marriott parking lot sink hole.

I am considering expanding this topic of digital security disasters to encompass a new book. I would like to take a historical and technical look at digital security failures on a case-by-case basis. Ideally the cases would be based on testimony from witnesses or participants wishing to (anonymously) share lessons with their colleagues.

My concept is simple: when a bridge fails in the "analog" world, everyone knows about it. The disaster is visible, and engineers can analyze and learn from the event. The lessons they take away make future bridges stronger and safer. I do not see this happening in the digital world. Organizations suffer disasters all the time due to poor techniques, tools, configuration, management decisions, and so on. Unfortunately, few people ever hear about these problems, so they are repeated elsewhere. The only parties to benefit are intruders. Security engineers never get to learn from the mistakes of others.

What would a sample story look like? As a simple example, I know of an ISP who suffered a two hour router ACL drop that allowed remote intruders to exploit their development network. The ISP suffered a major compromise by Russian intruders that required a dedicated multi-week, guerilla-warfare incident response effort. Several lessons can be learned: (1) a router with an ACL is not a firewall, especially when you can attack any high port using source port 20 TCP; (2) development networks with unpatched machines should not bear publicly routable IP addresses and be Internet facing; and (3) there is value in monitoring to detect when your defensive measures fail.

Would any of you be willing to share your stories with me? I would be willing to communicate in any reasonable manner you wish to preserve your identities and sensitivities. The goal of the book is to provide real-world cases that can teach lessons to fellow security engineers. I am not trying to embarrass or humiliate anyone. I do not expect to hear any company or personal names, and if you still provide them I will not repeat them in the book. I am most interested in stories that have plenty of technical details.

Please email your thoughts to richard at taosecurity dot com. Thank you.

Read More

More on Engineering Disasters and Bird Flu

Here's another anecdote from the Engineering Disasters story I wrote about recently. In 1956 the cruise ship Andrea Doria was struck and sunk by the ocean liner Stockholm. At that time radar was still a fairly new innovation on sea vessels. Ship bridges were dimly lit, and the controls on radar systems were not illuminated. It is possible that the Stockholm radar operators misinterpreted the readings on their equipment, believing the Andrea Doria was 12 miles away when it was really 2 miles away. The ships literally turned towards one another on a collision course, based on faulty interpretation of radar contact in the dense fog. Catastrophe ensued.

This disaster shows how humans can never be removed from the equation, and they are often at center stage when failures occur. The commentator on the show said a 10 cent ligh bulb illuminating the radar controls station could have shown the radar range was positioned in a setting different from that assumed by the operator. Following the Andrea Doria collision, illumintation was added to ship radar controls. This story reminded me that the latest security technology is worthless -- or even worse, damaging -- in the hands of people who are not trained or able to use it properly.

On a different subject, I heard an interview on NPR with Health and Human Services Secretary Mike Leavitt about bird flu. He likened the situation to "surveillance" of a dry forest during fire season. He said that the best defense was vigilance and rapid response. His analogy assumed being nearby when a small fire erupts. First responders who are quickly on the scene can stamp out a fire before it becomes uncontrollable. If the response team is unaware of the fire, it can spread and then be beyond containment. He concluded the interview saying "ultimately, another pandemic will come. Right now we are not prepared."

I thought his comments applied well to digital security incidents. NSM is surveillance, and incident response helps stamp out fires (or bird flu outbreaks) quickly before they exceed an organization's capacity to deal with them. Is your organization ready? If you want to know, TaoSecurity provides services like incident response training and CSIRT assessments and evaluations.

Read More

Further Thoughts on Engineering Disasters

My TiVo managed to save a few more episodes of Modern Marvels. You may remember I discussed engineering disasters last month. This episode of the show of the same title took a broader look at the problem. Three experts provided comments that resonated with me.

First, Dr. Roger McCarthy of Exponent, Inc. offered the following story about problems with the Hubble Space Telescope. When Hubble was built on earth, engineers did not sufficiently address issues with the weight of the lens on Earth and deflections caused by gravity. When Hubble was put in orbit, the lens no longer deflected and as a result it was not the proper shape. Engineers on Earth had never tested the lens because they could not figure out a way to do it.

So, they launched and hoped for the best -- only to encounter a disaster that required a $50 million orbital repair mission. Dr. McCarthy's comment was "A single test is worth a thousand expert opinions." This is an example of management by fact instead of management by belief, mentioned previously on this blog.

Second, Dr. Charles Perrow, author of Normal Accidents: Living With High-Risk Technologies, explained the makings of a disaster. Essentially, he said disasters are caused by the unforeseen consequences of multiple, individually non-devastating, failures in complex systems. Most catastrophes could be prevented if any one of the small failures had not occurred. Third, Mary Schiavo commented on the Challenger disaster. She described the well-known problems with operating the Shuttle's rocket O-rings in temperatures below 53 degrees F. The Shuttle had launched at lower temperatures prior to the Challenger explosion, but NASA knew they were risking catastrophe. Ms. Schiavo said NASA engineers begged their managers not to let Challenger launch, seeing that chunks of ice covered the launch pad and Shuttle. They were overruled and disaster occurred.

This struck a chord with me, because a few days earlier I read a new story in Time about how Steve Jobs gets Apple to bring innovative products to market:

Apple CEO Steve Jobs [will] tell you an instructive little story. Call it the Parable of the Concept Car. "Here's what you find at a lot of companies," he says, kicking back in a conference room at Apple's gleaming white Silicon Valley headquarters, which looks something like a cross between an Ivy League university and an iPod. "You know how you see a show car, and it's really cool, and then four years later you see the production car, and it sucks? And you go, What happened? They had it! They had it in the palm of their hands! They grabbed defeat from the jaws of victory!

"What happened was, the designers came up with this really great idea. Then they take it to the engineers, and the engineers go, 'Nah, we can't do that. That's impossible.' And so it gets a lot worse. Then they take it to the manufacturing people, and they go, 'We can't build that!' And it gets a lot worse."

When Jobs took up his present position at Apple in 1997, that's the situation he found. He and Jonathan Ive, head of design, came up with the original iMac, a candy-colored computer merged with a cathode-ray tube that, at the time, looked like nothing anybody had seen outside of a Jetsons cartoon. "Sure enough," Jobs recalls, "when we took it to the engineers, they said, 'Oh.' And they came up with 38 reasons. And I said, 'No, no, we're doing this.' And they said, 'Well, why?' And I said, 'Because I'm the CEO, and I think it can be done.'"

Would Steve Jobs have overruled the NASA engineers and launched Challenger? Who knows.

From what I have learned, disasters are prone to happen in complex, tightly-coupled systems. The only way to try to avoid them is to test and monitor their operation, exercise response, and then implement those plans when catastrophe occurs. Anything less is like launching a defective, untested Hubble and hoping for the best, and then paying through the nose to clean up the mess.

Here are a few footnotes to this post. Dr. McCarthy's company offers security engineering services, including services for information systems. They are described thus: "We have assembled one of the largest private collections of computerized accident and incident data in the world. Our web-based solutions put this information at your disposal, giving you comprehensive risk data quickly and at low cost." Dr. McCarthy was recently elected to the National Academy of Engineering, which has a Computer Science and Telecommunications Board with a Improving Cybersecurity Research in the United States project. My research for this story also led me to the System Safety Society.

Read More

Engineering Disaster Lessons for Digital Security

I watched an episode of Modern Marvels on the History Channel this afternoon. It was Engineering Disasters 11, one in a series of videos on engineering failures. A few thoughts came to mind while watching the show. I will provide commentary on each topic addressed by the episode.

• First discussed was the 1944 Cleveland liquified natural gas (LNG) fire. Engineers built a new LNG tank out of material that failed when exposed to cold, torching nearby homes and businesses when ignited. 128 people died. Engineers were not aware of the metal's failure properties, and absolutely no defensive measures were in place around the tank to protect civilian infrastructure.
  
  This disaster revealed the need to (1) implement plans and defenses to contain catastrophe, (2) monitor to detect problems and warn potential victims, and (3) thoroughly test designs against possible environmental conditions prior to implementation. These days LNG tanks are surrounded by berms capable of containing a complete spill, and they are closely monitored for problems. Homes and businesses are also located far away from the tanks.


• Next came the 1981 Kansas City Hyatt walkway collapse that killed 114 people. A construction change resulted in an incredibly weak implementation that failed under load. Cost was not to blame; a part that might have prevented failure cost less than $1. Instead, lack of oversight, poor accountability, broken processes, a rushed build, and compromise of the original design resulted in disaster. This case introduced me to the term "structural engineer of record," a person who assigns a seal to the plans used to construct a building. The two engineers of record for the Hyatt plans lost their licenses.
  
  I wonder what would happen if network architectures were stamped by "security engineers of record?" If they were not willing to afix their stamp, that would indicate problems they could not tolerate. If they are willing to stamp a plan, and massive failure from poor design occurs, the engineer should be fired.


• The third event was a massive sink hole in 1993 in an Atlanta Marriott hotel parking lot. A sewer drain originally built above ground decades earlier was buried 40 feet under the parking lot. A so-called "safety net" built under the parking lot was supposed to provide additional security by giving hotel owners time to evacuate the premises if a sink hole began to develop.
  
  Instead, the safety net masked the presence of the sink hole and let it enlarge until it was over 100 feet wide and beyond the net's capacity. Two people standing in the parking lot died when the sewer, sink hole, and net collapsed. This disaster demonstrated the importance of not operating a system (the sewer) outside of its operating design (above ground). The event also showed how products (the net) may introduce a false sense of security and/or unintended consequences.


• Next came the 1931 Yangzi River floods that killed 145,000 people. The floods were the result of extended rain that overcame levees built decades earlier by amateur builders, usually farmers protecting their lands. The Chinese government's relief efforts were hampered by the Japanese invasion and subsequent civil war. This disaster showed the weaknesses of defenses built by amateurs, for which no one is responsible. It also showed how other security incidents can degrade recovery operations.
  
  Does your organization operate critical infrastructure that someone else built before you arrived? Perhaps it's the DNS server that no one knows how to administer. Maybe its the time service installed on the Windows server that no one touches. What amateur levee is waiting to break in your organization?
  


• The final disaster revolved around the deadly substance asbestos. The story began by extolling the virtues of asbestos, such as its resistance to heat. This extremely user-friendly feature resulted in asbestos deployments in countless products and locations. In 1924 a 33-year-old, 20-year textile veteran died, and her autopsy provided the first concrete evidence of asbestos' toxicity. A 1930 British study of textile workers revealed abnormally high numbers of asbestos-related deaths. As early as 1918 insurance companies were relucant to cover textile workers due to their susceptibility to early death. As early as the 1930s the asbestos industry suppressed conclusions in research they sponsored when it revealed asbestos' harmful effects.
  
  By 1972, the US Occupational Safety and Health Administration arrived on the scene and chose asbestos as the first substance it would regulate. Still, today there are hundreds of thousands of pending legal cases, but asbestos is not banned in the US. This case demonstrated the importance of properly weighing risks against benefits. The need to independently measure and monitor risks outside of a vendor's promises was also shown.

I believe all of these cases can teach us something useful about digital security engineering. The main difference between the first four cases and the digital security world is the failure in the analog world is blatantly obvious. Digital failures can be far more subtle; it may take weeks or months (or years) for secuirty failures to be detected, unlike sink holes in parking lots. The fifth case, describing asbestos, is similar to digital security because harmful effects were not immediately apparent.

Read More