Netanyahu Channels Tufte at United Nations

This is not a political blog, and I don't intend for this to be a political post.

I recently watched Israeli Prime Minster Benjamin Netanyahu's speech to the United Nations on Thursday. I watched it because I am worried about Iran's nuclear weapons program and the Iranian security situation, to be sure.

However, what really intrigued me was the red line he actually drew on a diagram, in front of the United Nations. In the video I linked, it takes place at approximately the 26 minute mark. The screen capture at left shows this event.

The reason this caught my attention was that it reminded me of the Best Single Day Class Ever, taught by Edward Tufte. I attended his class in 2008 and continue to recommend it.

I've since blogged about Tufte on several occasions.

Netanyahu's action, to me, seems like pure Tufte. The primary goal of his speech was to tell Iran, and the world, that Israel is setting a "red line" involving Iran's nuclear weapons program. To show that, he literally drew a red line on a diagram representing Iranian progress on uranium enrichment.

Now, there's some confusion about what that red line really means. The point is that people are talking about the red line, and that means Netanyahu at least partially achieved his goal.

This is the take-away for those of us who speak in public: rather than develop Yet Another PowerPoint presentation, determine 1) what message you want your audience to remember, and then 2) figure out how you can escape from flat land to grab your audience's attention.

If you want to learn more about these techniques, take Tufte's course!

You can read a transcript of the speech as well as see the video. Besides the red line segment, I thought it was a powerful speech. I'm convinced that unless Iran changes course, Israel will disable Iran's uranium enrichment capability.

Tweet

Read More

Comparing Microsoft's Communication Methods

Today is Microsoft Patch Tuesday, which means if you so choose you can read posts by the Microsoft Security Response Center like February 2011 Security Bulletin Release. The advisory states "we have 12 bulletins addressing 22 vulnerabilities in Microsoft Windows, Office, Internet Explorer, and IIS (Internet Information Services). Three bulletins are rated Critical."

Microsoft communicates information about these vulnerabilities using two graphics.

The first is "Severity and Exploitability Index":



The second is "Bulletin Deployment Priority":



I'm not even going to start a discussion about why the first chart shows "risk" and then "impact" (isn't impact a component of risk?) I'm also not going to dwell about how the first column of the second chart has been "overloaded" to include only a small bit of information on the code affected, rather that prominently communicating that data in a column of its own.

Instead, I'd like to know who else finds this sort of red-yellow-blue presentation to be an assault on your senses? I mean, at the very least, isn't all the information from the top chart present in the bottom chart (despite more lovely coloring?)

In contrast to that communication method, I'd like to highlight content from a related Microsoft blog post titled Breaking up the Romance between Malware and Autorun. Why do I like this post? Check out this table:



Why do I like it?

• It shows 40 numbers. What you say? It only shows 36? I consider the NULL values to be valuable too because they demonstrate Microsoft wasn't tracking those malware families yet, or they didn't exist, etc.


• It identifies 10 malware families.


• It shows trends over time.


• The results are ranked by totals for 2H10.


• Nothing is colored RED to tell me THIS IS BAD.

I like to see content like that table because it treats the viewer like an adult who can at least read at the level of the sports pages in the newspaper, as the great Tufte says.

Tweet

Read More

Reflections on Four Tufte Books

This week I finished the four main books written by Edward Tufte, namely The Visual Display of Quantitative Information, 2nd ed, Envisioning Information, Visual Explanations, and Beautiful Evidence. I decided not to review them individually at Amazon.com for several reasons.

First, I received them as a set 2 1/2 years ago at The Best Single Day Class Ever, what I call Tufte's class. Tufte's class and written work present a single set of ideas and some material is presented from multiple angles in several books. This makes it congnitively difficult for me to review them individually. Second, I did not treat them like other books I read, meaning I did not mark them with my own notes and underlining. Frankly the books are like works of art and it would pain me to mark them up! That makes it tough for me to review my reading process and withdraw comments suitable for a book review. Third, so many people have already reviewed the books that I did not feel I would bring any real novelty or domain expertise to the discussion.

Rather, for this post I wanted to share a few ideas I learned from Tufte that I try to keep in mind when communicating. Some of these are reflected in my earlier post, but I'd like to share what has stayed with me during these past 2 1/2 years.

1. Do not let the medium define your message. PowerPoint culture is endemic in my workplace and in many others. Rather than considering the message to be communicated, too many people concentrate on what the PowerPoint "pitch" needs to look like. I don't exclusively mean appearance, although that is definitely a factor. I'm referring more to what bullets are supposed to reflect a message to an audience. Rather than leading with bullets, determine what message you are trying to communicate, then select a medium.


2. Replace "presentations" with conversations. I avoid delivering lectures as much as possible. Nothing kills the spirit like receiving a stack of 300 slides. That "deck" represents a plodding, instructor-paced, predetermined path where questions are more likely to be interpreted as interruptions of the "flow" of the class. After seeing Tufte in action in 2008, I stopped teaching my two day TCP/IP Weapons School class using slides. The second and now third editions of the class have no slides whatsoever. Instead I teach with workbooks, labs, and unscripted question-and-answer interactions with students.


3. Carry the burden or stay off the field. It is NOT easy to teach "Tufte style." Too many "presenters" and "instructors" fall into the seductive embrace of reading slides, facing the screen and not the students, hoping to get to the end of the pitch as soon as possible.
   
   Instead, imagine walking into a room with 100 or more people, giving each a paper handout with some possible discussion topics, and then asking what they would like to know about the security field. That is just what I did at the FIRST conference this year, and from what I heard, people liked it. I'll say now that it was a somewhat scary experience for me to focus purely on conversation and not just march through a 30 slide PowerPoint deck. However, this is the sort of approach we need to see in the field. I don't recommend it for every talk, but if you're up to carrying the burden, give it a try!


4. Seek data and graphic representations where possible. For me, this is probably harder than the previous point. Whereas talking in an unscripted manner is rough because of the mental gymnastics required, creating data-driven figures is tough because of the amount of preparation required. We struggle with this in our CIRT. We have thousands of data points but the collection, analysis, interpretation, and explanation of that information is much more difficult than I expected. As we add staff who spend less time fighting operational battles and more time contemplating the overall picture, I expect us to deliver the sorts of graphics that speak volumes to all sorts of audiences.


5. When the available tools stink, make your own. Tufte did this by publishing his books himself. He did not accept the limitations of the publishers who claimed he could not include the novel features found in his titles. We've encountered similar issues at work where existing data collection tools were just not suited for our needs. Several very talented and motivated team members built and continue to build new tools to get the job done. This is even more difficult than the previous point because it requires anticipating the sorts of data needed to describe, explain, and improve security operations. I expect a lot of progress in this area in 2011.

That's my "applied Tufte" for 2010. Here's hoping he publishes another book soon. The best New Year's resolution you could make for 2011 is to attend one of his classes, even if you have to pay yourself. You get all four books with paid tuition -- real books, not slide decks!
Tweet

Read More

Mike Cloppert on Defining APT Campaigns

Please stop what you're doing and read Mike Cloppert's latest post Security Intelligence: Defining APT Campaigns. Besides very clearly and concisely explaining how to think about APT activity, Mike includes some original Tufte-esque figures to demonstrate APT attribution and moving up the kill chain.

Read More

Papers Not PowerPoint, Plus Tips for Improvement

Recently I railed against PowerPoint. In this post I'd like to congratulate Black Hat and some of their Briefings speakers for submitting white papers, not just PowerPoint presentations.

This evening while cleaning out a tmp directory I noticed a copy of a white paper by IBM's Tom Cross from Black Hat DC 2010 titled Exploiting Lawful Intercept to Wiretap the Internet. The paper describes Tom's analysis of Cisco's implementation of CALEA for law enforcement-directed wiretaps. The paper is 18 pages, but the last 3 are basically citations. It's a great piece of work which I wish I had read earlier.

For me, this paper emphasized how much of a failure it is to try to deliver complicated information in PowerPoint form. I got more out of taking 20 minutes to read Tom's 15 pages of material than I could have trying to make sense out of his 41 slides. Tom is a good writer whose paper delivers solid arguments. Rather than just praise the paper and slam the PowerPoint, I'd like to show how Tom did use PowerPoint well so that I keep these ideas in mind when I need to brief audiences.

A speaker I listened to earlier this week said you can't expect an audience to take away more than one point from any slide, so why bother? In fact, if you adapt the ideas of the great Tufte, you should use PowerPoint only as a delivery mechanism for charts, diagrams, and other visuals.

Using this approach, the figure at right which appears in Tom's PowerPoint deck for Black Hat is just the kind of material that should appear in a PowerPoint presentation. You could imagine this diagram being in a handout given to the audience, but during the briefing Tom would no doubt want to point towards specific elements of the diagram while the audience watched. This justifies displaying the figure via PowerPoint, because it is the most effective medium for communicating the information.

I think the SNMP MIB extract displayed at left, also from Tom's PowerPoint, is justified as appearing in a slide. Tom isn't asking the audience to pay attention to every line on the slide, like someone might expect an audience to do with a slide full of bullets. Rather, Tom has highlighted two important excerpts, showing them as proof that within this MIB there are two elements which expose information to attackers. This information could also appear on a handout given to the audience. However, here I like seeing the information to prove Tom's point. It's almost like a "technical figure" for me.

On a related point, I did not see any PowerPoint posted for HD Moore's talk Metasploit and Money. However, HD posted a great 9 page white paper, which is archived. I think I already mentioned via Twitter that I enjoyed this paper, and I wonder if no slides were presented?

To summarize, if you're presenting complicated material, slides are generally not an effective delivery mechanism. At best they can supplement a briefing by being a vehicle for displaying figures or other visuals, but bullets are generally a waste of time. For details why, please see my posts on PowerPoint.

Read More

More PowerPoint Woes

Last year I attended The Best Single Day Class Ever, taught by Prof. Tufte. He changed my outlook on PowerPoint for ever. Today in FCW magazine I found a pointer to 8 PowerPoint Train Wrecks, like the slide Bill Gates is presenting at left. While following some of the linked presentations, I came across this line from the shmula blog:

While at Amazon, we were all told by Divine Fiat that ALL presentations — regardless of kind, cannot ever be on Powerpoint. Period. Bezos prefers prose and actual thoughts slapped in a report — an actual paper report with paragraphs, charts, sentences, an executive summary, introduction of problem, research approach and findings (body of paper), conclusions and recommendations — not choppy, half-thoughts on a gazillion slides.

Thank goodness. I am not crazy after all.

That same blog post makes other good points, and links to an imagined Barack Obama "Yes We Can" PowerPoint deck. Hilarious.

---

Richard Bejtlich is teaching new classes in Europe and Las Vegas in 2009. Online Europe registration ends by 1 Apr, and seats are filling. "Super Early" Las Vegas registration ends 15 Mar.

Read More

Black Hat USA 2008 Wrap-Up: Day 2

Please see Black Hat USA 2008 Wrap-Up: Day 1 for the first part of this two-part post.

Day two of the Black Hat USA 2008 Briefings began much better than day one.

• Rod Beckström, Director of the National Cyber Security Center in DHS, delivered today's keynote. I had read articles like WhiteHouse Taps Tech Entrepreneur For Cyber Defense Post so I wasn't sure what to think of Mr. Beckström. It turns out his talk was excellent. If Mr. Beckström had used a few less PowerPoint slides, I would have classified him as a Edward Tufte-caliber speaker. I especially liked his examination of history for lessons applicable to our current cyber woes. He spoke to the audience in our own words, calling the US an "open source community," the Declaration of Independence and Constitution our "code," the Civil War a "fork," and so on. Very smart.
  
  For example, Mr. Beckström provided context for the photo at left of Union Intelligence Service chief Allan Pinkerton, President Lincoln, and Major General John A. McClernand during Antietam (late 1862). Using the photo Mr. Beckström explained the relationship between the intelligence community, the government, and the military. After I answered his question "what made the Civil War unique?" (answer: the telegraph), Mr. Beckström described how Lincoln was the first "wired" President and how electronic warfare against cables first began.
  
  In addition to talking about the French and Indian War and our own Revolution (e.g., Washington learned guerilla tactics, Benedict Arnold as insider threat), Mr. Beckström spoke about how to characterize our current problem. He said "offense is a lot easier than defense." Unlike Moore's Law, we don't have laws for the physics of networking, or the economics of networks or security, or how to do risk management. Mr. Beckström noted security is a cost (so much for "enablement") and that minimization of total cost C (where C equals cost of security S plus expected cost of a loss L) is the main goal. (I wonder if he's read Managing Cyber Security Resources?) Mr. Beckström said the CISO budget should be based on reducing estimated loss, but it's usually based on a percentage of the CTO or CIO's budget that's unrelated to any problem faced by the security team.
  
  Most interesting to me, Mr. Beckström explained how he believes investment in protocols (like security DNS, BGP, SMS/IP, even POTS) could be cheap while yielding large benefits. I will have to watch for developments there.


• Next I saw Felix Lindner present Developments in Cisco IOS Forensics. It seemed like a lot of the ideas were present in his great talk from last year, but I liked this year's presentation anyway. FX is absolutely the authority on breaking Cisco IOS, he's an excellent speaker, and I learned a lot. FX discussed how attacks on IOS take the form of protocol, functionality/configuration, or binary exploitation attacks. Binary exploitation is of most interest to FX, and takes the form of binary modification of the runtime image, data structure patching, runtime configuration changes, and loading TCL backdoors. (The last is "widely used by people fired from ISPs"!)
  
  In order to gain some degree of visibility into binary exploitation attacks against IOS, FX recommends enabling core dumps. This does not affect performance (except slowing reboot time). Core dumps can be written to a FTP server, and will result from unsuccessful binary exploitation attempts or any time a router administrator invokes the "write core" command. Because there are over 100,000 IOS images in use today ("only" 15,000 or so are supported by Cisco), there is a high likelihood that a remote intruder will crash the router when trying a binary exploitation attack. Furthermore, it's possible to find the packets which caused the attack in the router's memory dump, since they will be in the queue of the attacked thread. I look forward to trying this and submitting a dump to Recurity Labs CIR.


• Greg Conti and Erik Dean presented Visual Forensic Analysis and Reverse Engineering of Binary Data. I thought one of their slides (presented at left) was, unintentionally, a powerful partial summary of the skill sets needed for certain levels of analysis of binary data in our field. For example, I am very comfortable in the lowest portion where binary data represents network packets. I am trying to learn more about binary data as memory. I have worked with binary data as files, but there's a lot going on there (as I learned from a talk on Office forensics, noted below).
  
  Greg and Erik demonstrated two new tools they wrote for visual analysis of binary data. Most surprisingly, they showed actual images rendered from the memory of a Firefox crash dump file. An example appears at right. They displayed the image by plotting every three bytes of memory as a RGB entry. They also noted that one day we could expect to see security analysts sitting with recognition posters of common patterns (e.g., diffuse means encryption or compression). That reminded me of the surface-to-air missile (SAM) emplacement images I studied in intel school.


• I joined Detecting & Preventing the Xen Hypervisor Subversions by Joanna Rutkowska and Rafal Wojtczuk. Joanna had to remove slides pending publication of a patch from Intel. (See my last post for notes on why the chipset is the new battleground.) She hinted that Intel is considering working with anti-virus vendors to run scans inside the chipset, which would be a bad idea. Joanna also talked about her company's product, HyperGuard, which can sit inside the Phoenix BIOS to perform integrity checking.
  
  Joanna's research started with Blue Pill as a means to put an OS inside a thin hypervisor (the Blue Pill), but her newest work involves attacking an existing hypervisor (like Xen). This is why her post 0wning Xen in Vegas! says Rafal will discuss how to modify the Xen’s hypervisor memory and consequently how to use this ability to plant hypervisor rootkits inside Xen (everything on the fly, without rebooting Xen). Hypervisor rootkits are very different creatures from virtualization based rootkits (e.g. Bluepill). This will be the first public demonstration of practical VMM 0wning.
  
  This presentation reminded me that I should have a permanent Xen instance running in my lab to improve familiarity with the technology.


• Following Joanna's talk I enjoyed Get Rich or Die Trying - Making Money on the Web, the Black Hat Way by Jeremiah Grossman and Arian Evans. They showed many real and amusing cases of monetizing attacks.


• Bruce Dang discussed Methods for Understanding Targeted Attacks with Office Documents. Everything for Bruce is "pretty easy," like writing custom Office document parsers to examine Office-based malware. Bruce ended his talk early so I moved next door to hear the Sensepost guys. I should have attended earlier -- it sounded like they created some extreme tunnels for getting data in and out of enterprise networks. They concluded by discussing how to load and execute binaries into the address space of SQL Server 2005 via SQL injection, because SQL Server 2005 has an embedded .NET CLR. Wow.

Overall, I think my conclusions from my last Black Hat Briefings still stand. However, I was surprised to see so much more action on the chipset level. I did not hear anything about the other extreme of the digital spectrum, the cloud. Perhaps that will be a topic next year, if the lawyers can be avoided?

Read More

The Best Single Day Class Ever

I had the great fortune to attend Edward Tufte's one day class Presenting Data and Information. I only knew Tufte from advertisements in the Economist. For example, the image at left was frequently used as an ad in the print magazine. I had not read any of his books although I knew of his criticism of PowerPoint, specifically with respect to the Challenger disaster.

This was the best one day class I have ever taken. It profoundly altered the way I think about presenting information and making arguments. If any part of your professional life involves delivering presentations, you must attend this class. It's a complete bargain for the price. I would like to see every professional at my company take this course. Following Tufte's advice would provide the single biggest productivity improvement and corresponding "return on investment" we are likely to see in my tenure.

There is no way for me to summarize Tufte's course. You should attend yourself, and read the four (!) textbooks he provides. I will try to capture points which made an impact upon me.

Substance, not Structure: When delivering a presentation, do whatever it takes to make your point. Be substance-driven, not method-driven. This means you determine what information you need to convey, not what you should put into PowerPoint. This really impressed me. PowerPoint is the currency for just about every presentation, conference, or other public event I attend. Imagine if we approached every event by deciding what effect we want to have upon the audience, instead of what slides we should create? Tufte stressed the power of sentences, saying sentences have "agency" but PowerPoint bullets do not. Sentences are tougher to write because they have nouns, verbs, and objects; bullets may have all, some, or none of those. PowerPoint also cripples arguments by stacking information in time and relying on the audience's short term memory. Instead, information should be arrayed in space, with as much spread out at once. The latter approach capitalizes on the human eye's "bandwidth of 80 Mbps per eye."

Credibility: Tufte emphasized that detail builds credibility, and audiences are constantly assessing the credibility of the speaker. Everything that can be documented and referenced and sourced should be; this resonated with my history degree. Every item of information should be communicative and should provide reasons to believe the speaker. Credibility arises from delivering an argument backed by evidence, and that material can be believed until an alternative explanation for the evidence, with as much rigor as the first explanation, appears. Speakers expand their credibility by explicitly addressing alternative explanations, rather than avoiding them.

Making an Impact: Too many of us exist in "flatland," i.e., the world of the computer screen, paper, and related media. To grab your audience's attention, bring something real from the 3D world to your presentation. This resonated with me too. At a recent week-long class for work with 42 other managers, I was told that some of the people in the class remembered me long after my initial introduction because I had a prop. The BusinessWeek magazine on "e-spionage" was on a table near me, so I told the class "I do this."

Image ref: Writing is About Putting Yourself to Words.

Presentation Design: Tufte advocates using what a colleague of mine calls a "placemat" format for delivering information. Tufte calls it a "tech report." Rather than standing in front of a PowerPoint slide deck, create a two-sided, 11" X 17" handout for the audience. Copy a format you've seen elsewhere. Pay attention to the pros; Tufte recommends Nature magazine for elite scientific and technical reporting or the New York Times for non-technical reporting. Include what Tufte calls a "supergraphic," an image that captures the audience's attention, like a highly detailed aerial photograph. (Whatever it is, ensure it is relevant to the audience!) He likes Gil Sans font. Include data on performance. There is no such thing as "information overload," only bad design. To clarify add detail -- don't remove it.

Fundamental Principles of Analytical Design:

1. Show comparisons.


2. Show causality.


3. Show multivariate data.


4. Integrate evidence; don't segregate by mode of production. (For example, Sports Illustrated's Web site has a "Video" section. Why aren't those videos simply next to the appropriate news stories?)


5. Document evidence.


6. Content above all else.

PowerPoint: PowerPoint only helps the bottom ten percent of speakers who would have no idea what to say without it. PowerPoint doesn't hinder the top ten percent of speakers who probably ignore their slides. PowerPoint devastates the middle 80 percent of speakers who think they are delivering information, when really they are (unconsciously) treating the audience as if they are too stupid to digest information in any other format. People can read 2-3 times faster than they can speak, so why should a presenter waste so much time with bullet points? Presentations should be a "high resolution data dump" (like a paper) and not a "press conference." Provide information in problem -> relevance -> solution format with a paragraph for each, with images, tables, "sparklines," and such integrated. You may use PowerPoint as a "projector operating system" ("POS," get it, get it?) to display tables, movies, or other media as necessary, but not as a bullet delivery tool.

Image ref: Presentation Zen: Contrasts in presentation style: Yoda vs. Darth Vader. Note the bullets are sentences, so they are actually more content-oriented than the usual PowerPoint bullets!

Active Person: The active person should be the audience, not the "speaker". Let the audience learn using its own cognitive style, not the method chosen by the presenter. Presenters should let speakers read the "placemat" or "tech report," then offer to answer questions. Asking questions is a sign that audience actually cares about the material. Leading the audience along a path chosen by the speaker, at the speaker's speed, using the speaker's cognitive style, and refusing to take questions because it "disrupts flow" is a disaster.

That's it for me. If you look a little you'll find other people's coverage of these training classes, like Colliers Atlas Blog or 21Apples.

What does this mean for me? I recently taught a one-day class on Network Security Operations. I decided to print the entire slide deck I've used for the last few years, suitable for a two or three day class. I decided to use that material solely as a reference, like Tufte uses his text books in his own classes. I asked the students what problems they were trying to solve in their own enterprises. Then I selected themes and spoke to them, using some of my slides as background or reference. I am trying to decide how to integrate this approach into my upcoming TCP/IP Weapons School class at Black Hat, which is mostly an examination of packet traces using Wireshark. I don't rely on slides for it.

Read More