Where are the SIPs from? Part II

The following sequence of commands and output gives us some idea of what a network security analyst's life was like before the development of Intrusion Prevention Systems. After a laboriously contstructed tcpdump filter,we run this for 36 hours on our dummy/honeypot host. We then have a collections of SIPs sending packets to various ports. Some of these are easily identifiable well known attack ports (ms-sql, telnet, ssh,etc.)

/usr/local/sbin/tcpdump -s 0 -i xl0 -ttt -w out.txt 'port not(domain or whois or http or https or syslog or ntp or smtp or 137 or 139)' and 'not(broadcast or icmp or igmp or arp)'

# tcpdump -r out.txt

tcpdump: WARNING: snaplen raised from 96 to 65535

21:07:30.303358 218.7.164.30.6000 > 192.168.0.12.ms-sql-s: S 69599232:69599232(0) win 16384 [tos 0x20]

21:30:08.976883 202.99.11.99.1231 > 192.168.0.12.ms-sql-m: udp 376 [tos 0x20]

22:20:11.329910 catv-86-101-50-119.catv.broadband.hu.1077 > 192.168.0.12.38507: udp 28 [tos 0x20]

23:01:48.360917 10.13.3.61.1092 > 192.168.0.12.ms-sql-m: udp 376 [tos 0x20]

23:40:09.452031 117.0.33.129.3285 > 192.168.0.12.telnet: S 543288824:543288824(0) win 5808 (DF) [tos 0x20]

00:36:06.236501 dynamic.91.192.169.55.vpcit.ru.1076 > 192.168.0.12.38507: udp 30 [tos 0x20]

00:46:25.553206 c-98-242-240-55.hsd1.fl.comcast.net.37443 > 192.168.0.12.ssh: S 3044387924:3044387924(0) win 5840 (DF) [tos 0x20]

01:17:34.032666 213.0.55.130.34982 > 192.168.0.12.38507: udp 31 [tos 0x20]

01:29:12.722357 94.123.212.150.13597 > 192.168.0.12.12712: udp 30 [tos 0x20]

02:17:27.658034 121.15.245.215.12200 > 192.168.0.12.3128: S 484387749:484387749(0) win 8192 (DF) [tos 0x20]

02:33:20.751286 118.222.228.38.4692 > 192.168.0.12.6429: S 1557396635:1557396635(0) win 65535 (DF) [tos 0x20]

02:33:21.387784 118.222.228.38.4692 > 192.168.0.12.6429: S 1557396635:1557396635(0) win 65535 (DF) [tos 0x20]

02:33:21.990932 118.222.228.38.4692 > 192.168.0.12.6429: S 1557396635:1557396635(0) win 65535 (DF) [tos 0x20]

02:58:36.094717 61.153.26.60.1517 > 192.168.0.12.ms-sql-m: udp 376 [tos 0x20]

03:12:16.084284 222.215.230.49.12200 > 192.168.0.12.8000: S 788992279:788992279(0) win 8192 (DF) [tos 0x20]

03:15:54.079656 222.215.230.49.12200 > 192.168.0.12.3128: S 796332311:796332311(0) win 8192 (DF) [tos 0x20]

03:19:51.292696 221.195.73.68.6000 > 192.168.0.12.7212: S 1454440448:1454440448(0) win 16384 [tos 0x20]

03:19:51.294608 221.195.73.68.6000 > 192.168.0.12.8000: S 798031872:798031872(0) win 16384 [tos 0x20]

04:36:18.268102 210.51.165.30.33386 > 192.168.0.12.ssh: S 1493736546:1493736546(0) win 5840 (DF) [tos 0x20]

05:38:50.224275 static-39-92-224-77.ipcom.comunitel.net.61031 > 192.168.0.12.12712: udp 94 [tos 0x20]

05:55:59.481908 75-165-69-40.tukw.qwest.net.4004 > 192.168.0.12.telnet: S 481811338:481811338(0) win 5840 (DF) [tos 0x20]

06:36:17.744371 128.55.237.114.broad.lyg.js.dynamic.163data.com.cn.62161 > 192.168.0.12.5900: S 3649363145:3649363145(0) win 65535 (DF) [tos 0x20]

06:36:18.500456 128.55.237.114.broad.lyg.js.dynamic.163data.com.cn.62161 > 192.168.0.12.5900: S 3649363145:3649363145(0) win 65535 (DF) [tos 0x20]

07:20:05.328165 8.63.191.61.broad.static.hf.ah.cndata.com.6000 > 192.168.0.12.2967: S 1176764416:1176764416(0) win 16384 [tos 0x20]

....

After some awkward awk statements, we have the SIP list ready for processing by geoiplookup LiteCity database:

tcpdump -r out.txt | awk -F">" '{print $1}' | awk -F" " '{print $2}' | awk -F"." '{print $1"."$2"."$3"."$4}' >> out_IP.txt

for i in `cat out_IP.txt`; do echo $i : `geoiplookup $i -f /usr/local/share/GeoIP/GeoLiteCity.dat`; done

218.7.164.30 : GeoIP City Edition, Rev 1: CN, 08, Suihua, (null), 46.640598, 126.996902, 0, 0

202.99.11.99 : GeoIP City Edition, Rev 1: CN, 22, Beijing, (null), 39.928902, 116.388298, 0, 0

catv-86-101-50-119.catv.broadband.hu : GeoIP City Edition, Rev 1: HU, 23, Veszprém, (null), 47.099998, 17.916700, 0, 0

192.168.0.5 : GeoIP City Edition, Rev 1: IP Address not found

192.168.0.5 : GeoIP City Edition, Rev 1: IP Address not found

10.13.3.61 : GeoIP City Edition, Rev 1: IP Address not found

117.0.33.129 : GeoIP City Edition, Rev 1: VN, 44, Hanoi, (null), 21.033300, 105.849998, 0, 0

dynamic.91.192.169 : GeoIP City Edition, Rev 1: can't resolve hostname ( dynamic.91.192.169 )

c-98-242-240-55.hsd1.fl.comcast : GeoIP City Edition, Rev 1: can't resolve hostname ( c-98-242-240-55.hsd1.fl.comcast )

....

But let us suppose we want to track only connections from the United States:

for i in `cat out_IP.txt`; do echo $i : `geoiplookup $i` | grep 'United States'; done

75-165-69-40.tukw.qwest.net : GeoIP Country Edition: US, United States

128.55.237.114 : GeoIP Country Edition: US, United States

128.55.237.114 : GeoIP Country Edition: US, United States

8.63.191.61 : GeoIP Country Edition: US, United States

173.1.171.82 : GeoIP Country Edition: US, United States

12.4.209.243 : GeoIP Country Edition: US, United States

208-110-155-97.customer.csolutions.net : GeoIP Country Edition: US, United States

Traceroutes to particular IPs are notoriously useless:

# traceroute 8.63.191.61

traceroute to 8.63.191.61 (8.63.191.61), 64 hops max, 40 byte packets

1 192.168.0.1 (192.168.0.1) 1.179 ms 1.153 ms 1.143 ms

2 * * *

3 68.87.207.113 (68.87.207.113) 11.692 ms 9.661 ms 9.477 ms

4 te-5-3-ur02.ferndale.wa.seattle.comcast.net (68.86.96.110) 9.541 ms 11.897 ms 10.621 ms

5 te-0-9-0-7-ar01.seattle.wa.seattle.comcast.net (68.86.96.105) 12.404 ms 13.245 ms 13.285 ms

6 pos-0-3-0-0-cr01.seattle.wa.ibone.comcast.net (68.86.90.209) 13.652 ms 12.481 ms 13.850 ms

7 te-3-2.car1.Seattle1.Level3.net (4.79.104.105) 15.77 ms 13.320 ms 13.734 ms

8 * * *

9 * * *

10 * * *

11 * * *

^C

# traceroute -P ICMP 8.63.191.61

traceroute to 8.63.191.61 (8.63.191.61), 64 hops max, 60 byte packets

1 192.168.0.1 (192.168.0.1) 0.460 ms 0.328 ms 0.289 ms

2 * * *

3 68.87.207.113 (68.87.207.113) 12.610 ms 8.716 ms 8.629 ms

4 te-5-3-ur02.ferndale.wa.seattle.comcast.net (68.86.96.110) 9.434 ms 8.360 ms 13.671 ms

5 te-0-9-0-7-ar01.seattle.wa.seattle.comcast.net (68.86.96.105) 12.748 ms 11.914 ms 11.173 ms

6 pos-0-3-0-0-cr01.seattle.wa.ibone.comcast.net (68.86.90.209) 19.632 ms 13.667 ms 12.556 ms

7 te-3-2.car1.Seattle1.Level3.net (4.79.104.105) 14.819 ms 13.441 ms 13.465 ms

8 * * *

9 * * *

10 * *

So now we have spent quite a bit of time extracting data for just one day's worth of activity. Obviously what is needed is a more comprehensive approach: real-time alerting, database, web-based information, long-term analysis and mapping. This is what you buy with your Intrusion Prevention System...

Read More

Insider Threat Myth Documentation

In my first book The Tao of Network Security Monitoring, published in July 2004, I tried to trace the origin of the "80% myth". In the following section reprinted from pages 31-34, and newly annotated now, I document what this means for insider vs outsider threat. (This section is also posted here at Informit.com.)

---

OUTSIDERS VERSUS INSIDERS: WHAT IS NSM’S FOCUS?

This book is about network security monitoring. I use the term network to emphasize the book’s focus on traffic and incidents that occur over wires, radio waves, and other media. This book does not address intruders who steal data by copying it onto a USB memory stick or burning it to a CD-ROM. Although the focus for much of the book is on outsiders gaining unauthorized access, it pertains equally well to insiders who transfer information to remote locations. In fact, once an outsider has local access to an organization, he or she looks very much like an insider. [10]

Should this book (and NSM) pay more attention to insiders? One of the urban myths of the computer security field holds that 80% of all attacks originate from the inside. This “statistic” is quoted by anyone trying to sell a product that focuses on detecting attacks by insiders. An analysis of the most respected source of computer security statistics, the Computer Crime and Security Survey conducted annually by the Computer Security Institute (CSI) and the FBI, sheds some light on the source and interpretation of this figure. [11] [Bejtlich: I question saying "most respected" now, but I wrote that in 2004 before we had other reporting.]

The 2001 CSI/FBI study quoted a commentary by Dr. Eugene Schultz that first appeared in the Information Security Bulletin. Dr. Schultz was asked:

I keep hearing statistics that say that 80 percent of all attacks are from the inside. But then I read about all these Web defacements and distributed denial of service attacks, and it all doesn’t add up. Do most attacks really originate from the inside?

Dr. Schultz responded:

There is currently considerable confusion concerning where most attacks originate. Unfortunately, a lot of this confusion comes from the fact that some people keep quoting a 17-year-old FBI statistic that indicated that 80 percent of all attacks originated from the [inside]...

Should [we] ignore the insider threat in favor of the outsider threat? On the contrary. The insider threat remains the greatest single source of risk to organizations. Insider attacks generally have far greater negative impact to business interests and operations. Many externally initiated attacks can best be described as ankle-biter attacks launched by script kiddies.

But what I am also saying is that it is important to avoid underestimating the external threat. It is not only growing disproportionately, but is being fueled increasingly by organized crime and motives related to espionage. I urge all security professionals to conduct a first-hand inspection of their organization’s firewall logs before making a claim that most attacks come from the inside. Perhaps most successful attacks may come from the inside (especially if an organization’s firewalls are well configured and maintained), true, but that is different from saying that most attacks originate from the inside. [12]

Dr. Dorothy Denning, some of whose papers are discussed in Appendix B, confirmed Dr. Shultz’s conclusions. Looking at the threat, noted by the 2001 CSI/FBI study as “likely sources of attack,” Dr. Denning wrote in 2001:

For the first time, more respondents said that independent hackers were more likely to be the source of an attack than disgruntled or dishonest insiders (81% vs. 76%).

Perhaps the notion that insiders account for 80% of incidents no longer bears any truth whatsoever. [13]

The 2002 and 2003 CSI/FBI statistics for “likely sources of attack” continued this trend. At this point, remember that the statistic in play is “likely sources of attack,” namely the party that embodies a threat. In addition to disgruntled employees and independent hackers, other “likely sources of attack” counted by the CSI/FBI survey include foreign governments (28% in 2003), foreign corporations (25%), and U.S. competitors (40%).

Disgruntled employees are assumed to be insiders (i.e., people who can launch attacks from inside an organization) by definition. Independent hackers are assumed to not be insiders. But from where do attacks actually originate? What is the vector to the target? The CSI/FBI study asks respondents to rate “internal systems,” “remote dial-in,” and “Internet” as “frequent points of attack.” In 2003, 78% cited the Internet, while only 30% cited internal systems and 18% cited dial-in attacks. In 1999 the Internet was cited at 57% while internal systems rated 51%. These figures fly in the face of the 80% statistic.

A third figure hammers the idea that 80% of all attacks originate from the inside. The CSI/FBI study asks for the origin of incidents involving Web servers. For the past five years, incidents caused by insiders accounted for 7% or less of all Web intrusions. In 2003, outsiders accounted for 53%. About one-quarter of respondents said they “don’t know” the origin of their Web incidents, and 18% said “both” the inside and outside participated.

At this point the idea that insiders are to blame should be losing steam. Still, the 80% crowd can find solace in other parts of the 2003 CSI/FBI study. The study asks respondents to rate “types of attack or misuse detected in the last 12 months.” In 2003, 80% of participants cited “insider abuse of net access” as an “attack or misuse,” while only 36% confirmed “system penetration.” “Insider abuse of net access” apparently refers to inappropriate use of the Internet; as a separate statistic, “unauthorized access by insiders” merited a 45% rating.

If the insider advocates want to make their case, they should abandon the 80% statistic and focus on financial losses. The 2003 CSI/FBI study noted “theft of proprietary information” cost respondents over $70 million; “system penetration” cost a measly $2.8 million. One could assume that insiders accounted for this theft, but that might not be the case. The study noted “unauthorized access by insiders” cost respondents only $406,000 in losses. [14]

Regardless of your stance on the outsider versus insider issue, any activity that makes use of the network is a suitable focus for analysis using NSM. Any illicit action that generates a packet becomes an indicator for an NSM operation. One of the keys to devising a suitable NSM strategy for your organization is understanding certain tenets of detection, outlined next.

Footnotes for these pages:

10. Remember that “local access” does not necessarily equate to “sitting at a keyboard.” Local access usually means having interactive shell access on a target or the ability to have the victim execute commands of the intruder’s choosing.

11. You can find the CSI/FBI studies in .pdf format via Google searches. The newest edition can be downloaded from http://www.gosci.com.

12. Read Dr. Schultz’s commentary in full at http://www.chi-publishing.com. Look for the editorial in Information Security Bulletin, volume 6, issue 2 (2001). Adding to the confusion, Dr. Shultz’s original text used “outside” instead of “inside,” as printed in this book. The wording of the question and the thesis of Dr. Shultz’s response clearly show he meant to say “inside” in this crucial sentence. [Looking back on this five years later, I am still confused by Dr. Schultz's meaning. If he really meant to say "some people keep quoting a 17-year-old FBI statistic that indicated that 80 percent of all attacks originated from the outside," then why not say "this 17-year-old FBI statistic is the opposite of your claim?"]

13. Dr. Dorothy Denning, as quoted in the 2001 CSI/FBI Study.

14. Foreshadowing the popularization of “cyberextortion” via denial of service, the 2003 CSI/FBI study reported “denial of service” cost over $65 million—second only to “theft of proprietary information” in the rankings.

---

My biggest regret reading this section involves trying to interpret Dr. Schultz's comments. If anyone can find a copy of an "FBI study" from approximately 1984 that discusses insider vs outsider threat, please let me know!

Reading this section now, I see the primary value as finding documentation that the "80% myth" refers to the idea that "80 percent of all attacks are from the inside." If you agree that an attack is not the same as an "incident," then you can see how Dr. Denning's comment about "the notion that insiders account for 80% of incidents" introduces more problems by talking about incidents and not attacks. If someone wants to throw "risk" in there, you now have a third meaning.

What I find sad is that so many people carelessly cite the "FBI" or "CSI" studies as supporting whatever "80%" claim they want, but if asked to point to the actual study they could never do so. In my first book I at least tried to document what was available at that time.

---

Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

Read More

Where are the SIPs from?

The geoiplookup utility is very helpful in assessing the country where the SIPs (Source Internet Protocol Address) are from. We might be able to say 'hackers' instead of SIPs, if it were ultimately possible to deduce from a SIP the originators of an attack on any given Firewall. And that is important to remember: IPs can be spoofed, the country of origin tells you nothing about the operator themselves, criminals and nation-states terrorists can be leased. So withhold judgement based on IP!!

However,if I log to a file a tcpdump something like this:

/usr/local/sbin/tcpdump -s 0 -i xl0 -n -tt 'port not(ssh or domain or whois or http or https or syslog or ntp or 137 or 139)' and 'not(broadcast or icmp or igmp or arp)'

My dummy/honeypot host receives about 100 or so entries per day from foreign IPs that look like this:

1241804920.098123 IP 85.249.160.55.1225 > 192.168.0.12.38507: UDP, length 20

1241805774.527866 IP 201.67.52.249.59649 > 192.168.0.12.12712: UDP, length 21

1241806316.686063 IP 218.6.12.230.6000 > 192.168.0.12.2967: Flags [S], seq 114163712, win 16384, length 0

1241806357.627835 IP 61.153.26.60.1517 > 192.168.0.12.1434: UDP, length 376

1241807555.257870 IP 91.150.223.226.1519 > 192.168.0.12.38507: UDP, length 63

1241813077.431641 IP 98.247.212.4.1980 > 192.168.0.12.23: Flags [SEW], seq 1693462630, win 5840, options [mss 1460,sackOK,TS val 117815450 ecr 0,nop,wscale 0], length 0

...

The syslog entries that my Netgear FVS318 firewall is forwarding to my OpenBSD dummy/honeypot host contain similar attacks but many more legitimate connection attempts like this:

May 9 09:06:45 192.168.0.1 rferris Blocked Sites Log[46161]:portforward forwarded , SIP:216.35.67.135: 80, DIP:98.247.182.78: 19899,

May 9 09:07:06 192.168.0.1 rferris Blocked Sites Log[46166]:portforward forwarded , SIP:68.87.69.146: 53, DIP:98.247.182.78: 17337,

May 9 09:07:06 192.168.0.1 rferris Blocked Sites Log[46169]:portforward forwarded , SIP:76.96.30.119: 110, DIP:98.247.182.78: 17681,

I can get an excellent approximation of those IPs whose port attempts simply can be easily counted as part of any possibly normal connectivity by subtracting (as above in the tcpdump filters) those SIP ports that are not mail, whois, dns, http, ntp, etc. :

grep -f file1 syslog | grep -v -f file2 | awk -F":" '{print $5}' | sort | uniq >> out.txt

where file1 is:

Hacker

Blocked

and file2 is:

25,

43,

53,

80,

110,

123,

587,

As the astute reader will point, this technique completely neglects hacking attacks from legitimate ports! And also much legitimate traffic comes over port 443 (ssl) which I have excluded here.(Of course, that is what an IDS is for... Much more on that later...) The next step in my investigation is lookup the country of origin for the IP. Batching whois queries are frowned upon. GeoIPLookup fills this need:

for i in `cat out.txt`; do echo $i:`geoiplookup $i`;done;

10.13.3.61:GeoIP Country Edition: IP Address not found

113.56.251.166:GeoIP Country Edition: CN, China

115.132.83.188:GeoIP Country Edition: MY, Malaysia

116.3.98.197:GeoIP Country Edition: CN, China

116.54.196.108:GeoIP Country Edition: CN, China

117.127.93.18:GeoIP Country Edition: CN, China

118.100.85.190:GeoIP Country Edition: MY, Malaysia

118.22.208.254:GeoIP Country Edition: JP, Japan

....

Hacking is an international activity. It is nice to be silently stalked by so many foreign countries!

for i in `cat out.txt`; do geoiplookup $i >> geoiplookup.txt ;done;

cat geoiplookup.txt | sort | uniq -c | sort -r

55 GeoIP Country Edition: CN, China

20 GeoIP Country Edition: US, United States

10 GeoIP Country Edition: RU, Russian Federation

10 GeoIP Country Edition: BR, Brazil

8 GeoIP Country Edition: UA, Ukraine

5 GeoIP Country Edition: VN, Vietnam

4 GeoIP Country Edition: MY, Malaysia

4 GeoIP Country Edition: KR, Korea, Republic of

4 GeoIP Country Edition: IT, Italy

3 GeoIP Country Edition: FR, France

2 GeoIP Country Edition: TR, Turkey

2 GeoIP Country Edition: JP, Japan

2 GeoIP Country Edition: GB, United Kingdom

2 GeoIP Country Edition: DE, Germany

2 GeoIP Country Edition: CA, Canada

1 GeoIP Country Edition: ZA, South Africa

1 GeoIP Country Edition: VE, Venezuela

1 GeoIP Country Edition: UY, Uruguay

1 GeoIP Country Edition: TW, Taiwan

1 GeoIP Country Edition: TH, Thailand

1 GeoIP Country Edition: SI, Slovenia

1 GeoIP Country Edition: SE, Sweden

1 GeoIP Country Edition: QA, Qatar

1 GeoIP Country Edition: PL, Poland

1 GeoIP Country Edition: PH, Philippines

1 GeoIP Country Edition: PA, Panama

1 GeoIP Country Edition: NZ, New Zealand

1 GeoIP Country Edition: NO, Norway

1 GeoIP Country Edition: MX, Mexico

1 GeoIP Country Edition: MD, Moldova, Republic of

1 GeoIP Country Edition: JM, Jamaica

1 GeoIP Country Edition: IP Address not found

1 GeoIP Country Edition: IN, India

1 GeoIP Country Edition: HK, Hong Kong

1 GeoIP Country Edition: FI, Finland

1 GeoIP Country Edition: EG, Egypt

1 GeoIP Country Edition: CO, Colombia

What of "IP Address not found"!? (Another question to resolve..) However, let us suppose we are just interested the U.S. connections. Keep in mind some of these are legitimate ssl (443) connections.

# for i in `cat out.txt`; do echo $i:` geoiplookup $i` | grep "United States";done;

152.26.20.72:GeoIP Country Edition: US, United States

168.75.65.98:GeoIP Country Edition: US, United States

173.1.171.82:GeoIP Country Edition: US, United States

173.69.171.116:GeoIP Country Edition: US, United States

173.8.113.195:GeoIP Country Edition: US, United States

205.214.57.202:GeoIP Country Edition: US, United States

208.111.159.155:GeoIP Country Edition: US, United States

209.85.201.125:GeoIP Country Edition: US, United States

63.226.235.106:GeoIP Country Edition: US, United States

63.231.190.174:GeoIP Country Edition: US, United States

64.251.8.230:GeoIP Country Edition: US, United States

66.35.46.195:GeoIP Country Edition: US, United States

68.142.94.151:GeoIP Country Edition: US, United States

68.37.225.206:GeoIP Country Edition: US, United States

72.42.151.135:GeoIP Country Edition: US, United States

74.63.193.230:GeoIP Country Edition: US, United States

98.220.41.92:GeoIP Country Edition: US, United States

98.247.182.78:GeoIP Country Edition: US, United States

98.247.212.4:GeoIP Country Edition: US, United States

99.152.215.137:GeoIP Country Edition: US, United States

It is useful to see how much information can be gained without packet inspection. next up...tracking SIPs to their networks...

Read More

Highlights from 2009 Verizon Data Breach Report

Last year I posted Verizon Business Report Speaks Volumes, providing excerpts that resonated with me. Verizon released another edition last month, with plenty of commentary on their blog and elsewhere. I wanted to record a few highlights here for my own reference but also to counter arguments I continue to see elsewhere about the so-called prevalence of insider threats.

This is a polite way of trying to demolish the most deeply entrenched urban myth in security history.



This shows the 2009 results.



This is an historical way to look at breach source data.



The following chart is the one that insider threat proponents will try to use to justify their position. It shows that, on average, a breach caused by a single insider will result in many more records being stolen than one caused by an outsider. Incidentally, this is what I have said previously as well!



However, when looking at the problem in aggregate, outsiders cause more damage.



If the big red dot doesn't say it all, I don't know what will.

Verizon captures this scenario using a "pseudo-risk" calculation.



Pete Lindstrom makes an interesting point about this calculation, but I don't think it is necessarily without merit.

I'd like to briefly turn to the detection and response elements I found interesting.

The following shows someone from Verizon has been to the Best Single Day Class Ever. That big red dot shows "months" from compromise to discovery is dominant.



Detection methods continue to be pathetic.



This is probably because, although logs are collected, hardly anyone reviews them.



This is probably because only a third of companies have an IR team.



Most companies are probably relying on their anti-virus software to save them. This is too bad, because the explosion in customized malware means it probably won't.



All of this is why my TCP/IP Weapons School 2.0 class teaches students how to analyze data to detect and respond to intrusions, rather than rely on automated tools which fail.

---

Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

Read More

Logs from the Cloud

I received an email with the following notice today:

Amazon CloudFront Adds Access Logging Capability:

AWS today released access logs for Amazon CloudFront. Access logs are activity records that show you details about every request delivered through Amazon CloudFront. They contain a comprehensive set of information about requests for your content, including the object requested, the date and time of the request, the edge location serving the request, the client IP address, the referrer and the user agent. It’s easy to get started using access logs: you just specify the name of the Amazon S3 bucket you want to use to store the logs when you configure your Amazon CloudFront distribution. There are no fees for using the access logs, beyond normal Amazon S3 charges to write, store and retrieve the logs.



The Amazon Elastic MapReduce team has also built a sample application, CloudFront LogAnalyzer, that will analyze your Amazon CloudFront access logs. This tool lets you use the power of Amazon Elastic MapReduce to quickly turn Access Logs into the answers to the most commonly asked questions about your business. Additionally, several partners have also built solutions that help you analyze these access logs; you can find more information about these in the AWS Solutions Catalog.

Looking at the Developer Guide entry for Access Logs, we see the following sorts of data will be recorded:

The log files use the W3C extended log file format 
 (for more information, go to http://www.w3.org/TR/WD-logfile.html).

The files contain information for each record in the following order:

Date of the request (in UTC)
Time (when the server finished processing the request; in UTC)
Edge location that served the request 
 (a variable-length string with a minimum of 3 characters)
Bytes served
Client IP address (no hostname lookups occur)
HTTP access method
DNS name (either the CloudFront distribution name or your CNAME, 
 whichever the end user specified in the request)
URI stem (e.g., /images/daily-ad.jpg)
HTTP status code (e.g., 200)
Referrer
User agent

An entry might look like this:

#Version: 1.0
#Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status 
 cs(Referer) cs(User-Agent)

02/01/2009 01:13:11 FRA2 182 10.10.10.10 GET d2819bc28.cloudfront.net /view/my/file.html 200 
 www.displaymyfiles.com Mozilla/4.0%20(compatible;%20MSIE%205.0b1;%20Mac_PowerPC)

02/01/2009 01:13:12 LAX1 2390282 12.12.12.12 GET www.singalong.com /soundtrack/happy.mp3 304 
 www.unknownsingers.com Mozilla/4.0%20(compatible;%20MSIE%207.0;%20Windows%20NT%205.1) 

I think this is a good start, but I'll leave it to Cloudsecurity.org for expert commentary!

---

Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

Read More

Thoughts on Cyber Command

I've been blogging about various cyber command proposals for a few years, but right now there is some real movement at the combatant command level. Ellen Nakashima's article Cyber-Command May Help Protect Civilian Networks offers the latest details.

The Pentagon is considering whether to create a new cyber-command that would oversee government efforts to protect the military's computer networks and would also assist in protecting the civilian government networks, the head of the National Security Agency said yesterday [Tuesday].

The new command would be headquartered at Fort Meade, the NSA's director, Lt. Gen. Keith B. Alexander, told the House Armed Services terrorism subcommittee.

Alexander, who is a front-runner to assume control of the command if it is created, said its focus would be to better protect the U.S. military's computers by marrying the offensive and defensive capabilities of the military and the NSA.

Through the command, the NSA would also provide technical support to the Department of Homeland Security, which is in charge of protecting civilian networks and helps safeguard the energy grid and other critical infrastructure from cyber-attack, Alexander said.

He stressed that the NSA does not want to run or operate the civilian networks, but help Homeland Security improve its efforts...

As proposed by the Pentagon, the command would fall under the U.S. Strategic Command, which is tasked with defending against attacks on vital interests.

The highlighted sections reinforce number 2 of my Predictions for 2008 made in December 2007. A few months prior I argued that the US Needs Cyber NORAD.

The written testimonies are posted on the U.S. House of Representatives, House Armed Services Committee Web site.

The new Cyber Command will most likely be a subordinate unified command under US Strategic Command.

I'd like to briefly respond to Robert Graham's post Why Cyber Commands Fail. He says in part:

What the military wants is a hacker squad that they can give a specific objective, and have the hackers carry out that objective within a specific timeframe. For example, they might tell hackers to take out Iran's radar at midnight so that fighter jets can enter their airspace a few minutes later to bomb their nuclear plants. That's not going to work.

What you could do is tell hackers to go after Iran and do whatever they can to disrupt their nuclear developments. One hacker might find a way to shut down safety controls and cause a nuclear meltdown, another might jam the centrifuges, another might change the firmware on measuring equipment to incorrect measure the concentration of U238.

Or, you could give the hackers six months to infiltrate Iran's computers, then come back with a list of options. Maybe disabling the radar system will be one of them, maybe not. But that's not the sort of thing the military is tasked to do - that's more an intelligence operation the CIA would be doing..

China and Russia understand this. They don't directly employ hackers or tell the hackers to accomplish certain goals. They let the hackers have free range to do whatever they want. If the hackers come across something interesting, such as plans for the Joint Strike Fighter, the government buys it, but no government official ever told the hackers specifically to steal those plans...

So how can the United States get in on this sort of asymmetric warfare action?

The first thing is that you have to stoke some sort of nationalism in the way that Russia and China do. I'm not sure this is in our character (especially under the current president), however, so we'd probably have to find some alternative. Instead of pro-USA nationalism we could instead focus on human rights activism. The government could spend a lot of time talking to the press about the sorts of human rights abuses that go on in Russia and China. Get our own USA hackers thinking about human rights as their own causus belli.

The second thing they need to do is create a climate where our own hackers can operate. I would gladly hack into Iranian computers, but I'm not sure how this fits into US law...

This would be similar to the "letters of mark and reprisal" used by governments during the 1700s. In those days, national navies were too small to patrol the entire ocean. Therefore, governments licensed privateers to prey upon a hostile nation's shipping. The privateers kept half the booty, and gave the other half to their respective government. This is essentially what China and Russia have done.

A third thing our military would need to do is train our hackers in the target language. Foreign hackers usually learn English, but American hackers rarely learn foreign languages, especially Russian, Chinese, or Farsi (Iranian). If we want to encourage our hackers to go after those countries in the same way they come after us, we need to encourage them to learn those languages...

The fourth thing our military would need to do is fix their horrid purchasing processes...

Note that I think the individuals who run our military are very, very smart. I've met several generals and colonels who understand this. The problem is that while individuals are smart, the organization is dumb as a rock. The organization crushes precisely the sort of creative thinking need to have a successful "cyber" offensive capability.

Robert has a lot of good ideas here. In Air Force Cyber Panel I talked about a clash of models between the United States and places like China. On the one hand we have a military-industrial complex supported by a vast contracting force vs a country with a true "people's army," containing uniformed military, semi-military, and pure civilians who work with the others to achieve broadly common goals.

I don't think we will ever see any official support for the privateer concept. China doesn't even recognize their own people's involvement in hacking, since they frequently repeat the line that "China doesn't support hacking."

The major benefit I see from a Cyber Command is providing a career path and organizational support for military personnel. Until that exists many people who would want to be in the military doing cyber operations will reach a point where leaving their service is their best option.

---

Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

Read More

Port 6000 Scans

I receive lots of 6000 port scans on my Netgear FVS318 Firewall. My guess is many of you do as well. In changing the Firewall to forward all the ports to an OpenBSD "dummy/honeypot" host, I am able to sniff the packets that come into my WAN (without a tap), thus increasing my information/control over them vs. Firewall syslog output. X11 can be configured with default remote port on 6000. My guess is I receive a two dozen plus IPs attempting to connect with this port per week:

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode

listening on xl0, link-type EN10MB (Ethernet), capture size 65535 bytes

...

1241648685.844326 IP 218.58.220.130.6000 > 192.168.0.12.2967: Flags [S], seq 1519714304, win 16384, length 0

1241649491.716071 IP 80.13.217.205.6000 > 192.168.0.12.2967: Flags [S], seq 1519714304, win 16384, options [mss 1460], length 0

....

1241653701.125407 IP 221.195.73.68.6000 > 192.168.0.12.7212: Flags [S], seq 1454440448, win 16384, length 0

1241653701.128528 IP 221.195.73.68.6000 > 192.168.0.12.8000: Flags [S], seq 798031872, win 16384, length 0

1241656783.765870 IP 119.161.130.75.6000 > 192.168.0.12.2967: Flags [S], seq 640548864, win 16384, length 0

....

/var/log# grep 119.161.130.75 syslog

[Under FVS318 normal operation]:

May 4 10:45:13 192.168.0.1 rferris [59299]:TCP(2967) Dest IP :98.247.182.78, Src IP :119.161.130.75

May 4 12:43:54 192.168.0.1 rferris [59342]:TCP(2967) Dest IP :98.247.182.78, Src IP :119.161.130.75

May 4 17:36:29 192.168.0.1 rferris [59440]:TCP(2967) Dest IP :98.247.182.78, Src IP :119.161.130.75

May 4 22:28:39 192.168.0.1 rferris [59534]:TCP(2967) Dest IP :98.247.182.78, Src IP :119.161.130.75

May 5 03:52:54 192.168.0.1 rferris [62240]:TCP(2967) Dest IP :98.247.182.78, Src IP :119.161.130.75

May 5 09:05:59 192.168.0.1 rferris [64532]:TCP(2967) Dest IP :98.247.182.78, Src IP :119.161.130.75

May 5 09:05:59 192.168.0.1 rferris Hacker Log[64533]:PROTO_TCP, SIP:119.161.130.75: 6000, DIP:98.247.182.78: 2967, Suspicious TCP Data

May 5 14:36:22 192.168.0.1 rferris [64635]:TCP(2967) Dest IP :98.247.182.78, Src IP :119.161.130.75

May 5 19:34:38 192.168.0.1 rferris [64831]:TCP(2967) Dest IP :98.247.182.78, Src IP :119.161.130.75

May 6 00:34:49 192.168.0.1 rferris [70599]:TCP(2967) Dest IP :98.247.182.78, Src IP :119.161.130.75

May 6 05:22:39 192.168.0.1 rferris [73607]:TCP(2967) Dest IP :98.247.182.78, Src IP :119.161.130.75

May 6 10:11:50 192.168.0.1 rferris [76793]:TCP(2967) Dest IP :98.247.182.78, Src IP :119.161.130.75

May 6 14:11:41 192.168.0.1 rferris [76868]:TCP(2967) Dest IP :98.247.182.78, Src IP :119.161.130.75

[With dummy/honeypot host] FVS318 performs as such

May 6 20:15:15 192.168.0.1 rferris [216]:TCP(2967) Dest IP :98.247.182.78, Src IP :119.161.130.75

May 6 20:15:15 192.168.0.1 rferris Blocked Sites Log[217]:portforward forwarded , SIP:119.161.130.75: 6000, DIP:98.247.182.78: 2967,

May 7 01:04:14 192.168.0.1 rferris [5964]:TCP(2967) Dest IP :98.247.182.78, Src IP :119.161.130.75

May 7 01:04:14 192.168.0.1 rferris Blocked Sites Log[5965]:portforward forwarded , SIP:119.161.130.75: 6000, DIP:98.247.182.78: 2967,

May 7 05:57:33 192.168.0.1 rferris [10106]:TCP(2967) Dest IP :98.247.182.78, Src IP :119.161.130.75

May 7 05:57:33 192.168.0.1 rferris Blocked Sites Log[10107]:portforward forwarded , SIP:119.161.130.75: 6000, DIP:98.247.182.78: 2967,

Read More