Thanks to ronaldo in #snort-gui I learned about the LBNL/ICSI Enterprise Tracing Project. According to the site:
A goal of this project is to characterize internal enterprise traffic recorded at a medium-sized site, and to determine ways in which modern enterprise traffic is similar to wide-area Internet traffic, and ways in which it is quite different.
We have collected packet traces that span more than 100 hours of activity from a total of several thousand internal hosts. This wealth of data, which we are publicly releasing in anonymized form, spans a wide range of dimensions.
I decided to take a look at this data through the lens of Structured Traffic Analysis, which I discuss in Extrusion Detection and (IN)SECURE Magazine. I downloaded lbl-internal.20041004-1303.port001.dump.anon and took the following actions.
First I ran capinfos to get a sense of the nature of the trace.
$ sha256 lbl-internal.20041004-1303.port001.dump.anon
> lbl-internal.20041004-1303.port001.dump.anon.sha256
$ capinfos lbl-internal.20041004-1303.port001.dump.anon
File name: lbl-internal.20041004-1303.port001.dump.anon
File type: libpcap (tcpdump, Ethereal, etc.)
Number of packets: 84574
File size: 5907016 bytes
Data size: 33872987 bytes
Capture duration: 600.507393 seconds
Start time: Mon Oct 4 16:03:41 2004
End time: Mon Oct 4 16:13:41 2004
Data rate: 56407.28 bytes/s
Data rate: 451258.22 bits/s
Average packet size: 400.51 bytes
We can see this trace occupies 10 minutes in October 2004, at 451 Kbps, with 84574 packets.
Next I run Tcpdstat to learn a little more about the traffic.
$ tcpdstat lbl-internal.20041004-1303.port001.dump.anon
DumpFile: lbl-internal.20041004-1303.port001.dump.anon
FileSize: 5.63MB
Id: 200410041603
StartTime: Mon Oct 4 16:03:41 2004
EndTime: Mon Oct 4 16:13:41 2004
TotalTime: 600.51 seconds
TotalCapSize: 4.34MB CapLen: 74 bytes
# of packets: 84574 (32.30MB)
AvgRate: 451.17Kbps stddev:304.48K
### IP flow (unique src/dst pair) Information ###
# of flows: 260 (avg. 325.28 pkts/flow)
Top 10 big flow size (bytes/total in %):
37.9% 18.0% 15.8% 7.4% 6.8% 5.0% 1.3% 1.1% 0.7% 0.7%
### IP address Information ###
# of IPv4 addresses: 143
Top 10 bandwidth usage (bytes/total in %):
56.1% 55.9% 35.0% 23.0% 12.5% 2.7% 1.7% 1.3% 1.3% 1.0%
### Packet Size Distribution (including MAC headers) ###
<<<<
[ 32- 63]: 12784
[ 64- 127]: 17662
[ 128- 255]: 27008
[ 256- 511]: 7531
[ 512- 1023]: 2416
[ 1024- 2047]: 17173
>>>>
### Protocol Breakdown ###
<<<<
protocol packets bytes bytes/pkt
------------------------------------------------------------------------
[0] total 84574 (100.00%) 33872987 (100.00%) 400.51
[1] ip 84514 ( 99.93%) 33859701 ( 99.96%) 400.64
[2] tcp 82817 ( 97.92%) 33278039 ( 98.24%) 401.83
[3] http(s) 1727 ( 2.04%) 1251300 ( 3.69%) 724.55
[3] http(c) 1579 ( 1.87%) 267624 ( 0.79%) 169.49
[3] imap 488 ( 0.58%) 122352 ( 0.36%) 250.72
[3] ssh 176 ( 0.21%) 26337 ( 0.08%) 149.64
[3] other 78847 ( 93.23%) 31610426 ( 93.32%) 400.91
[2] udp 399 ( 0.47%) 88116 ( 0.26%) 220.84
[3] dns 50 ( 0.06%) 8669 ( 0.03%) 173.38
[3] other 349 ( 0.41%) 79447 ( 0.23%) 227.64
[2] icmp 375 ( 0.44%) 35880 ( 0.11%) 95.68
[2] ipsec 923 ( 1.09%) 457666 ( 1.35%) 495.85
>>>>
You get some of the same information as noted in Capinfos, but you also get some primitive protocol breakdowns. Unfortunately, 93.23% of the TCP traffic is unrecognized "other."
Let's see if Tethereal does any better:
taosecurity:/home/analyst/lbl$ tethereal -n -r lbl-internal.20041004-1303.port001.dump.anon -q -z io,phs
===================================================================
Protocol Hierarchy Statistics
Filter: frame
frame frames:84574 bytes:33872987
eth frames:84574 bytes:33872987
ip frames:84514 bytes:33859701
tcp frames:82817 bytes:33278039
udp frames:399 bytes:88116
isakmp frames:176 bytes:53996
short frames:176 bytes:53996
short frames:207 bytes:32742
short frames:923 bytes:457666
icmp frames:375 bytes:35880
short frames:30 bytes:11340
arp frames:28 bytes:1792
===================================================================
Unfortunately, Tethereal statistics don't tell you really anything different from Tcpdstat. Usually Tethereal statistics are more informative, but not here. For the sake of comparison, here is what Wireshark GUI statistics tell you.
Notice the format is different (but more human-friendly), and there is no way to copy or save it to a file. That would be a nice feature. (Tshark shows the same output as Tethereal, incidentally.)
The next step is to let Argus parse the file and then let Argus summarize the protocols it sees.
taosecurity:/home/analyst/lbl$ argus -r lbl-internal.20041004-1303.port001.dump.anon -w lbl.arg
taosecurity:/home/analyst/lbl$ ragator -r lbl.arg -w lbl.arg.ragator
taosecurity:/home/analyst/lbl$ racount -ar lbl.arg.ragator
racount records total_pkts src_pkts dst_pkts
total_bytes src_bytes dst_bytes
tcp 234 82817 39423 43394
33203201 10825712 22377489
udp 84 399 341 58
87969 77032 10937
icmp 36 375 224 151
35682 21416 14266
arp 4 28 28 0
1792 1792 0
non-ip 4 32 32 0
11494 11494 0
sum 363 83651 40048 43603
33340138 10937446 22402692
The next step is to see the IP addresses involved in this trace.
taosecurity:/home/analyst/lbl$ rahosts -nr lbl.arg.ragator
13.59.236.185
33.115.84.19
56.173.106.169
57.161.221.95
57.172.228.116
59.11.88.73
59.79.189.88
59.133.234.45
59.152.11.128
59.214.234.155
59.223.4.38
59.223.8.17
69.152.121.223
92.1.70.86
92.2.245.156
118.123.53.121
118.132.250.187
118.133.86.156
118.133.157.28
118.160.89.230
118.172.218.242
128.3.2.67
128.3.44.26
128.3.44.90
128.3.44.94
128.3.44.98
128.3.44.101
128.3.44.112
128.3.44.167
128.3.44.242
128.3.45.7
128.3.45.10
128.3.45.84
128.3.45.105
128.3.45.128
128.3.45.164
128.3.45.225
128.3.45.232
128.3.46.51
128.3.46.146
128.3.46.165
128.3.46.179
128.3.46.190
128.3.46.202
128.3.46.232
128.3.46.246
128.3.46.252
128.3.47.46
128.3.47.49
128.3.47.58
128.3.47.114
128.3.47.119
128.3.47.161
128.3.47.183
128.3.47.191
128.3.47.207
128.3.47.209
128.3.47.255
128.3.70.147
128.3.71.140
128.3.95.149
128.3.96.157
128.3.96.230
128.3.97.58
128.3.97.204
128.3.99.54
128.3.99.102
128.3.99.118
128.3.100.81
128.3.100.204
128.3.148.125
128.3.161.74
128.3.161.96
128.3.161.98
128.3.161.165
128.3.161.182
128.3.161.223
128.3.161.230
128.3.162.146
128.3.164.191
128.3.164.194
128.3.164.203
128.3.189.187
128.3.189.248
128.3.190.85
128.3.193.169
128.3.193.172
128.3.194.133
128.3.194.169
128.3.194.231
128.3.204.42
128.3.209.152
128.3.212.21
128.3.212.208
131.243.63.245
131.243.89.55
131.243.89.131
131.243.91.153
131.243.91.229
131.243.140.105
131.243.140.156
131.243.141.187
131.243.160.216
131.243.208.56
131.243.208.210
131.243.219.216
137.107.86.84
148.184.171.6
148.184.171.104
148.184.175.97
148.184.191.214
159.29.113.169
163.27.195.211
163.27.232.226
167.130.77.99
169.182.111.161
172.16.34.231
194.80.36.186
198.166.39.133
201.52.39.133
202.46.87.173
203.13.173.243
204.116.246.71
205.103.33.197
207.215.132.184
207.235.114.53
207.235.115.253
207.235.214.252
207.235.255.108
207.245.43.126
208.0.11.26
208.233.189.150
208.235.59.226
216.192.122.101
218.105.16.20
218.131.115.53
218.165.163.184
218.195.4.173
218.201.93.0
That's a lot of addresses for a 10 minute trace. Given the preponderance of 128.3.0.0/16 addresses, I'm guessing that is the HOME_NET.
The next step involves creating what I call session combinations. Essentially I remove the source port as a factor and I group on source IP, destination IP, and destination port.
taosecurity:/home/analyst/lbl$ ra -nn -r lbl.arg.ragator -s
saddr daddr dport proto | sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 |
uniq -c
1 a6:c6:c9:23:cc: a9:71:1d:9f:85: 321
1 3b:d:21:32:30:a 80:b:98:3b:b9:e 2457
1 33.115.84.19 128.3.47.46.5554 tcp
1 33.115.84.19 128.3.47.46.9898 tcp
1 33.115.84.19 128.3.44.101.5554 tcp
1 33.115.84.19 128.3.44.101.9898 tcp
1 33.115.84.19 128.3.45.105.5554 tcp
1 33.115.84.19 128.3.45.105.9898 tcp
1 33.115.84.19 128.3.46.146.5554 tcp
1 33.115.84.19 128.3.46.146.9898 tcp
1 33.115.84.19 128.3.46.202.5554 tcp
1 33.115.84.19 128.3.46.202.9898 tcp
1 33.115.84.19 128.3.46.232.5554 tcp
1 33.115.84.19 128.3.46.232.9898 tcp
1 33.115.84.19 128.3.47.209.5554 tcp
1 33.115.84.19 128.3.47.209.9898 tcp
1 34:c9:c8:fa:af: a9:71:1d:9f:85: 381
1 34:c9:c8:fa:af: a9:71:1d:9f:85: 390
1 69.152.121.223 128.3.46.179 icmp
1 118.132.250.187 128.3.44.112.1518 tcp
1 118.132.250.187 128.3.44.112.1525 tcp
4 128.3.44.26 128.3.190.85.143 tcp
1 128.3.44.26 128.3.47.255.138 udp
1 128.3.44.26 128.3.97.204.53 udp
4 128.3.44.26 128.3.164.194.143 tcp
1 128.3.44.26 128.3.189.187.138 udp
1 128.3.44.26 128.3.189.248 icmp
1 128.3.44.26 128.3.189.248.138 udp
1 128.3.44.26 128.3.189.248.139 tcp
1 128.3.44.26 128.3.189.248.2074 tcp
1 128.3.44.90 128.3.212.208.514 udp
1 128.3.44.98 128.3.97.204.53 udp
2 128.3.44.98 128.3.99.118.993 tcp
1 128.3.44.98 128.3.164.191.5730 tcp
1 128.3.44.101 128.3.97.58.123 udp
1 128.3.44.101 128.3.99.54.123 udp
2 128.3.44.112 59.11.88.73.80 tcp
5 128.3.44.112 59.223.4.38.80 tcp
2 128.3.44.112 59.223.8.17.80 tcp
1 128.3.44.112 128.3.47.255.137 udp
1 128.3.44.112 128.3.47.255.138 udp
3 128.3.44.112 128.3.97.204.53 udp
2 128.3.44.112 218.201.93.0.443 tcp
6 128.3.44.112 59.79.189.88.80 tcp
1 128.3.44.112 128.3.164.194.143 tcp
1 128.3.44.112 148.184.171.6 icmp
1 128.3.44.112 148.184.171.6.135 tcp
2 128.3.44.112 148.184.171.6.139 tcp
1 128.3.44.112 148.184.171.6.389 udp
2 128.3.44.112 148.184.171.6.445 tcp
2 128.3.44.112 218.105.16.20.80 tcp
2 128.3.44.112 218.195.4.173.80 tcp
2 128.3.44.112 118.133.157.28.80 tcp
4 128.3.44.112 118.133.86.156.80 tcp
2 128.3.44.112 148.184.175.97 icmp
1 128.3.44.112 148.184.175.97.135 tcp
1 128.3.44.112 148.184.175.97.139 tcp
2 128.3.44.112 148.184.175.97.389 udp
1 128.3.44.112 148.184.175.97.445 tcp
1 128.3.44.112 163.27.195.211.443 tcp
2 128.3.44.112 163.27.195.211.80 tcp
1 128.3.44.112 163.27.232.226.80 tcp
1 128.3.44.112 205.103.33.197.80 tcp
2 128.3.44.112 208.235.59.226.80 tcp
4 128.3.44.112 118.132.250.187.443 tcp
1 128.3.44.112 148.184.171.104 icmp
1 128.3.44.112 148.184.171.104.139 tcp
1 128.3.44.112 148.184.171.104.445 tcp
2 128.3.44.112 148.184.191.214.389 udp
2 128.3.44.112 207.235.214.252.80 tcp
1 128.3.44.112 207.235.255.108.5002 tcp
1 128.3.44.167 131.243.208.56.123 udp
1 128.3.44.242 128.3.212.208.514 udp
1 128.3.45.7 128.3.96.157.22 tcp
1 128.3.45.7 128.3.99.102.53 udp
1 128.3.45.10 208.0.11.26.80 tcp
1 128.3.45.10 128.3.47.255.137 udp
1 128.3.45.10 128.3.47.255.138 udp
1 128.3.45.10 128.3.97.204 icmp
2 128.3.45.10 128.3.97.204.53 udp
1 128.3.45.10 128.3.148.125.1521 tcp
26 128.3.45.10 137.107.86.84.80 tcp
1 128.3.45.10 203.13.173.243 icmp
1 128.3.45.10 203.13.173.243.53 udp
1 128.3.45.10 56.173.106.169.80 tcp
1 128.3.45.10 59.214.234.155.80 tcp
2 128.3.45.10 169.182.111.161.80 tcp
1 128.3.45.84 128.3.212.208.514 udp
1 128.3.45.105 128.3.96.157.67 udp
1 128.3.45.128 118.123.53.121.80 tcp
5 128.3.45.128 207.245.43.126.80 tcp
55 128.3.45.128 218.131.115.53.80 tcp
1 128.3.45.128 207.215.132.184.80 tcp
14 128.3.45.128 208.233.189.150.80 tcp
1 128.3.45.164 128.3.97.204.53 udp
1 128.3.45.164 128.3.161.182.139 tcp
1 128.3.45.164 128.3.161.223.138 udp
1 128.3.45.164 167.130.77.99.80 tcp
1 128.3.45.225 128.3.47.255.138 udp
1 128.3.45.225 128.3.70.147.161 udp
1 128.3.45.225 128.3.71.140.161 udp
6 128.3.45.225 128.3.97.204.53 udp
1 128.3.45.225 172.16.34.231.161 udp
1 128.3.45.232 202.46.87.173.80 tcp
1 128.3.46.51 128.3.212.208.514 udp
1 128.3.46.146 128.3.212.21 2054
1 128.3.46.146 128.3.96.230 2054
1 128.3.46.146 33.115.84.19 2054
1 128.3.46.146 128.3.162.146 2054
1 128.3.46.165 128.3.161.223.138 udp
1 128.3.46.165 128.3.161.223.139 tcp
1 128.3.46.165 128.3.161.223.2645 tcp
1 128.3.46.165 128.3.164.194.993 tcp
1 128.3.46.165 128.3.209.152 icmp
1 128.3.46.190 128.3.161.74 icmp
1 128.3.46.190 128.3.47.255.138 udp
1 128.3.46.190 128.3.161.165 icmp
1 128.3.46.190 128.3.161.223.139 tcp
1 128.3.46.190 128.3.161.230 icmp
1 128.3.46.190 128.3.164.194.993 tcp
1 128.3.46.190 131.243.141.187 icmp
1 128.3.46.246 128.3.209.152 icmp
4 128.3.46.252 128.3.95.149.111 udp
1 128.3.47.46 128.3.212.208.514 udp
1 128.3.47.49 131.243.219.216.137 udp
1 128.3.47.58 128.3.209.152 icmp
1 128.3.47.114 128.3.212.208.514 udp
1 128.3.47.119 128.3.47.255.138 udp
1 128.3.47.119 128.3.193.169.139 tcp
1 128.3.47.119 128.3.209.152 icmp
2 128.3.47.161 128.3.164.194.993 tcp
1 128.3.47.161 128.3.164.203.389 tcp
1 128.3.47.183 128.3.47.255.138 udp
1 128.3.47.183 128.3.189.248.139 tcp
1 128.3.47.183 204.116.246.71.1863 tcp
6 128.3.47.183 218.165.163.184.80 tcp
1 128.3.47.191 128.3.47.255.138 udp
1 128.3.47.191 131.243.89.131.161 udp
1 128.3.47.191 131.243.91.153.161 udp
1 128.3.47.191 131.243.91.229.161 udp
3 128.3.47.207 128.3.2.67.80 tcp
1 128.3.47.207 128.3.161.96.88 tcp
1 128.3.47.207 128.3.97.204.53 udp
1 128.3.47.207 128.3.164.194.993 tcp
1 128.3.47.207 128.3.193.169.139 tcp
1 128.3.47.207 128.3.193.169.80 tcp
2 128.3.47.207 128.3.193.172.80 tcp
1 128.3.47.207 128.3.194.133.161 udp
1 128.3.47.207 128.3.194.169.161 udp
1 128.3.47.207 128.3.194.231.161 udp
1 128.3.47.207 131.243.140.156 icmp
1 128.3.47.207 131.243.140.156.1026 tcp
1 128.3.47.207 131.243.140.156.135 tcp
1 128.3.47.207 131.243.140.156.445 tcp
1 128.3.96.157 128.3.45.105 icmp
1 128.3.96.230 128.3.47.46 icmp
1 128.3.96.230 128.3.44.101 icmp
1 128.3.96.230 128.3.45.105 icmp
1 128.3.96.230 128.3.46.146 icmp
7 128.3.96.230 128.3.46.146.161 udp
1 128.3.96.230 128.3.46.202 icmp
1 128.3.96.230 128.3.46.232 icmp
1 128.3.96.230 128.3.47.209 icmp
1 128.3.100.81 57.161.221.95.500 udp
1 128.3.100.81 59.133.234.45.500 udp
1 128.3.100.81 57.172.228.116.500 udp
1 128.3.100.81 118.172.218.242.500 udp
1 128.3.100.204 92.1.70.86.500 udp
1 128.3.100.204 92.2.245.156.500 udp
1 128.3.100.204 118.160.89.230.500 udp
1 128.3.100.204 131.243.63.245.500 udp
1 128.3.161.98 128.3.46.190.1050 tcp
1 128.3.161.165 128.3.46.190.1047 tcp
1 128.3.161.165 128.3.46.190.1048 tcp
1 128.3.161.223 128.3.46.165.139 tcp
1 128.3.162.146 128.3.46.146 icmp
1 128.3.164.191 128.3.44.98.4543 tcp
1 128.3.164.194 128.3.44.112.1395 tcp
1 128.3.204.42 128.3.44.26.38293 udp
1 128.3.209.152 128.3.47.58.38293 udp
1 128.3.209.152 128.3.46.165.38293 udp
1 128.3.209.152 128.3.46.246.38293 udp
1 128.3.209.152 128.3.47.119.38293 udp
1 128.3.212.21 128.3.46.146 icmp
1 128.3.212.208 128.3.44.90 icmp
1 128.3.212.208 128.3.44.94.137 udp
1 128.3.212.208 128.3.45.84 icmp
1 128.3.212.208 128.3.45.84.137 udp
1 128.3.212.208 128.3.46.51 icmp
1 128.3.212.208 128.3.46.51.137 udp
1 128.3.212.208 128.3.47.46 icmp
1 128.3.212.208 128.3.44.242 icmp
1 128.3.212.208 128.3.47.114 icmp
1 128.3.212.208 128.3.47.114.137 udp
1 131.243.89.55 128.3.47.58.139 tcp
1 131.243.140.105 128.3.46.190.1057 tcp
1 131.243.160.216 128.3.46.190.1119 tcp
1 131.243.208.210 128.3.44.167 icmp
1 148.184.191.214 128.3.44.112 icmp
1 194.80.36.186 128.3.46.232 icmp
1 207.235.114.53 128.3.47.183.4206 tcp
1 207.235.115.253 128.3.44.112.4973 tcp
1 216.192.122.101 128.3.44.94.49201 tcp
1 229.97.122.203 1 0 man
I like creating these session combinations because they show me connections to hosts and destination ports. I can review these target ports, for example, to look for sessions which might be interesting. This is as far as we can go, because all of the application layer details for these sessions have been eliminated by the Tcpmkpub anonymization tool.
At some point I plan to update this methodology using Argus 3.0, and automate the process.
0 komentar:
Posting Komentar