Bejtlich/Bianco ShmooCon DVD

Thanks to David Bianco, I received a copy of a DVD of our ShmooCon 2006 presentation, Network Security Monitoring with Sguil (.ppt). The cover is posted at left, and clicking on it will show a larger version. I am not sure if the Feds will appreciate the Che Guevara theme the next time my security clearance is reviewed. If you want to order you own copy, you can visit MediaArchives.com. As far as I know I do not get a penny from DVD sales, unless there was some hidden clause in the form Heidi Potter asked me to sign!

By the way, this blog has been on fire this month. Where are you all coming from? If you started reading this blog this month, would you mind posting a comment saying where you heard of it, and if you plan to return next month? Thank you!

Read More

Miss the Internet of the 1970s? It's still here.

Imagine the following conversation took place some time before 15 January 2001.

Alice: "Why don't we create a Web page that anyone can edit?"

Bob: "Cool. How do we prevent 'bad people' from posting 'bad things'?" [Note that "bad people" and "bad things" are entirely subjective.]

Alice: "Don't worry, people will be nice."

Bob: "What if they are not nice?"

Alice: "We'll keep track of the IP addresses people use to post content. We'll block bad IP addresses."

Bob: "What if bad people post bad content using anonymous proxy servers? What about NAT, such that hundreds of people can be using the same public IP address?"

Alice: "Don't frighten me with your sorcerer's ways."

Bob: "So what do we call this system?"

Alice: "Wikipedia!"

Now, people are shocked -- shocked I say -- when anyone can edit pages they would wish said something else.

The Wikipedia model works when the user community is small and the participants trust each other. When was the last time that was true? Oh yes -- the Internet of the 1970s. Is that true now, or at least in 2001 when Wikipedia was founded? Well, the community was certainly smaller back then. But a small user base does not hold up well as a defense model. As Wikipedia's activity has grown, it has attracted the attention of people who are more likely to act maliciously. Sounds the Internet as a whole, from the 1970s into the 1980s, followed by the explosion of users in the 1990s.

I would personally never use Wikipedia as a resource for any serious research. I might use it as a starting point, but why should I trust what it says? Am I going to go back through the editing history and note that 195.89.26.53 made a change that looks suspicious, but 216.192.4.32 seems more reliable? That is ridiculous.

I think Wikipedia is fundamentally broken. Here's what would reduce it to scrap -- a MalWikiBot. The MalWikiBot would edit Wikipedia pages at random. Maybe it would replace whole sentences with material found on other Web pages. Perhaps it would change dates, measurements, and other numeric quantities. MalWikiBot couldn't be blocked using existing Wikipedia techniques because it would use bot nets to appear to come from legitimate IP addresses. On some days it would delete whole pages, but that would be far too obvious. Better to silently corrupt small sections of data in a manner not immediately obvious.

Wikipedia is going to need to at least restrict changes to authenticated users. Sign up with a username and an email address. At the very least a MalWikiBot writer would need to overcome that small hurdle before changing pages on the fly. If the current Wikipedia "security model" continues, I predict ongoing decline as users lose faith in the integrity of its data.

Read More

DoD 8570.01-M Posted

Thanks to David Bianco for sending me to this article about the manual for DoD 8570.1 being posted here. The .pdf looks like a scan of a hard copy document. I couldn't search it using xpdf.

Read More

Review of Running IPv6 Posted

Amazon.com just posted my five-star review of Running IPv6 by Iljitsch van Beijnum. It is so much easier to write reviews for great books! From the review:

"When I read and reviewed O'Reilly's IPv6 Network Administration by Niall Richard Murphy and David Malone, I called their book "a must-have book for all network administrators." Upon seeing Apress' Running IPv6 by Iljitsch van Beijnum, I wondered if I would waste my time reading and reviewing another book on IPv6. Now I'm glad I digested Running IPv6 -- it's my first must-read book of 2006. The books are complementary, so I recommend them both."

What a great book.

Read More

IPv6 Behind NAT Using FreeBSD and Miredo

Thanks to the generosity of a TaoSecurity Blog reader, I have been experimenting with a dual-stack IPv4 and IPv6 system at a university. I connect to the IPv4 address using OpenSSH. Once on the box, I can use IPv6.

I've been looking for ways to connect my home network directly to IPv6. At the moment I'm using a common gateway/router to perform NAT for my cable network connection. I needed a way to provide IPv6 for systems behind the NAT. Enter Teredo and the Miredo project.

Now, before you decide that I'm giving this protocol my "thumbs up," I'm going to explicitly tell you I just wanted to get the software working and use ping6. That's it for now.

Teredo, which is now a draft RFC, is a Microsoft protocol. Basically you take IPv6 traffic, tunnel it in UDP, and send it to a relay server. The relay pulls off the UDP and sends the traffic using IPv6 to the destination. The process is reversed for return traffic. Obviously sending your traffic elsewhere, especially to one of the Microsoft relays, is enough to scare most people.

Installing Miredo is simple. Thanks to author Rémi Denis-Courmont responding to my troubleshooting emails, the latest version compiles flawlessly on FreeBSD 6.0. The standard ./configure, make, make install is all that is needed.

Once installed, I run Miredo in the foreground.

orr:/home/richard$ sudo miredo --foreground

***********************************************************************
*                          IMPORTANT NOTICE                           *
*                                                                     *
* At the time of release of this version of the program, the IETF had *
* not yet published the Teredo protocol specification (RFC). As such, *
* this version of the program still uses experimental provisional     *
* settings, which will most likely be altered when the specification  *
* is published. A new version of the program will then be released to *
* take these changes into consideration. Until then, this program     *
* might not work properly and should be considered experimental.      *
***********************************************************************

miredo[684]: Starting...
miredo[685]: Qualified (NAT type: restricted)
miredo[685]: Teredo pseudo-tunnel started
miredo[685]:  (address: 3ffe:831f:8ac3:9ddd:0:3650:ba0c:d759, MTU: 1280)

Miredo creates a tun0 interface for IPv6.

orr:/home/richard$ ifconfig -a  
fxp0: flags=8943 mtu 1500
        options=8
        inet6 fe80::203:47ff:fe0f:1f3c%fxp0 prefixlen 64 scopeid 0x1 
        inet 192.168.2.5 netmask 0xffffff00 broadcast 192.168.2.255
        ether 00:03:47:0f:1f:3c
        media: Ethernet autoselect (100baseTX )
        status: active
plip0: flags=108810 mtu 1500
lo0: flags=8049 mtu 16384
        inet6 ::1 prefixlen 128 
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 
        inet 127.0.0.1 netmask 0xff000000 
tun0: flags=80d1 mtu 1280
        inet6 fe80::203:47ff:fe0f:1f3c%tun0 prefixlen 64 scopeid 0x4 
        inet6 fe80::5445:5245:444f%tun0 prefixlen 64 scopeid 0x4 
        inet6 3ffe:831f:8ac3:9ddd:0:3650:ba0c:d759 prefixlen 32 
        Opened by PID 685

Teredo encapsulates IPv6 inside UDP packets sent to port 3544.

Here is what it looks like to Tcpdump when Teredo starts. All we can see at this point is Miredo doing a DNS lookup for its default relay server, followed by UDP traffic to that server.

11:08:15.389255 IP 192.168.2.5.64226 > 192.168.2.1.53:  58453+ A? teredo.via.ecp.fr. (35)
11:08:15.394528 IP 192.168.2.1.53 > 192.168.2.5.64226:  58453 1/0/0 A 138.195.157.221 (51)
11:08:15.395024 IP 192.168.2.5.51631 > 138.195.157.221.3544: UDP, length 77
11:08:19.396616 IP 192.168.2.5.51631 > 138.195.157.221.3544: UDP, length 77
11:08:23.396031 IP 192.168.2.5.51631 > 138.195.157.221.3544: UDP, length 77
11:08:27.396404 IP 192.168.2.5.51631 > 138.195.157.222.3544: UDP, length 77
11:08:27.517795 IP 138.195.157.222.3544 > 192.168.2.5.51631: UDP, length 117
11:08:27.518031 IP 192.168.2.5.51631 > 138.195.157.221.3544: UDP, length 77
11:08:27.639923 IP 138.195.157.221.3544 > 192.168.2.5.51631: UDP, length 117
11:09:01.396212 IP 192.168.2.5.51631 > 138.195.157.221.3544: UDP, length 77
11:09:01.514967 IP 138.195.157.221.3544 > 192.168.2.5.51631: UDP, length 117

Tethereal strips off the UDP traffic by default and shows the underlying IPv6 traffic. Keep this in mind if you're using Tethereal and think you're seeing native IPv6. This is the same trace as examined above with Tcpdump.

Now with the help of Tethereal, we see Miredo making ICMPv6 router solicitations. Later we see ICMPv6 router advertisements from fe80::8000:dd8:753c:6222.

  1 2006-01-30 11:08:15.389255  192.168.2.5 -> 192.168.2.1  DNS Standard query A
 teredo.via.ecp.fr
  2 2006-01-30 11:08:15.394528  192.168.2.1 -> 192.168.2.5  DNS Standard query response A 138.195.157.221
  3 2006-01-30 11:08:15.395024 fe80::8000:5445:5245:444f -> ff02::2      ICMPv6 Router solicitation
  4 2006-01-30 11:08:19.396616 fe80::8000:5445:5245:444f -> ff02::2      ICMPv6 Router solicitation
  5 2006-01-30 11:08:23.396031 fe80::8000:5445:5245:444f -> ff02::2      ICMPv6 Router solicitation
  6 2006-01-30 11:08:27.396404 fe80::5445:5245:444f -> ff02::2      ICMPv6 Router solicitation
  7 2006-01-30 11:08:27.517795 fe80::8000:dd8:753c:6222 -> fe80::5445:5245:444f ICMPv6 Router advertisement
  8 2006-01-30 11:08:27.518031 fe80::5445:5245:444f -> ff02::2      ICMPv6 Router solicitation
  9 2006-01-30 11:08:27.639923 fe80::8000:dd8:753c:6222 -> fe80::5445:5245:444f 
ICMPv6 Router advertisement
 12 2006-01-30 11:09:01.396212 fe80::5445:5245:444f -> ff02::2      ICMPv6 Router solicitation
 13 2006-01-30 11:09:01.514967 fe80::8000:dd8:753c:6222 -> fe80::5445:5245:444f ICMPv6 Router advertisement

Here is what the routing table for IPv6 looks like once Miredo is running.

orr:/home/richard$ netstat -nr -f inet6
Routing tables

Internet6:
Destination                       Gateway                       Flags      Netif Expire
default                           link#4                        ULS        tun0
::1                               ::1                           UH          lo0
3ffe:831f::/32                    link#4                        UC         tun0
3ffe:831f:8ac3:9ddd:0:3650:ba0c:d759 link#4                        UHL         lo0
fe80::%fxp0/64                    link#1                        UC         fxp0
fe80::203:47ff:fe0f:1f3c%fxp0     00:03:47:0f:1f:3c             UHL         lo0
fe80::%lo0/64                     fe80::1%lo0                   U           lo0
fe80::1%lo0                       link#3                        UHL         lo0
fe80::%tun0/64                    link#4                        UC         tun0
fe80::5445:5245:444f%tun0         link#4                        UHL         lo0
fe80::203:47ff:fe0f:1f3c%tun0     link#4                        UHL         lo0
ff01::/32                         ::1                           U           lo0
ff02::%fxp0/32                    link#1                        UC         fxp0
ff02::%lo0/32                     ::1                           UC          lo0
ff02::%tun0/32                    link#4                        UC         tun0

Interface tun0 is the default for IPv6. That is good news. Let's try to ping6 an IPv6 enabled host.

orr:/home/richard$ ping6 -c 2 www6.olympus-zone.net
PING6(56=40+8+8 bytes) 3ffe:831f:8ac3:9ddd:0:3650:ba0c:d759 --> 2001:1638:305:4::1
16 bytes from 2001:1638:305:4::1, icmp_seq=0 hlim=56 time=766.023 ms
16 bytes from 2001:1638:305:4::1, icmp_seq=1 hlim=56 time=394.536 ms

--- www6.olympus-zone.net ping6 statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 394.536/580.279/766.023/185.744 ms

Awesome. Here is how Tcpdump sees the traffic.

11:09:16.646506 IP 192.168.2.5.55205 > 192.168.2.1.53:  58174+ AAAA? www6.olympus-zone.net. (39)
11:09:16.648419 IP 192.168.2.1.53 > 192.168.2.5.55205:  58174 1/0/0 AAAA 2001:1638:305:4::1 (67)
11:09:16.649738 IP 192.168.2.5.51631 > 138.195.157.221.3544: UDP, length 64
11:09:16.977910 IP 138.195.157.221.3544 > 192.168.2.5.51631: UDP, length 48
11:09:16.978090 IP 192.168.2.5.51631 > 213.172.48.140.51246: UDP, length 40
11:09:17.130790 IP 213.172.48.140.51246 > 192.168.2.5.51631: UDP, length 64
11:09:17.130940 IP 192.168.2.5.51631 > 213.172.48.140.51246: UDP, length 56
11:09:17.415124 IP 213.172.48.140.51246 > 192.168.2.5.51631: UDP, length 56
11:09:17.649746 IP 192.168.2.5.51631 > 213.172.48.140.51246: UDP, length 56
11:09:18.043784 IP 213.172.48.140.51246 > 192.168.2.5.51631: UDP, length 56

Packet 3 would appear to be the ICMPv6 request, with packet 4 the response. But what about the last 6 packets?

Here is the same traffic in Tethereal.

 14 2006-01-30 11:09:16.646506  192.168.2.5 -> 192.168.2.1  DNS Standard query AAAA www6.olympus-zone.net
 15 2006-01-30 11:09:16.648419  192.168.2.1 -> 192.168.2.5  DNS Standard query response AAAA 2001:1638:305:4::1
 16 2006-01-30 11:09:16.649738 3ffe:831f:8ac3:9ddd:0:3650:ba0c:d759 -> 2001:1638:305:4::1 ICMPv6 Echo request
 17 2006-01-30 11:09:16.977910 fe80::8000:5445:5245:444f -> 3ffe:831f:8ac3:9ddd:0:3650:ba0c:d759 IPv6 IPv6 no next header
 18 2006-01-30 11:09:16.978090  192.168.2.5 -> 213.172.48.140 UDP Source port: 51631  Destination port: 51246
 19 2006-01-30 11:09:17.130790 213.172.48.140 -> 192.168.2.5  UDP Source port: 51246  Destination port: 51631
 20 2006-01-30 11:09:17.130940  192.168.2.5 -> 213.172.48.140 UDP Source port: 51631  Destination port: 51246
 21 2006-01-30 11:09:17.415124 213.172.48.140 -> 192.168.2.5  UDP Source port: 51246  Destination port: 51631
 22 2006-01-30 11:09:17.649746  192.168.2.5 -> 213.172.48.140 UDP Source port: 51631  Destination port: 51246
 23 2006-01-30 11:09:18.043784 213.172.48.140 -> 192.168.2.5  UDP Source port: 51246  Destination port: 51631

Tethereal sees the ICMPv6 request and reply, but it can't decode the last 6 packets.

I plan to investigate this further.

Read More

FreeBSD Networking over FireWire

You might be familiar with Apple's implementation of IP over FireWire. This allows connecting two computers directly over FireWire ports. FreeBSD offers two drivers that provide networking over FireWire. fwe is a non-standard protocol, but it is implemented by default in the GENERIC kernel. fwip implements RFC 2734 (IPv4 over IEEE 1394) and RFC 3146 (Transmission of IPv6 Packets over IEEE 1394 Networks); it is available via kernel module.

I decided to have my laptop orr talk to my server janney using FireWire. To implement FireWire, orr uses an Adaptec DuoConnect PC Card Adapter and janney uses an Adaptec DuoConnect PCI Adapter. Both provide FireWire and USB 2.0.

Each system is running FreeBSD 6.0.

The laptop dmesg sees the following when the FireWire adapter is inserted.

cardbus0: CIS pointer is 0!
cardbus0: Resource not specified in CIS: id=10, size=800
cardbus0: Resource not specified in CIS: id=14, size=4000
fwohci0:  mem 0x88004000-0x880047ff,0x88000000-0x88
003fff irq 11 at device 0.0 on cardbus0
fwohci0: OHCI version 1.10 (ROM=1)
fwohci0: No. of Isochronous channels is 4.
fwohci0: EUI64 08:00:28:56:02:00:49:8a
fwohci0: Phy 1394a available S400, 3 ports.
fwohci0: Link S400, max_rec 2048 bytes.
firewire0:  on fwohci0
fwe0:  on firewire0
if_fwe0: Fake Ethernet address: 0a:00:28:00:49:8a
fwe0: Ethernet address: 0a:00:28:00:49:8a
fwe0: if_start running deferred for Giant
sbp0:  on firewire0
fwohci0: Initiate bus reset
fwohci0: node_id=0xc000ffc0, gen=1, CYCLEMASTER mode
firewire0: 1 nodes, maxhop <= 0, cable IRM = 0 (me)
firewire0: bus manager 0 (me)
cardbus0: CIS pointer is 0!
cardbus0: Resource not specified in CIS: id=10, size=1000
ohci0:  mem 0x88000000-0x88000fff irq 11 at device
0.4 on cardbus0
ohci0: [GIANT-LOCKED]
usb1: OHCI version 0.0
usb1: unsupported OHCI revision
ohci0: USB init failed

The server dmesg sees the following at boot.

ohci0:  mem 0xf7fff000-0xf7ffffff irq 22 at device 
8.0 on pci4
ohci0: [GIANT-LOCKED]
usb1: OHCI version 1.0
usb1:  on ohci0
usb1: USB revision 1.0
uhub1: NEC OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 3 ports with 3 removable, self powered
ohci1:  mem 0xf7ffe000-0xf7ffefff irq 21 at device 
8.1 on pci4
ohci1: [GIANT-LOCKED]
usb2: OHCI version 1.0
usb2:  on ohci1
usb2: USB revision 1.0
uhub2: NEC OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub2: 2 ports with 2 removable, self powered
ehci0:  mem 0xf7ffdc00-0xf7ffdcff irq 20 at d
evice 8.2 on pci4
ehci0: [GIANT-LOCKED]
usb3: EHCI version 0.95
usb3: companion controllers, 3 ports each: usb1 usb2
usb3:  on ehci0
usb3: USB revision 2.0
uhub3: NEC EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub3: 5 ports with 5 removable, self powered
fwohci0:  mem 0xf7ffd000-0xf7ffd7ff,0xf7ff8000-0xf7
ffbfff irq 22 at device 12.0 on pci4
fwohci0: OHCI version 1.10 (ROM=1)
fwohci0: No. of Isochronous channels is 4.
fwohci0: EUI64 00:50:42:b5:c0:0d:0d:af
fwohci0: Phy 1394a available S400, 3 ports.
fwohci0: Link S400, max_rec 2048 bytes.
firewire0:  on fwohci0
fwe0:  on firewire0
if_fwe0: Fake Ethernet address: 02:50:42:0d:0d:af
fwe0: Ethernet address: 02:50:42:0d:0d:af
fwe0: if_start running deferred for Giant
sbp0:  on firewire0
fwohci0: Initiate bus reset
fwohci0: node_id=0xc800ffc1, gen=1, CYCLEMASTER mode
firewire0: 2 nodes, maxhop <= 1, cable IRM = 1 (me)
firewire0: bus manager 1 (me)

Here is the fwe interface that is created when the FireWire adapter is active on orr.

fwe0: flags=108802 mtu 1500
        options=8
        ether 0a:00:28:00:49:8a
        ch 1 dma -1

Here is the fwe interfaced that is created when the FireWire adapter is active on janney.

fwe0: flags=108943 mtu 1500
        options=8
        inet6 fe80::50:42ff:fe0d:daf%fwe0 prefixlen 64 scopeid 0x2 
        inet 10.1.1.2 netmask 0xffffff00 broadcast 10.1.1.255
        ether 02:50:42:0d:0d:af
        ch 1 dma 0

When I plug a FireWire cable into each host, dmesg on orr sees the following.

fwohci0: BUS reset
fwohci0: node_id=0xc800ffc1, gen=2, CYCLEMASTER mode
firewire0: 2 nodes, maxhop <= 1, cable IRM = 1 (me)
firewire0: bus manager 1 (me)
firewire0: New S400 device ID:005042b5c00d0daf

Server janney sees similar messages.

fwohci0: BUS reset
fwohci0: node_id=0x8800ffc0, gen=4, non CYCLEMASTER mode
firewire0: 2 nodes, maxhop <= 1, cable IRM = 1
firewire0: bus manager 1 
firewire0: New S400 device ID:080028560200498a

First I assign an IP to fwe0 on orr.

orr:/home/richard$ sudo ifconfig fwe0 inet 10.1.1.1 netmask 255.255.255.0 up
orr:/home/richard$ ifconfig fwe0
fwe0: flags=108943 mtu 1500
        options=8
        inet6 fe80::800:28ff:fe00:498a%fwe0 prefixlen 64 scopeid 0x4 
        inet 10.1.1.1 netmask 0xffffff00 broadcast 10.1.1.255
        ether 0a:00:28:00:49:8a
        ch 1 dma 0

Next I assign an IP to fwe0 on janney.

janney:/home/richard$ sudo ifconfig fwe0 inet 10.1.1.2 netmask 255.255.255.0 up
janney:/home/richard$ ifconfig fwe0
fwe0: flags=108943 mtu 1500
        options=8
        inet6 fe80::50:42ff:fe0d:daf%fwe0 prefixlen 64 scopeid 0x2 
        inet 10.1.1.2 netmask 0xffffff00 broadcast 10.1.1.255
        ether 02:50:42:0d:0d:af
        ch 1 dma 0

Now the two systems can communicate over FireWire.

janney:/home/richard$ ping 10.1.1.1
PING 10.1.1.1 (10.1.1.1): 56 data bytes
64 bytes from 10.1.1.1: icmp_seq=0 ttl=64 time=0.694 ms
64 bytes from 10.1.1.1: icmp_seq=1 ttl=64 time=0.333 ms
64 bytes from 10.1.1.1: icmp_seq=2 ttl=64 time=0.312 ms
^C
--- 10.1.1.1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.312/0.446/0.694/0.175 ms

One can sniff the fwe0 interface with Tcpdump.

07:36:42.479420 IP6 fe80::50:42ff:fe0d:daf > ff02::1:ff0d:daf: HBH ICMP6, multicast listener 
 reportmax resp delay: 0 addr: ff02::1:ff0d:daf, length 24
07:36:42.479667 IP6 fe80::50:42ff:fe0d:daf > ff02::2:f2f0:bec6: HBH ICMP6, multicast listener
 reportmax resp delay: 0 addr: ff02::2:f2f0:bec6, length 24
07:36:42.479818 arp who-has 10.1.1.2 tell 10.1.1.2
07:36:42.498496 IP6 :: > ff02::1:ff0d:daf: ICMP6, neighbor solicitation, who has
 fe80::50:42ff:fe0d:daf, length 24
07:36:43.756817 IP6 fe80::50:42ff:fe0d:daf > ff02::1:ff0d:daf: HBH ICMP6, multicast listener 
 reportmax resp delay: 0 addr: ff02::1:ff0d:daf, length 24
07:36:50.962703 IP6 fe80::50:42ff:fe0d:daf > ff02::2:f2f0:bec6: HBH ICMP6, multicast listener
 reportmax resp delay: 0 addr: ff02::2:f2f0:bec6, length 24
07:36:54.422718 arp who-has 10.1.1.1 tell 10.1.1.2
07:36:54.422793 arp reply 10.1.1.1 is-at 0a:00:28:00:49:8a
07:36:54.423012 IP 10.1.1.2 > 10.1.1.1: ICMP echo request, id 34306, seq 0, length 64
07:36:54.423050 IP 10.1.1.1 > 10.1.1.2: ICMP echo reply, id 34306, seq 0, length 64

You can see that the fwe driver supports IPv6. The last four packets are IPv4.

Next I decided to try the fwip driver. I loaded the fwip kernel module on each system. First, orr.

orr:/home/richard$ sudo kldload fwip
orr:/home/richard$ kldstat
Id Refs Address    Size     Name
 1   12 0xc0400000 63072c   kernel
 2    2 0xc0a31000 74b0     snd_csa.ko
 3    3 0xc0a39000 1d408    sound.ko
 4    1 0xc0a57000 c3a4     r128.ko
 5    2 0xc0a64000 eeec     drm.ko
 6   16 0xc0a73000 568dc    acpi.ko
 7    1 0xc2236000 5000     if_fwip.ko

This created the fwip0 interface, which I gave an IP address.

orr:/home/richard$ ifconfig fwip0
fwip0: flags=108802 mtu 1500
        lladdr 8.0.28.56.2.0.49.8a.a.2.ff.fe.0.0.0.0
orr:/home/richard$ sudo ifconfig fwip0 inet 172.16.1.1 netmask 255.255.255.0 up
orr:/home/richard$ ifconfig fwip0
fwip0: flags=108843 mtu 1500
        inet6 fe80::a00:2856:200:498a%fwip0 prefixlen 64 scopeid 0x5 
        inet 172.16.1.1 netmask 0xffffff00 broadcast 172.16.1.255
        lladdr 8.0.28.56.2.0.49.8a.a.2.ff.fe.0.0.0.0

Notice dmesg output for orr.

fwip0:  on firewire0
fwip0: Firewire address: 08:00:28:56:02:00:49:8a @ 0xfffe00000000, S400, maxrec 2048

Now, janney.

janney:/home/richard$ sudo kldload fwip
janney:/home/richard$ kldstat 
Id Refs Address    Size     Name
 1    5 0xc0400000 641298   kernel
 2   16 0xc0a42000 568dc    acpi.ko
 3    1 0xc214a000 5000     if_fwip.ko
janney:/home/richard$ ifconfig fwip0
fwip0: flags=108802 mtu 1500
        lladdr 0.50.42.b5.c0.d.d.af.a.2.ff.fe.0.0.0.0
janney:/home/richard$ sudo ifconfig fwip0 inet 172.16.1.2 netmask 255.255.255.0 up
janney:/home/richard$ ifconfig fwip0
fwip0: flags=108843 mtu 1500
        inet6 fe80::250:42b5:c00d:daf%fwip0 prefixlen 64 scopeid 0x6 
        inet 172.16.1.2 netmask 0xffffff00 broadcast 172.16.1.255
        lladdr 0.50.42.b5.c0.d.d.af.a.2.ff.fe.0.0.0.0

Notice janney's dmesg.

fwip0:  on firewire0
fwip0: Firewire address: 00:50:42:b5:c0:0d:0d:af @ 0xfffe00000000, S400, maxrec 2048

Now the two systems can communicate using the fwip driver.

janney:/home/richard$ ping -c 1 172.16.1.1
PING 172.16.1.1 (172.16.1.1): 56 data bytes
64 bytes from 172.16.1.1: icmp_seq=0 ttl=64 time=0.733 ms

--- 172.16.1.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.733/0.733/0.733/0.000 ms

You can also sniff fwip0. Take a look at the differences.

orr:/home/richard$ tcpdump -n -r fwe0.short.lpc -c 2 icmp
reading from file fwe0.short.lpc, link-type EN10MB (Ethernet)
07:36:54.423012 IP 10.1.1.2 > 10.1.1.1: ICMP echo request, id 34306, seq 0, length 64
07:36:54.423050 IP 10.1.1.1 > 10.1.1.2: ICMP echo reply, id 34306, seq 0, length 64

orr:/home/richard$ tcpdump -n -r fwip0.short.lpc -c 2 icmp
reading from file fwip0.short.lpc, link-type APPLE_IP_OVER_IEEE1394 (Apple IP-over-IEEE 1394)
07:41:52.593940 IP 172.16.1.2 > 172.16.1.1: ICMP echo request, id 38914, seq 0, length 64
07:41:52.593970 IP 172.16.1.1 > 172.16.1.2: ICMP echo reply, id 38914, seq 0, length 64

The first reports "Ethernet" because fwe is an Ethernet emulation layer. The second reports Apple IP-over-IEEE 1394 because that is an entirely new protocol. Check it out in Tethereal.

Frame 8 (102 bytes on wire, 102 bytes captured)
    Arrival Time: Jan 30, 2006 07:41:52.593940000
    Time delta from previous packet: 0.000231000 seconds
    Time since reference or first frame: 13.327087000 seconds
    Frame Number: 8
    Packet Length: 102 bytes
    Capture Length: 102 bytes
    Protocols in frame: ap1394:ip:icmp:data
Apple IP-over-IEEE 1394, Src: 005042B5C00D0DAF, Dst: 80B2FFC100000000
    Destination: 80B2FFC100000000
    Source: 005042B5C00D0DAF
    Type: IP (0x0800)
Internet Protocol, Src: 172.16.1.2 (172.16.1.2), Dst: 172.16.1.1 (172.16.1.1)
    Version: 4
    Header length: 20 bytes
    Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
        0000 00.. = Differentiated Services Codepoint: Default (0x00)
        .... ..0. = ECN-Capable Transport (ECT): 0
        .... ...0 = ECN-CE: 0
    Total Length: 84
    Identification: 0x0696 (1686)
    Flags: 0x00
        0... = Reserved bit: Not set
        .0.. = Don't fragment: Not set
        ..0. = More fragments: Not set
    Fragment offset: 0
    Time to live: 64
    Protocol: ICMP (0x01)
    Header checksum: 0x19f0 [correct]
        Good: True
        Bad : False
    Source: 172.16.1.2 (172.16.1.2)
    Destination: 172.16.1.1 (172.16.1.1)
Internet Control Message Protocol
    Type: 8 (Echo (ping) request)
    Code: 0 
    Checksum: 0xe88d [correct]
    Identifier: 0x9802
    Sequence number: 0x0000
    Data (56 bytes)

0000  43 de 09 a3 00 09 3e e2 08 09 0a 0b 0c 0d 0e 0f   C.....>.........
0010  10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f   ................
0020  20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f    !"#$%&'()*+,-./
0030  30 31 32 33 34 35 36 37                           01234567

I noticed each host reported error messages like the following. I believe these were caused by the fwip driver.

fwohci0: txd err= 3 miss Ack err

I believe fwip has a future going forward, since it implements a standard.

For a rough idea of how fast these interfaces were, I transferred the same 100 MB file using the fwe, fwip, and fxp/xl wired Ethernet interfaces. Here are the results.

1. fwe: 104857600 bytes received in 00:09 (10.16 MB/s)


2. fwip: 104857600 bytes received in 00:09 (11.01 MB/s)


3. fxp/xl through a switch: 104857600 bytes received in 00:08 (11.21 MB/s)


4. fxp/xl via crossover cable: 104857600 bytes received in 00:09 (10.75 MB/s)

All the results are roughly the same, so the bottleneck is probably the hard drive of the laptop.

This opens some interesting possibilities for networking. Maybe I will buy a FireWire hub and connect multiple machines simultaneously.

I hope to try some of the debugging and memory access features of FireWire in the future.

Read More

QEMU on FreeBSD, with Networking

Maybe you've heard of QEMU, an "open source processor emulator." It's not quite VMware, since there doesn't seem to be a concept of persistent state and there are definitely not snapshots. However, when I saw the variety of ready-to-run system images at OSZoo.org, I decided to try it on FreeBSD 6.0.

Luckily there are several QEMU ports. I installed emulators/qemu from the latest FreeBSD 6.0 package. I next installed emulators/kqemu-kmod using the port.

janney:/root# cd /usr/ports/emulators/kqemu-kmod
janney:/usr/ports/emulators/kqemu-kmod# make
=> kqemu-0.7.2.tar.gz doesn't seem to exist in /usr/ports/distfiles/kqemu.
=> Attempting to fetch from http://fabrice.bellard.free.fr/qemu/.
kqemu-0.7.2.tar.gz                            100% of   77 kB  102 kBps
===>  Extracting for kqemu-kmod-0.7.2_1
=> MD5 Checksum OK for kqemu/kqemu-0.7.2.tar.gz.
=> SHA256 Checksum OK for kqemu/kqemu-0.7.2.tar.gz.
===>  Patching for kqemu-kmod-0.7.2_1
===>  Applying FreeBSD patches for kqemu-kmod-0.7.2_1
===>  Configuring for kqemu-kmod-0.7.2_1
===>  Building for kqemu-kmod-0.7.2_1
Warning: Object directory not changed from original /usr/ports/emulators/kqemu-kmod/work/kqemu
@ -> /usr/src/sys
machine -> /usr/src/sys/i386/include
cc -O2 -fno-strict-aliasing -pipe -D_KERNEL -DKLD_MODULE -nostdinc -I-   -I. -I@ 
-I@/contrib/altq -I@/../include -I/usr/include -finline-limit=8000 -fno-common  
-mno-align-long-strings -mpreferred-stack-boundary=2  -mno-mmx -mno-3dnow -mno-sse -mno-sse2 
-ffreestanding -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes  -Wmissing-prototypes
-Wpointer-arith -Winline -Wcast-qual  -fformat-extensions -std=c99 -c kqemu-freebsd.c
ld  -d -warn-common -r -d -o kqemu.kld kqemu-mod-i386.o kqemu-freebsd.o
touch export_syms
awk -f /sys/conf/kmod_syms.awk kqemu.kld  export_syms | xargs -J% objcopy % kqemu.kld
ld -Bshareable  -d -warn-common -o kqemu.ko kqemu.kld
objcopy --strip-debug kqemu.ko
janney:/usr/ports/emulators/kqemu-kmod# make install
===>  Installing for kqemu-kmod-0.7.2_1
===>   Generating temporary packing list
===>  Checking if emulators/kqemu-kmod already installed
install -o root -g wheel -m 555   kqemu.ko /boot/kernel
kldxref /boot/kernel
if mount |/usr/bin/grep ^devfs >/dev/null ; then  : ;  else  if [ ! -e /dev/kqemu ]; then mknod 
/dev/kqemu c 250 0 ;  fi ;  /bin/chmod 666 /dev/kqemu ;  fi
===>   Registering installation for kqemu-kmod-0.7.2_1

Next I had to enable network connectivity. The host OS had xl0 as a live interface. QEMU would use the tap0 interface.

# cat /dev/null > /dev/tap0
# ifconfig tap0 create

# kldload bridge.ko
# sysctl net.link.ether.bridge_cfg=xl0,tap0
net.link.ether.bridge_cfg:  -> xl0,tap0
# sysctl net.link.ether.bridge.enable=1
net.link.ether.bridge.enable: 0 -> 1

I needed to create this small script to enable networking as well.

$ cat /etc/qemu-ifup 
#!/bin/sh
ifconfig ${1} 0.0.0.0 

Now I needed an image to run. I decided to use the NetBSD 2.0.2 x86 image, since it wasn't too large and I figured I would be familiar enough with NetBSD once it was running.

After downloading and extracting the image, I was ready to try running it.

# qemu -net nic -net tap netbsd_2.0.2.img

Here is the initial boot screen.



Now the system is booted.



The root login for this system image is "piripicchio" (it's Italian). Once logged in, I configured the ne2 interface with an IP address on the same segment as the host OS.



Next I added a default route, a nameserver, and a normal user account with a password.

# route add default 192.168.2.1
# echo "nameserver 192.168.2.1" > /etc/resolv.conf
# useradd -m richard
# passwd richard

Now I wanted to enable sshd. After trying to just start the daemon, I realized I needed to generate at least a DSA key as shown below.



With sshd listening, I could log in remotely.

$ ssh 192.168.2.77
The authenticity of host '192.168.2.77 (192.168.2.77)' can't be established.
DSA key fingerprint is 5d:7f:a2:08:b0:3f:f7:e2:45:35:79:60:45:50:09:5d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.2.77' (DSA) to the list of known hosts.
richard@192.168.2.77's password: 
NetBSD 2.0.2 (GENERIC) #0: Wed Mar 23 08:53:42 UTC 2005

Welcome to NetBSD!

Erase is backspace.                                                     
If one studies too zealously, one easily loses his pants.
                -- A. Einstein.
: {1} uname -a
NetBSD  2.0.2 NetBSD 2.0.2 (GENERIC) #0: Wed Mar 23 08:53:42 UTC 2005  
jmc@faith.netbsd.org:/home/builds/ab/netbsd-2-0-2-RELEASE/i386/200503220140Z
-obj/home/builds/ab/netbsd-2-0-2-RELEASE/src/sys/arch/i386/compile/GENERIC i386
: {2} df -h
Filesystem    Size     Used     Avail Capacity  Mounted on
/dev/wd0a     846M     291M      513M    36%    /
kernfs        1.0K     1.0K        0B   100%    /kern

I think that is pretty cool for a free virtual machine. As I learn more about QEMU I will share it here.

I found these three posts to be helpful when getting QEMU working.

Read More