Network Monitoring Platforms on VMware Workstation

Several of you have asked about my experiences using FreeBSD sensors inside VMware Workstation. I use VMs in my Network Security Operations class. I especially use VMs on the final day of training, when each team in the class gets access to a VM attack host, a VM target, a VM sensor, and a VM to be monitored defensively. As currently configured, each host has at least one NIC bridged to the network. The sensor VMs have a second interface with no IP also bridged to the network. When any VM takes action against another, the sensors see it. This scenario does not describe how a VM sensor might watch traffic from a tap, however.

I decided to document how to use VMware to create a sensor that sniffs traffic from a tap. I outline two scenarios. The first uses a port aggregator tap with a single interface out to a sensor. The second uses a traditional tap with two interfaces out to a sensor. The VMware Workstation host OS in this study is Windows Server 2003 Enterprise x64 Edition Service Pack 1 on a Shuttle SB81P with a Broadcom Gigabit NIC and a quad port 10/100 Adaptec PCI NIC. I should mention at this point that this scenario is strictly for use in the classroom. I would never deploy an operational sensor inside a VM on a Windows server platform. I might consider running a sensor in a VM on a Linux server platform. Windows is not built for sniffing duties. Even with the DHCP service disabled, I still cannot tell the Windows interfaces to be configured without an IP address. If anyone has comments on this, please share.

The first step I take is to identify the interface I wish to use for management and the interfaces I wish to use for sniffing. A look at the Network Connections for this system shows the following interfaces are available:



I am using one of the Adaptec interfaces as a host management interface. The Broadcom Gigabit NIC is plugged into the single output from a port aggregator tap. Two other Adaptec interfaces are plugged into the two outputs of a traditional tap. The remaining Adaptec interface is not connected to anything.

Three of the NICs are in the process of "Acquiring network addresses" even with DHCP disabled on the server. Overall this output somewhat confusing, especially if you want to match up interfaces to physical NIC ports. Here is output from ipconfig /all:



Windows is calling the management interface Ethernet adapter Local Area Connection 3. You can see it has the highest of the four Adaptec MAC addresses -- 00-00-D1-EC-F5. I do not know why Windows decided to call it LAC 3. LAC 2 is disconnected. LAC (which doesn't have a number at all -- it's simply Ethernet adapter Local Area Connection) is the Broadcom Gigabit NIC connected to the port aggregator tap. LACs 3 and 4 are connected to the two outputs of the traditional tap.

Notice the LAC does not correspond to the name of the interface shown in the screen shots! For example, LAC 3 is called Ethernet Adapter #4. (Why again did I choose to demonstrate this on Windows?)

With our NICs identified, we can match them up to VMware interfaces. Here is the summary page for the VMware Virtual Network Editor.



This screen is a little cramped, so take a look at the next screen shot showing the Host Virtual Network Mapping.



What I'm doing here is specifically assigning VMnets to individual physical interfaces. This will allow me to assign these VMnets to virtual interfaces on each VM, which I do next. Before starting that process, here is the auto interface bridging selection tab in VMware Workstation:



This shows that three of my adapters are specifically selected to not be automatically bridged.

Now let's look at the host configuration for the VM sensor. The box has two interfaces. The first is automatically bridged. The second has a custom setup.



For the first interface, lnc0, it will use the automatic bridge settings to connect to an automatically chosen adapter. This will be LAC3.



The second interface has a custom setting. Here it will listen to the Broadcom Gigabit interface plugged into the port aggregator tap.



Once I boot the sensor VM, I can SSH to its management interface (lnc0) and see ifconfig output:

# ifconfig -a
lnc0: flags=108843 mtu 1500
        inet 192.168.2.91 netmask 0xffffff00 broadcast 192.168.2.255
        inet6 fe80::20c:29ff:fe5f:51ea%lnc0 prefixlen 64 scopeid 0x1
        ether 00:0c:29:5f:51:ea
lnc1: flags=108802 mtu 1500
        ether 00:0c:29:5f:51:f4
plip0: flags=108810 mtu 1500
lo0: flags=8049 mtu 16384
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4

Once I bring up the lnc1 interface, it sees ICMP traffic as I desire:

# ifconfig lnc1 up -arp
# tcpdump -c 4 -n -i lnc1 -s 1515 icmp
tcpdump: WARNING: lnc1: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lnc1, link-type EN10MB (Ethernet), capture size 1515 bytes
23:21:06.853616 IP 69.243.40.166 > 216.239.37.99: icmp 40: echo request seq 2304
23:21:06.925324 IP 216.239.37.99 > 69.243.40.166: icmp 40: echo reply seq 2304
23:21:07.829845 IP 69.243.40.166 > 216.239.37.99: icmp 40: echo request seq 2560
23:21:07.874166 IP 216.239.37.99 > 69.243.40.166: icmp 40: echo reply seq 2560
4 packets captured
259 packets received by filter
0 packets dropped by kernel

No problem. Now let's see how we can handle combining dual outputs from the traditional tap.

The first issue is dealing with the limitation of having only three virtual NICs in any VM. To address this, we will redeploy lnc1 to watch one of the outputs from the traditional tap, and create lnc2 to watch the other.

With this setup, Ethernet 2 is watching VMnet 3 and Ethernet 3 is watching VMnet 4.



With the interfaces created and the sensor booted, I bond them with the following script:

#!/bin/sh
kldload ng_ether
ifconfig lnc1 promisc -arp up
ifconfig lnc2 promisc -arp up

ngctl mkpeer . eiface hook ether
ngctl mkpeer ngeth0: one2many lower one
ngctl connect lnc1: ngeth0:lower lower many0
ngctl connect lnc2: ngeth0:lower lower many1

ifconfig ngeth0 -arp up

When done, I have ngeth0 ready and I can sniff with it:

# tcpdump -c 4 -n -i ngeth0 -s 1515 icmp
tcpdump: WARNING: ngeth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ngeth0, link-type EN10MB (Ethernet), capture size 1515 bytes
23:29:49.524910 IP 69.243.40.166 > 216.239.37.99: icmp 40: echo request seq 3328
23:29:49.567338 IP 216.239.37.99 > 69.243.40.166: icmp 40: echo reply seq 3328
23:29:50.510332 IP 69.243.40.166 > 216.239.37.99: icmp 40: echo request seq 3584
23:29:50.555589 IP 216.239.37.99 > 69.243.40.166: icmp 40: echo reply seq 3584
4 packets captured
58 packets received by filter
0 packets dropped by kernel

This testing shows you can get a sensor running in a VM to see the same sorts of traffic a physical sensor might see.

Incidentally, the folks at VMware notice I use their products, given their October 2005 post to the VMTN Blog.

I would be interested in hearing if you run sensors inside VMs, for either research or production purposes.