Answering PHK's "Why Bother?" with FreeBSD Question

In the October issue of Daemon News, Poul-Henning Kamp asks "Why Bother?" He wants to know why people use FreeBSD when Linux gets most of the attention from users and vendors. He also wants to know why developers should continue to work on FreeBSD.

I will tailor my response for FreeBSD, as that is the BSD with which I am most familiar. Some of my arguments will apply to other variants. Some of my reasons even apply to other open source operating systems, like Linux. Few will apply to closed operating systems, least of which Windows.

1. FreeBSD is open source, with a business-friendly license. Being an open source, BSD-licensed operating system means I am free to modify the OS as I see fit and can continue to support and evolve any part of it, should the official developers decide to abandon any aspect of the project. I can use FreeBSD in commercial projects as long as I retain the copyright notice and disclaimer with the product.

2. FreeBSD is an integrated, complete product. FreeBSD isn't just a kernel supplemented by software. The developers provide easy (albeit time-consuming on slower machines) means to keep the system up-to-date and functioning properly. (Note: I will publish an article shortly that takes an in-depth look at keeping FreeBSD up-to-date.)

3. All FreeBSD source code is available via CVS. This feature allows users to quickly examine any aspect of the OS to determine if it will suit their needs. For example, I can look in a file like src/sys/dev/aac/aac_pci.c to see if it might support new hardware I'm considering purchasing. I can then checkout the complete CURRENT source or download a CURRENT .iso snapshot (via snapshots.se.freebsd.org or snapshots.jp.freebsd.org). This is perhaps the single greatest measurable advantage compared to closed-source software.

4. FreeBSD developers are accesible and analyze problems in context. I am constantly surprised to find so many developers posting answers to freebsd-questions and interacting helpfully with newsgroups like freebsd-stable and others. When a user posts a thoughtful question which demonstrates he's done some homework, he very frequently gets multiple responses that treat his problem within context of the entire OS. FreeBSD developers work within an operating system, not in isolation.

5. The FreeBSD ports tree offers over 12,000 applications, tailored for FreeBSD. When running FreeBSD, there's almost never a need to download and compile a tar.gz archive from a third party site. Instead, visit FreshPorts.org, see what's available, and then install either a precompiled package or build from source code within the ports system. When you see a program in the ports tree, you have some level of confidence that it will work on your system. When installed using the ports tree or as a precompiled package, you can now manage, track, and upgrade that application. (Note: I am planning a second article on this aspect of FreeBSD as well.)

6. FreeBSD's security features and track record are excellent. I can deploy a FreeBSD-based security appliance, with no ports listening other than OpenSSH, in less than 20 minutes (depending on CD-ROM speed). This system can defend itself without a network- or host-based firewall. FreeBSD has had an order of magnitude lower number of remote root vulnerabilities compared to Windows 2000, for example. Tools like FreeBSD-update, Portsnap, Portaudit and Portupgrade make keeping the system up-to-date easy. The VuXML project tracks issues not only in the OS but also the ports collection.

7. FreeBSD (as well as OpenBSD and NetBSD) Continue to Innovate. This is remarkable, despite the much smaller development community, user base, and lack of large commercial sponsorship. FreeBSD's SMP, network performance, TrustedBSD, and the ndis Windows driver wrapper are impressive. The OpenBSD project has brought us OpenSSH, Pf, OpenBGPD, Pfsync and CARP, and OpenNTPD. NetBSD introduced IPv6 and plenty of clean code that ends up in other projects. This is not to say that Linux or other UNIX derivatives don't innovate. I just amazed that a much smaller community has created these leading network services within a short period of time. I only expect better features out of FreeBSD's 5.x tree now that 5.3 has attained the STABLE mark.

There are seven reasons why I "bother" with FreeBSD. I would like to hear your comments, either via email or through postings to freebsd-advocacy or advocacy.daemonnews.org.

Read More

Thoughts on the United States Air Force Computing Plans

As a former intelligence officer and computer network defender I was asked my thoughts on the US Air Force's new computing deal with Microsoft. In short, Microsoft will provide core server software, maintenance and upgrade support, and Dell will supply more than 525,000 Microsoft desktop Windows and Office software licenses to the Air Force.

From a business perspective, this is an important deal for Microsoft. For all of their seeming independence, the services tend to watch each other closely to see what technological advances are being considered or pursued. When the Navy began work on its Navy Marine Corp Intranet (NMCI), Air Force leaders scrambled to "catch up" to match the "progress" the Navy was assumed to be making. (NMCI has since produced mixed results for the Navy and financial woes for EDS, prime NMCI contractor.)

I have first-hand knowledge of the Air Force's response to NMCI. In the fall of 2000, the AFCERT sent me to Washington, DC to participate in the "redesign" of the Air Force enterprise network. Over a three week period we were expected to create plans to revamp the whole Air Force communications and security architecture. The new network was supposed to be running by June 2001. Obviously this did not happen, although it was the launch of the "One Air Force, One Network" campaign. The entire November 2000 (.pdf) issue of Intercom magazine was devoted to the project, while shorter articles explained the goals in brief.

The core of the project involved several key ideas:

- Server consolidation, particularly email
- Network access consolidation, with bases having links through their major commands
- An Air Force portal ("my.AF")

The Air Force is still following their "One Air Force, One Network" plans. You can see the Air Force CIO's September 2004 presentation (.ppt) reiterates these ideas, several of which I believe are valid. For example, it may not be necessary for each of the 100+ Air Force bases and operating locations to maintain their own connections to the Internet. It is certainly not necessary for each base to maintain its own Web, DNS, and mail servers, as these pieces of public infrastructure are one of the main ways external intruders can compromise Air Force assets.

Many reactions (e.g., Slashdot.org) to the new Microsoft/Air Force deal center on the standardization on Windows throughout the Air Force. Here I am disappointed. In the mid-1990s I used Solaris where it mattered, on servers and on intelligence projects. I remember using Windows for Workgroups 3.11 for office applications. Within the last five years Microsoft has replaced a lot of this infrastructure and handles core email and Web duties in many locations.

Regarding the implementation of a homogeneous infrastructure, I have a mixed opinion. Standardization can be a force for good when it eases system configuration, deployment, and patching. The question is, do the benefits of standardization outweigh the potential for thorough, widespread exploitation by a worm targeting standardized deployments? For years I've advocated deploying redundant architecture, running alternate operating systems, to mitigate this fate. If you want your Web site running IIS on Windows, have an Apache on UNIX backup ready.

I would have been pleased to see the Air Force adopt a thin client strategy for its desktop users, preferably based on Solaris or even a Linux distribution. I think the Air Force was too "corporate" to make such a radical step as to abandon its Microsoft desktop infrastructure. A TechWorld.com story noted that AF CIO "Gilligan acknowledged that in grappling with the patch-update issue, the Air Force had considered transitioning to open-source software but determined the transition costs would simply be too high." Ironically, that same article mentioned this:

"The Air Force endures about one network-based attack per week that successfully exploits new vulnerabilities, Gilligan said. 'There's some disruption and loss of capability,' he pointed out, noting that Air Force bases all over the world support the operations of the war in Afghanistan and Iraq. 'We're spending more money patching and fixing than buying software,' said Gilligan. It's not unusual for patching of vulnerabilities to take months to complete, he said."

So instead of taking a serious look at the root cause of its patching and exploitation costs (both financial and in mission impact), the Air Force sought a better deal from the vendor producing flawed software. This is sad. TechWorld's Ellen Messmer wrote "The US Air Force has had enough of Microsoft's security problems. But rather than switch to an alternative, it has struck a deal with CEO Steve Ballmer for a specially configured version of Windows." Will Microsoft sell this "special version" elsewhere, and if so, is the Air Force the guinea pig paying to develop this version?

Had the Air Force decided to break away from Microsoft, the other services would have definitely taken notice. In fact, corporate America would have taken notice. I hope vendors like Sun, IBM, and Red Hat step up their efforts to infiltrate the government space and introduce more secure products where it matters.

On a related note, the Air Force continues to lead the information warfare domain with its Advanced Course in Engineering (ACE) Cyber Security Boot Camp.

Read More

Five Ways Sguil is Different

On Wednesday I mentioned that a chapter from my book appeared in a new form at Informit.com. A snort-users reader asked how Sguil differed from ACID and BASE. In short, there are five reasons:

1. Sguil is a real-time interface to Snort alerts (and more).
2. Sguil is a Snort alert management system with integrated analyst accountability features.
3. Sguil offers growing alert handling capabilities.
4. Sguil is built to minimize "window management," "form management," and other non-analytical tasks.
5. Most importantly, Sguil is not limited to investigating events using Snort alert data alone.

To read explanations of each point, please see my response to the snort-users mailing list. You'll also find in that message three "features" that are not present in Sguil.

I should have mentioned that Sguil is the single tool most likely to provide analysts with the information they need to make a decision. With Sguil, a Snort alert is not the end of the investigation -- it's only the beginning.

Read More

Metanetworks Claims "first wire-speed 10G Ethernet IDS/IPS product in the world"

While browsing the tcpdump-workers mailing list I came across a post describing the The Meta Traffic Processor PCI Card. This device "is a standard 32-bit/33MHz, PCI half-card with two copper Ethernet ports. The MTP appears to the host's operating system as a standard network interface card capable of enforcing from 600 to 1500 stateful policies to capture and/or block specific packets. The policies can be directly derived from public domain signatures or they can be completely user defined." These "policies" can be Snort rules or one day BPF filters.

The product appears to be the result of work done for a NSF grant. Metanetworks founder Livio Ricciulli described his project to NANOG in May 04.

I had heard from a .mil type that DoD was working with a vendor to run Snort-like applications in hardware. Perhaps this is what he meant?

I intend to contact Livio to see if his card works with FreeBSD.

Read More

Several recent Blog entries described ways to keep FreeBSD applications up-to-date. Based on my use of these tools, this is how I chose to update one of my servers this morning. First I updated the ports tree, INDEX-5, and INDEX.db:

cd /usr/ports
portsnap fetch
portsnap update
make fetchindex
portsdb -u

Next I checked to see which applications needed to be updated:

janney:/usr/ports# portversion -v -l "<"
bash-3.0.15                 <  needs updating (port has 3.0.16_1)
freebsd-update-1.6          <  needs updating (port has 1.6_1)
sudo-1.6.8.1                <  needs updating (port has 1.6.8.4)

I prefer to update applications by using precompiled packages provided by the FreeBSD team. Unfortunately, the packages-5-stable FTP site hasn't been updated since 15 Nov. I decided to press ahead and update these three packages on my own by building the upgrades from source.

I used portupgrade to (in the order of the switches shown), be verbose, update all packages whose versions are outdated, update packages that depend on the package being updated, update packages that the package being update considers dependencies, and create packages in /usr/ports/packages/All during the process:

janney:/usr/ports# portupgrade -varRp

We'll use the packages built during this process on server janney to update some of the packages on laptop orr.

When done I had three new packages in /usr/ports/packages/All:

janney:/usr/ports/packages/All# ls -alt
total 135276
drwxr-xr-x  21 root  wheel       512 Nov 25 11:03 ..
-rw-r--r--   1 root  wheel    545101 Nov 25 11:03 bash-3.0.16_1.tbz
drwxr-xr-x   2 root  wheel      1536 Nov 25 11:03 .
-rw-r--r--   1 root  wheel     10097 Nov 25 10:59 portsnap-0.2_1.tbz
-rw-r--r--   1 root  wheel     29219 Nov 25 10:58 freebsd-update-1.6_1.tbz
-rw-r--r--   1 root  wheel     99157 Nov 25 10:58 sudo-1.6.8.4.tbz

I expected bash, freebsd-update, and sudo to be there. These three were the packages identified by portversion as being out-of-date. A new portsnap package was created as part of the upgrade process for freebsd-update, since portsnap depends upon freebsd-update to function.

Now that I had these new packages, I turned to updating laptop orr. I followed the five steps I first did for janney, and a 'portversion -v -l "<"' to see what needed updating. Laptop orr has a lot more packages installed compared to janney. If I want to avoid updating packages on orr via building from source through the ports tree, I need to obtain updated packages either from the FreeBSD project or from a system that's built the same packages.

Standard FreeBSD systems do not seem to have a means to build packages without installing them. I believe OpenBSD has this capabilitity. Since I prefer to not maintain a "package builder" with ever application I need, I take a dual-pronged approach. First, I wait until the FreeBSD project provides updated packages. Second, for critical applications that tend to be installed on many systems, I create my own packages. This second approach is what this Blog entry is about.

Based on my update on server janney, I know I can provide updated bash, sudo, and freebsd-update packages for laptop orr. To do that I NFS mount /usr/ports/packages/All on janney to orr, then run portupgrade:

orr:/usr/ports# mount -t nfs janney:/usr/ports/packages/All /usr/ports/packages/All
orr:/usr/ports# portupgrade -varRPP
--->  Session started at: Thu, 25 Nov 2004 11:26:02 -0500
** No need to upgrade 'tcpflow-0.21' (>= tcpflow-0.21). (specify -f to force)
--->  Checking for the latest package of 'security/sudo'
--->  Found a package of 'security/sudo': sudo-1.6.8.4.tbz (sudo-1.6.8.4)
--->  Upgrade of security/sudo started at: Thu, 25 Nov 2004 11:26:09 -0500
--->  Upgrading 'sudo-1.6.8.1' to 'sudo-1.6.8.4' (security/sudo) using a package
--->  Updating dependency info
--->  Uninstallation of sudo-1.6.8.1 started at: Thu, 25 Nov 2004 11:26:11 -0500
--->  Fixing up dependencies before creating a package
--->  Backing up the old version
--->  Uninstalling the old version
--->  Deinstalling 'sudo-1.6.8.1'
[Updating the pkgdb  in /var/db/pkg ... - 167 packages found (-1 +0) (...) done]
--->  Uninstallation of sudo-1.6.8.1 ended at: Thu, 25 Nov 2004 11:26:15 -0500 (consumed 00:00:03)
pkg_info: can't find package 'sudo-1.6.8.4.tbz' installed or in a file!
--->  Installation of sudo-1.6.8.4 started at: Thu, 25 Nov 2004 11:26:15 -0500
--->  Installing the new version via the package
Will not overwrite existing /usr/local/etc/sudoers file.
--->  Removing temporary backup files
--->  Installation of sudo-1.6.8.4 ended at: Thu, 25 Nov 2004 11:26:18 -0500 (consumed 00:00:03)
--->  Cleaning out obsolete shared libraries
[Updating the pkgdb  in /var/db/pkg ... - 168 packages found (-0 +1) . done]
--->  Upgrade of security/sudo ended at: Thu, 25 Nov 2004 11:26:20 -0500 (consumed 00:00:10)
...truncated...
orr:/usr/ports# umount /usr/ports/packages/All

When done, bash, freebsd-update, and sudo were updated.

Read More

FreeBSD Ports Tree Breaks 12,000 Ports

Last night the FreeBSD ports tree broke the 12,000 mark. The tree has added about 2000 ports per year for the past four years. This graph shows the number of ports added per year since 1995.

I commend FreshPorts for providing such an excellent interface to the tree, and for keeping up with the growth in the number of applications available. FreshPorts recently integrated VuXML data, allowing users to visually see what port versions have a security issue. More details and information on other upgraded FreshPorts features are available here.

Read More

Using Portindex to Generate INDEX-5

Now that we've seen how to keep the ports tree up-to-date using tools like Portsnap, and seen how to generate an INDEX-5 file with 'make index', I'd like to offer an alternative INDEX-5 generation mechanism. Matthew Seaman's Portindex is a Perl tool replacing a similar application of the same name. The old version was pulled from the ports tree when the developer started acting strangely.

Here's the problem Portindex solves. If you use CVSup to update your ports tree, you will not have an INDEX-5 file. The INDEX-5 file included in CVS was reportedly always out-of-date, so you had to issue a command like 'portsdb -U' or 'make index' to create the INDEX-5 file. This was very time-consuming.

Portindex is a much faster alternative, once you've set up the system. Portindex is available in the ports tree as sysutils/p5-FreeBSD-Portindex , so installation is easy (as long as your ports tree is updated!) My installation is based on Matthew's excellent documentation at his site and in the Portindex man pages.

Once installed, you need to ininitalize the cache Portindex uses:

janney:/root# cache-init
Processing make describe output for path "/usr/ports": .........[1000].........F
reeBSD::Portindex::Tree::_scan_makefiles(): Can't open Makefile in /usr/ports/de
vel/jude_community -- No such file or directory at /usr/local/bin/cache-init lin
e 100
[2000].........[3000].........[4000].........[5000].........[6000].........[7000
].........[8000].........[9000].......FreeBSD::Portindex::Tree::_scan_makefiles(
): Can't open Makefile in /usr/ports/textproc/csv2txt -- No such file or directo
ry at /usr/local/bin/cache-init line 100
..[10000].........[11000].........<11993>

This took a long time -- hours -- on my dual PIII system. Once done, I had these new files:

janney:/var/db/portindex$ ls -al
total 9966
drwxrwxr-x  2 root  operator      512 Nov 24 18:25 .
drwxr-xr-x  9 root  wheel         512 Nov 24 18:22 ..
-rw-r--r--  1 root  operator     8192 Nov 24 21:10 __db.001
-rw-r--r--  1 root  operator   270336 Nov 24 21:10 __db.002
-rw-r--r--  1 root  operator   344064 Nov 24 21:10 __db.003
-rw-r-----  1 root  operator  9617408 Nov 24 21:10 portindex-cache.db
-rw-r--r--  1 root  operator       25 Nov 24 18:25 portindex-timestamp

Next I had to update the ports tree and give Portindex the output in a form it could use:

janney:/root# script /tmp/cvsup.out cvsup -g -L2 -h cvsup2.freebsd.org
 /usr/local/etc/ports-supfile 
Script started, output file is /tmp/cvsup.out
Parsing supfile "/usr/local/etc/ports-supfile"
Connecting to cvsup2.freebsd.org
Connected to cvsup2.freebsd.org
Server software version: SNAP_16_1h
Negotiating file attribute support
Exchanging collection information
Establishing multiplexed-mode data connection
Running
Updating collection ports-all/cvs
 Delete ports/INDEX-5
 Edit ports/Tools/portbuild/scripts/buildscript
  Add delta 1.17 2004.11.25.00.02.38 kris
...edited...
 Checkout ports/x11-wm/ratpoison/files/ratpoison.desktop
 Edit ports/x11-wm/ratpoison/pkg-plist
  Add delta 1.6 2004.11.24.17.32.29 hq
Shutting down connection to server
Finished successfully

Script done, output file is /tmp/cvsup.out

Notice that at the end of this process, INDEX-5 is gone:

janney:/root# ls -al /usr/ports/IND*
-rw-r--r--  1 root  wheel  12250112 Nov 24 08:11 /usr/ports/INDEX.db

We saw earlier that CVS executed ' Delete ports/INDEX-5' when updating /usr/ports.

Now run 'cache-update' against the CVS output:

janney:/root#  cache-update -i /tmp/cvsup.out
cache-update:1: Updating cached data for /usr/ports/audio/daapd
cache-update:2: Updating cached data for /usr/ports/audio/libsidplay2
cache-update:3: Updating cached data for /usr/ports/chinese/MT
cache-update:4: Updating cached data for /usr/ports/chinese/zhcon
cache-update:5: Updating cached data for /usr/ports/comms/efax-gtk
cache-update:6: Updating cached data for /usr/ports/comms/libticables
cache-update:7: Updating cached data for /usr/ports/comms/minicom
cache-update:8: Updating cached data for /usr/ports/databases/myodbc
cache-update:9: Updating cached data for /usr/ports/databases/slony1
cache-update:10: Updating cached data for /usr/ports/deskutils/hot-babe
cache-update:11: Updating cached data for /usr/ports/devel/bison
cache-update:12: Updating cached data for /usr/ports/devel/gettext
...edited...
cache-update:78: Updating cached data for /usr/ports/x11-wm/ratpoison
cache-update:79: Updating cached data for /usr/ports/x11/xextensions

Finally run 'portindex' to create INDEX-5:

janney:/root# portindex -o /usr/ports/INDEX-5
Accumulating dependency information: .........[1000].........[2000].........[300
0].........[4000].........[5000].........[6000].........[7000].........[8000]...
......[9000].........[10000].........[11000].........[12000]<12000>
Writing INDEX file: .........[1000].........[2000].........[3000].........[4000]
.........[5000].........[6000].........[7000].........[8000].........[9000].....
....[10000].........[11000].........[12000]<12000>
janney:/root# ls -al /usr/ports/IND*
-rw-r--r--  1 root  wheel   5959708 Nov 24 21:47 /usr/ports/INDEX-5
-rw-r--r--  1 root  wheel  12250112 Nov 24 08:11 /usr/ports/INDEX.db

I'm not sure if I will continue to use this method. As I just described it, you must use CVSup. Matthew offers 'find-updated' as a way to scan the /usr/ports tree and discover differences. I believe this could be paired with Portsnap to let Portsnap update the tree and Portindex update INDEX-5. That will probably be another Blog entry...

Read More