FreeBSD Quality

The topic of the quality of FreeBSD has recently appeared in several places. Earlier this week SecurityFocus reported on the results of a study by Coverity. From Coverity's 27 June 2005 press release:

Coverity "released software defect and security vulnerability results for FreeBSD 6.0... [and] found 306 software defects in FreeBSD's 1.2 million lines of code, or an average of 0.25 defects per 1,000 lines of code."

That is interesting, considering they did the study well over a month ago, before 6.0 was even in BETA status. Also:

"FreeBSD security is getting better very quickly - over the course of a year, FreeBSD's code size doubled, while the total number of defects went down by 50%."

The SecurityFocus story made this observation:

"Not all the potential flaws found by analysis tools are security holes. For FreeBSD, while 306 problems were flagged by Coverity's software, only 5 issues could be triggered by user input. The software classified another 12 vulnerabilities as buffer overruns, another potentially serious security issue. The FreeBSD project has analyzed the flaws and fixed the issues."

For some commentary on the study, check out this OSNews.com thread.

Over on the freebsd-stable mailing list, Alexey Yakimovich started a Quality of FreeBSD thread by complaining about ATA errors. I thought Robert Watson's reply provided very useful insights into the problems of operating system development and testing.