The BSDs: Sophisticated, Powerful, and (Mostly) Free

September 26, 2002
By: Brett Glass

What is BSD? If you ask a typical computer "expert," he or she is likely to reply (incorrectly!) that it is "an operating system." The correct answer, however, is more complex than that. BSD is -- among other things -- a culture, a philosophy, and a growing collection of software, most (though not all) of which is available for free and with source code.

Here are the origins of BSD and the operating systems it has spawned. BSD stands for "Berkeley Software Distribution," the name first given to the University of California at Berkeley's own toolkit of enhancements for the UNIX operating system. Created by the students and faculty, BSD was not part of UNIX itself, which was created by Bell Labs. Rather, it was a widely distributed package of software enhancements for UNIX -- a supplement that made the operating system, which was originally strictly a research vehicle, useful in the real world.

Over time, however, BSD took on a life of its own. It evolved to include replacements for nearly every part of UNIX -- so much so that only the omission of six computer files prevented it from being a complete operating system by itself. Industrious programmers quickly developed replacements for these six files and made the BSDs into usable operating systems.

The 'BSDs' Defined

Nowadays, the term "The BSDs" refers to the family of operating systems which were derived, to a greater or lesser extent, from BSD. The five best known BSDs are FreeBSD, NetBSD, OpenBSD, BSD/OS, and Darwin (which serves as the foundation for Apple's MacOS X). But virtually all modern operating systems -- from Windows to BeOS to Linux -- rely on crucial BSD code to run. In fact, as explained later in this article, the Internet itself owes its existence to the BSD legacy.

The BSDs are known for the logical consistency of their commands and architectures, as well as for robustness and stability. While other operating systems -- including Microsoft Windows and (more recently) Linux -- have been the darlings of the trade press (and of course end users), none of the BSDs have ever won a popularity contest. Nonetheless, the BSDs have always been the choice of system administrators who cared more about integrity, security, and reliability, than sizzle and flash. The long BSD tradition of cautious development, extensive peer review, and thorough testing makes them some of the most reliable software ever developed. In fact, as far as anyone knows, only one worm has ever been developed that attacked any of the BSDs.

BSDs have always been the choice of system administrators who cared more about integrity, security, and reliability, than sizzle and flash.

Created this year, the worm broke into FreeBSD systems not through a standard feature of the operating system but via a bug in the Apache Web server package -- which isn't installed by default. Using the most recent version of Apache immunizes the machine from this worm.

Of the people, by the people, for the people
Because it was the product of taxpayer-supported research, BSD source code can be (and is!) used in any commercial or non-commercial software product, free of charge. For this reason, virtually every computer operating system we use today --including Microsoft Windows, Microsoft NT/2000/XP, OS/2, Linux, and every commercial version of UNIX -- contains at least some BSD code. The Internet as we know it owes its existence to the TCP/IP "stack", or networking code, of BSD, which was revolutionary in that it incorporated networking directly into the kernel of the operating system. (DARPA, the Defense Advanced Research Project Agency, funded the development.) This prescient (and, in some ways, radical) innovation paved the way for the age of the Internet, while the BSD code itself created a common starting point for the network facilities in all modern operating systems.

The 'Mr. Chips' of Operating Systems

One might well be justified in calling BSD the "Mr. Chips" of operating systems. In the final scene of the classic movie "Goodbye, Mr. Chips", a doctor remarks that it is a shame that the title character -- a masterful schoolteacher now on his death bed -- has never had children. Referring to the many youngsters whose lives he had helped to shape, Mr. Chips replies that he has indeed had children... thousands of them.

The same is true of BSD. Consider its licensing policies versus Linux, for example. When code is licensed under the GNU General Public License or GPL (as is Linux), the license effectively eliminates any financial rewards anyone -- whether an individual or a corporation -- might hope to gain from improving upon it. It does this by compelling an author who uses any part of the code to give up the right to charge a license fee for the finished product.

In contrast, BSD is certified under a truly free license which leaves anyone free to do whatever he or she wishes with the code, including make money by licensing it or use it as part of another work. The minor restrictions imposed by the BSD license apply not to what may be done with the code, but rather what may be done to the developer. (The previous developer may not be held liable for errors, or -- under some versions of the license -- be deprived of credit for his or her work.)

The result: BSD code is everywhere. While some operating systems -- most especially Microsoft's -- have sold more complete copies, BSD has achieved far greater, if more indirect, success by supplying vital components to every modern environment. BSD's licensing scheme also promotes standardization by allowing both commercial and non-commercial products to use the same code, allowing the source code itself to serve as an easily reproducible standard.

The rise of the Internet, for example, was largely due to the fact that all computer and operating system vendors could use the BSD networking code rather than having to reinvent the wheel or develop proprietary protocols. The benefits which the world has reaped from the BSD approach to licensing and technology sharing are immeasurable.

It has been said that UNIX is to computer science what Shakespeare is to English. Just knowledge of Shakespeare's classic works is essential for an English degree, no one should be granted a degree or technical certification in computer science without a working knowledge of UNIX technology and architecture.

The Birth of UNIX and BSD

The UNIX operating system was created at Bell Laboratories -- then the research arm of AT&T -- in the early 1970's. The name was an intentional pun on "Multics", the name of a very sophisticated and intricate operating system developed at MIT. UNIX, which adopted a philosophy commonly known to engineers as KISS ("Keep it simple, stupid"), was said to be "castrated Multics" because of its emphasis on simplicity. Written in the C language (itself developed at Bell Labs), UNIX was designed to run interactively -- rather than accepting and returning output in batches, like most computers of the day -- on relatively simple hardware.

UNIX Goes to College
The first scholarly paper on UNIX was presented by researchers Ken Thompson and Dennis Ritchie at the Symposium on Operating Systems Principles at Purdue University in November 1973. The paper sparked immediate interest on the part of professors and graduate students at UC Berkeley, who installed UNIX Version 4 on a PDP-11/45 in January 1974. The operating system proved popular with students and faculty alike, and before long, many at the University were using it, improving it, and writing software for it.

UNIX soon became the focus of several important academic research projects. INGRES, UC Berkeley's pioneering database software project, shipped its software to run on UNIX. The first general-purpose tools for the creation of language compilers and translators were developed on and for UNIX as well. DARPA funded the development of networking features. (These were developed for ARPANet, the precursor to today's Internet.) A new research organization -- the Computer Science Research Group, or CSRG -- was formed at Berkeley to do this work.

Adding TCP/IP networking to the UNIX kernel was a technological breakthrough that led directly to the "wired" world we know today.

Adding TCP/IP networking to the UNIX kernel was a technological breakthrough that led directly to the "wired" world we know today. Prior to that time, computer communications were an "on again, off again" affair. Programs that allowed computers to send and receive information, such as the UNIX-to-UNIX Copy Program (uucp), weren't part of the operating system kernel itself-- they were loaded and unloaded as needed and did not necessarily use a common set of protocols. Network services weren't available to every program that ran on the computer, and there was no way to log in remotely from across a local area or wide area network. Berkeley's networking code, which was designed to be "broken out" and incorporated into other operating systems as well, changed this situation and set the standard for the future.

Before long, Berkeley's improvements and enhancements to UNIX -- collectively called the "Berkeley Software Distribution" or BSD -- had become indispensable.

Who Owns UNIX?

Relations between developers at AT&T and at Berkeley were always cordial (Ken Thompson did, in fact, spend time as a visiting professor at UC Berkeley during a sabbatical from Bell Labs). However, it wasn't long before the "bean counters" at AT&T became interested in exploiting the valuable intellectual property that UNIX had become. After the breakup of the Bell System, AT&T -- previously barred by the courts from selling computers or software -- recognized that it could make lots of money by licensing UNIX to computer manufacturers. The BSD code, which was both indispensable to UNIX aficionados and very much intertwined with AT&T's code, was the intellectual property of the University. Darpa's research grants meant that if you could pay for the computer tape, you could get the BSD portion, with source code, for free. AT&T wanted a product that was entirely and exclusively its own -- one that it could sell.

The two organizations reached a parting of the ways, a painful separation due to the cultural gulf between academics and corporate. In the mid-to-late 80's AT&T, hoping to maintain market control of the UNIX standard from UC Berkeley, developed its own version of UNIX, called System V, whose commands and programming interfaces were subtly different from (but not necessarily better than) those of "classic" Berkeley UNIX. Customers, particularly the US Government, eventually resisted this divergence, and insisted on buying compatible systems. Thus the POSIX standard, which mandates a minimum level of compatibility, was born.

AT&T also resorted to legal action. When a company called BSDi announced a commercial operating system based on the BSD code, UNIX Systems Laboratories (originally owned by AT&T and then spun off to Novell) sued the University and BSDi for misuse of the "UNIX" trademark, and also for copyright infringement. (The suit stretched from 1992 until 1994, and was ultimately settled when it was revealed that AT&T had likewise breached some of its licensing agreements with the University.) For the duration of the lawsuit, the status of BSD and both the commercial and non-commercial projects based on it was in doubt.

Anticipating that the lawsuit would be tossed out, many people began considering, and developing an inexpensive UNIX-like operating system. These developers also recognized that with just a bit more work, BSD could be turned into a complete UNIX-like operating system -- free and unencumbered by AT&T restrictions.

Bill and Lynne Jolitz were the first to attempt this task, producing an operating system called 386BSD in 1993. The FreeBSD and NetBSD projects carried on these efforts, combining BSD with software from other sources to produce complete UNIX-like operating system distributions. OpenBSD split from NetBSD around 1996 in part due to personality clashes, and in part due to a desire to focus on security issues.

Around the same time, Linux surfaced. Based on the Minix kernel written by computer science professor Andrew Tannenbaum, and unencumbered by the spectre of a lawsuit, Linux began to gain momentum and became the best known freely redistributable UNIX-like operating system.

Is BSD UNIX?

Ironically, while the BSDs are truer to the UNIX pedigree than many operating systems which are now called UNIX, none of them may legally be called UNIX. To bear the UNIX trademark, now owned by a corporation called The Open Group, an operating system must pass a series of expensive tests which prove that it conforms to the "Single UNIX Specification". While large computer manufacturers can afford the testing and certification required to use the brand name, smaller vendors and open source projects -- such as FreeBSD, NetBSD, and OpenBSD -- cannot. Nor can the volunteers who participate in these projects afford the legal firepower to convince a court that "UNIX" is now a generic term. Therefore, even though most computer professionals refer to these distributions as "Berkeley UNIX," the BSDs may not bear the UNIX trademark today.

In short, "the BSDs" is the best term to use, collectively, for the UNIX-like operating systems derived from the BSD code. Since the University of California at Berkeley is no longer producing new releases of BSD (the last was "4.4 BSD Lite," released in June 1995), three of the four open source BSD development projects now define "BSD."

The Four Open Source BSDs

Four BSD-based operating systems are under active development and are available, with source code, to all comers. These include FreeBSD, NetBSD, OpenBSD, and Darwin. Darwin originally served as the foundation for NeXT's NeXTStep, and now underlies MacOS X.

Of these, the first three are licensed under Berkeley's truly free BSD License, which allows completely unfettered use of the code for any purpose. But Darwin, breaking with BSD tradition, is licensed under the "Apple Public Source License", which contains anti-commercialization provisions similar to those of the GPL. The APSL also compels the forfeiture of patent rights on any invention embodied in a program that uses the code. As a result, code from Darwin is not commercially useful except to Apple, which has the sole right to build commercial software from it. Because of its licensing, Darwin is unlikely to aid in the creation of industry standards or of commercial products from companies other than Apple.

FreeBSD: Quality, Plus Lots of Bells and Whistles

Of the truly free BSDs, the one with the most bells and whistles is FreeBSD. Currently at version 4.6.2, FreeBSD has the largest development team, the largest user base, the largest number of ported applications, and the largest collection of active e-mail lists. It also has the best documentation-- besides the usual UNIX manual pages, it comes with The FreeBSD Handbook, an extensive "how-to" guide written by the operating system's developers and users. While you can download free versions of the OS and documentation, you can purchase packaged versions at the FreeBSD Mall.

Unlike most other operating systems (including most distributions of Linux), FreeBSD is extremely easy to install directly via an Internet connection. No CD-ROM is required, though one must download two 1.4 MB floppy disk image files and use them to create bootstrap floppies. One can also download a burnable, bootable CD-ROM image directly from the FreeBSD FTP server. FreeBSD CD-ROMs and DVD-ROMs are also available from several manufacturers listed on the FreeBSD Web site.

FreeBSD has the largest development team, the largest user base, the largest number of ported applications, and the largest collection of active e-mail lists.

FreeBSD's text-based, menu driven installation procedure isn't the most flashy in the industry, but is less likely to fail on unfamiliar hardware than the fancier GUI installers used by some other operating systems. Unfortunately, installation can at times be confusing to novices who don't understand certain UNIX concepts and idioms. (For example, UNIX disks are first divided into "slices" -- what MS-DOS calls "partitions." File systems, called "partitions", are then laid out within the slices.) The online Handbook helps with the process, though, and one can type "A" at key points in the installation to obtain "automatic" settings that are likely to work on a particular machine. The default installation enables BSD's SoftUpdates feature, which speeds disk access while protecting against corruption due to power failures. Software-based RAID support is built in. An installation from CD-ROM can take 10 minutes or less, while an online install can vary from 30 minutes to a few hours depending upon the speed of the connection. Once FreeBSD is installed, the user is encouraged to add useful third-party software, such as Apache, from CD-ROM or via the Internet.

Despite being a bit larger than OpenBSD or NetBSD, FreeBSD is a top performer, often besting Linux in Web service and database benchmarks. Newcomers to the BSDs, or UNIX-like operating systems in general, will find it easy to start with FreeBSD and then move on to environments with less "hand-holding" (e.g. OpenBSD or NetBSD) if they need the more specialized features of those systems.

FreeBSD currently runs on Intel-compatible 32-bit processors (including the AMD Athlon) and on the DEC Alpha processor (now out of production), and is being ported to Itanium, AMD's x86-64 (Opteron/Sledgehammer), PowerPC, and Sparc64. While porting the operating system helps to flush out subtle bugs, portability is not FreeBSD's specialty-- it's primarily of interest to owners of Intel-compatible hardware. 16 MB of RAM and 500 MB of disk space are sufficient for installation on an x86-compatible platform, but more of both is better.

OpenBSD: Rock Solid Security, Fanatical Attention to Detail

OpenBSD is the perfectionist's version of BSD. Almost Spartan compared to the others, it installs with many features intentionally disabled to avoid potential security holes. Its highly focused development team is constantly tweaking, critiquing, and auditing every line of the code, and their commitment to excellence shows in the operating system's track record-- only one security hole that would allow an intruder to break in from the Internet has been discovered in the past 6 years. While the other BSDs have begun to catch up with OpenBSD's security practices, none of them can claim anywhere near as impressive a track record. Nor can Linux, all of whose distributions have experienced serious security holes and have been subject to attack by worms.

OpenBSD is the perfectionist's version of BSD. Almost Spartan compared to the others, it installs with many features intentionally disabled to avoid potential security holes.

OpenBSD, like FreeBSD, can be installed over the Internet, but the process requires a bit more technical knowledge. The project urges users to buy CD-ROMs instead-- the proceeds from sales of the disks support the project and some of the developers. The only disadvantage of this is that CDs stocked at stores may be out of date by the time you receive them. (The latest release is OpenBSD 3.1; important patches for this version are available on the OpenBSD Web site.)

OpenBSD also spawned the OpenSSH project, whose purpose is to develop software implementing the SSH (Secure Shell) protocol. OpenSSH, like OpenBSD, is BSD-licensed, which means that anyone can use the code for any purpose.

OpenBSD requires more technical knowledge and skill to use effectively than FreeBSD, and therefore is not the best choice for beginners. However, because of its reputation for security, experienced UNIX administrators frequently use it to build firewall routers and secure servers. And, like NetBSD and OpenBSD, it incorporates SoftUpdates for faster disk access (though this feature is not turned on by default).

Unlike FreeBSD, OpenBSD is available and actively developed for a wide range of hardware platforms, from x86 to Mac to Sun. It does not, however, cover as many platforms as does NetBSD (described below). 20 MB of RAM and about 300 MB of disk are required for smooth operation on an x86-compatible machine, but as always, more is better.

NetBSD: Now playing everywhere

NetBSD is the portability champ of the BSDs, running on everything from generic x86 boxes to exotic hardware such as the BeBox and the Sega Dreamcast. (The impressive list of platforms, shown in small print in the right margin of the project's home page, is too long to fit on one screen.) More similar to OpenBSD than FreeBSD (OpenBSD was a spinoff of the NetBSD project), it is likewise simple and relatively small. And while it doesn't have OpenBSD's intense focus on security, it's no slouch in that department-- the NetBSD project has reported several important concerns that the OpenBSD team did not catch. Keeping the operating system portable means keeping the code clean, and so the other BSDs often borrow code from NetBSD. (FreeBSD, for example, got its code to handle PCMCIA cards from NetBSD.) At this writing, the latest version of NetBSD is 1.5.2, and it requires some patches after installation to bring it up to date and fix the latest bugs. Editor's Note: Since the story was turned in, the latest version of NetBSD is now 1.6. We thank reader John Schieferle Uhlenbrock for this observation.

Because it is portable and is not subject to the anti-commercialization h constraints imposed by the GPL, NetBSD is cropping up in an increasing number of embedded systems. A New York company called Wasabi Systems, founded by one of the NetBSD developers, specializes in porting the operating system to new microprocessors and systems. (The company also sells NetBSD CD-ROMs for users who don't wish to do an FTP install.) Due to NetBSD's ease of portability, Wasabi was able to get up and running on AMD's x86-64 platform several months sooner than other teams could port Linux.

NetBSD is the portability champ of the BSDs, running on everything from generic x86 boxes to exotic hardware such as the BeBox and the Sega Dreamcast.

In most areas other than portability, NetBSD takes a middle road between OpenBSD and FreeBSD. It has more applications in its "ports" collection than OpenBSD, but fewer than FreeBSD. Similarly, its development team is larger than OpenBSD's and smaller than FreeBSD's.

Many users, especially those who like to tinker with the code, run NetBSD on x86 machines. But if you're not a developer looking for an embedded operating system, you'll most likely be interested in NetBSD if you're seeking an up-to-date OS for an unusual or unsupported computer. Users of old Motorola-based Macintoshes and Sun workstations are especially fond of NetBSD, because this lean operating system performs well on these older platforms. If you're installing NetBSD on an x86-based system, make sure you have at least 16 MB of RAM and at least 300 MB of disk space.

BSD vs. Linux

What's the difference between the BSDs and Linux? Other than the licensing issues mentioned earlier, the main differences are architecture and philosophy.

Linux, by itself, is not a complete operating system -- it's the "kernel" of the operating system -- the part that controls access to resources such as CPU time and peripherals. Packaged versions of Linux, called "distributions," combine Linux with other code -- of which, ironically, many portions are copied directly from BSD. Each vendor (for example, Caldera, Red Hat, or SuSE) sells its own slightly different brew, consisting of Linux plus other software.

The BSDs, by contrast, are complete operating systems that include not only the kernel but the programs, utilities, and configuration files that work with it. Because each of the BSDs is developed as an integrated whole, the parts are finely tuned to work together. They thus have a solidity and consistency that Linux distributions often lack. There is also less duplication of effort in the creation of documentation. Each BSD works the same no matter where you got the disk, so you won't find (as often happens with Linux) that your system is different than what's described in a book or magazine article because you bought it from a different vendor.

The BSD philosophy also differs from that of Linux. Proponents of Linux tend to take a "revolutionary" stance, seeing their work as a war to compete with, and destroy, Microsoft and other commercial software vendors. But the BSDs are content to coexist with commercial software, and in fact are happy to allow commercial software to use what they create.

The BSDs are focused not on winning a political or economic battle, but rather on technical excellence. Changes to the code tend to be more carefully considered, refined, and presented at technical conferences before they become part of what's released to the world at large. For this reason, administrators and users who need to keep systems up and running 24x7 often prefer the BSDs to the more "edgy" and experimental Linux. All of the BSDs are capable of running applications created for Linux (and, in some cases, run them faster), so there's no downside to using one of the BSDs rather than Linux.

The Future of BSD

The three truly free BSDs once had strong rivalries, with the developers in each group bickering -- online and off -- with the others. But of late, the groups -- which have, despite their differences, continued to use one another's code -- have seen an increasing convergence. NetBSD, OpenBSD, and FreeBSD will never merge into one operating system (nor would this be desirable-- a software monoculture would magnify the effect of any newly discovered security hole or bug). However, leaders within the development teams have expressed a desire for increased cooperation. Apple, which is now importing large portions of FreeBSD into its Darwin operating system, has also raised the BSDs' profile and may cooperate with the other projects (though, at least for the moment, code covered under Apple's license cannot be used in any of the other BSDs without subjecting them to problematic intellectual property constraints).

From this perspective, it's not important whether any of the BSDs ever grabs headlines like Linux, or surpasses it in sales. Regardless of which platform we use, we all owe BSD a great debt -- not only for the BSD code that all of them contain, but for the existence of the Internet and the many operating system innovations that were first tried in BSD.

Source