One of the most difficult decisions that embedded developers face after choosing their microprocessor hardware involves selecting the GNU/Linux distribution. The scariest thing about getting started with Linux is the sheer volume of information available to the developer. It's especially daunting to wade through the millions of articles, Web pages, and news groups dedicated to Linux while facing hours of frustrating research. Also, not all Linux distributions are created equal. In some cases, a distribution from one company will be missing key features or board support that's available in others. A developer must better understand the entire problem prior to searching for a Linux distribution for a particular need.
If I Only GNU Linux begins with GNU. (GNU is Not Unix.) The target of the GNU Project was to develop a complete free UNIX-style operating system. GNU/Linux is one variant of such a system. It consists of the Linux kernel and packages like Compilers (gcc), Assembler, Linker etc. (binutils), and many others needed for a whole system (textutils, fileutils, ... EMACS, ...). These tools are known colloquially as the GNU Tools.For example, the GNU compiler (GCC) and Binary Utilities (binutils) form a collection of the necessary tool executables for compiling or cross-compiling, linking, and assembling C or assembly code for particular processor types. These processor types are typically known as "arches," which is verbal shorthand for "architectures." This means that a particular version of the binutils must address the needs of the particular processor type and family.
All GNU components and the Linux kernel are "GPLed," meaning that the free use of the code is tied to the General Public License (GPL). No discussion of Linux or the GNU tools themselves would be complete without at least mentioning the potential legal issues inherent in using GPLed code. While this discussion is out of the scope of this article, it is an important issue for anyone using Linux or distributing Linux applications code and drivers.
The Usual Suspects The starting point for any Linux search is to visit its primary kernel site, which features the latest official updates and patches to the Linux kernel (see the table) . For any open-source software, there is always SourceForge.net. This is a good place to find hard-to-describe applications and drivers that aren't currently in a Linux distribution. You can visit many other Linux organizations from these two locations. As an instance, "penguinppc.org" is a good starting point for PowerPC processors.For commercial Linux vendors, the starting point depends on whether you're looking from the software or hardware side. From the hardware side, it often makes sense to go to the particular hardware site and see what Linux distributions are supported by the particular evaluation hardware. In this article, a Freescale MPC8560 evaluation board is used, so visiting the Freescale Semiconductor site makes sense.
The software side features three types of commercial choices. The first and most obvious choices are the "big gorillas" of embedded systems, such as LynuxWorks, MontaVista, TimeSys, and Wind River Systems. The second type, such as Arabella Software and ELinOS, serves up less expensive development solutions. The third type includes solutions that strictly follow the ideals of the GNU Project by providing their solutions strictly as free software, like DENX Software Engineering. In all cases, however, the first consideration in the Linux distribution search first is: "What is available for what hardware?"
A Hard(ware) Day's Night Once the developer is clear on the required software (i.e., a full Linux distribution including both the Linux code and the GNU tools), the choice of hardware must be examined. Recently, a quiet change has been occurring in embedded hardware design: No longer does one need to create custom hardware by first obtaining a processor and then obtaining appropriate board-level devices for that processor. Processor design wins these days are often determined by how much inexpensive evaluation hardware is available for a particular processor. Most if not all of these ready-made development platforms come with the actual design schematics, giving the hardware design team a leg up on in-house board design.But how does this affect the Linux distribution effort? Such an effort is driven not only by the particular processor, but also by the evaluation platform. Software developers are due for a rude awakening if there's no Linux support available for their evaluation platform. Unfortunately, in many cases, the software engineer had no input in evaluation-platform selection. Software-development teams thus may be stricken with hardware that's not immediately available for developing the application.
A good place to start when addressing this problem is with the vendors and organizations mentioned previously. But, because free and commercial solutions are available, another important development decision emerges.
Build Or Buy? Linux is free. The GNU tools are free. So why is there a build-versus-buy decision implicit in the Linux distribution effort? Support for both Linux and the GNU tools themselves most assuredly has an associated cost. An organization involved with Linux must make a basic decision on support costs. Support costs may be known because it's a purchased product, or it will have to be performed in-house. Such costs often are hidden away when examining the overall expense of using Linux. If a company already has a Linux guru on staff, it's usually cheaper to perform the support on an in-house basis. Otherwise, it would be wise to purchase Linux support from a third-party vendor.One advantage of the build-versus-buy debate is that it not only forces the developer to gain a greater understanding of Linux support costs, it also illuminates the need for useful development tools. For years, this was a huge drawback to using Linux in commercial products. This is no longer the case, however, as the advent of the Eclipse framework has spawned another quiet revolution over the last few years.
In a manner similar (but not identical) to the Linux and GNU free software model, the Eclipse framework is a great leveler of the playing field for small organizations eager to provide their own Linux development environments without maintaining a large team of dedicated engineers. Not only have small players in the field adopted the Eclipse framework for their own use, larger vendors also have joined the Eclipse community. Because Eclipse is open-sourced, it can even be used as a standalone Linux development environment.
How Fast? One discussion running through the embedded community is the buzz concerning Linux in a real-time environment. All too often, the issue is clouded with fear, uncertainty, and doubt (FUD) from vendors looking to differentiate where there may not be any differentiation. Up until recently Linux had no true real-time capability per se, but there were alternatives.One alternative is the Real-Time Application Interface (RTAI) for Linux, a small patch to the Linux kernel promising RTOS-level (real-time operating-system) performance. A different proprietary approach is TimeSys' CPU and Network Reservation products, which promise true RTOS performance via a licensed product. Still another approach is the modified kernel approach chosen by Arabella Software. More recently, the standard v2.6 Linux kernel increased kernel performance via an improved kernel scheduler, further blurring the difference between Linux and a conventional RTOS.
Now that several alternatives exist for the real-time Linux performance issue, the developer also must weigh the benefit of a faster kernel against the performance and cost targets. While using RTAI may be a good bet, as it is orthogonal to the Linux kernel, the choice of the TimeSys or Arabella solutions may be necessary even if it forces a deviation from the standard Linux kernel. This deviation could come back to haunt users when the Linux kernel changes.
A Real (-Time) Example This article examines a particular series of Linux kernel decisions surrounding support for the Freescale PowerQuicc III processor. Our small organization is heavily invested in exactly which processors to support to hit the appropriate percentage of our market. Because we already had a long history with the PowerQuicc I and II processors, it made sense to put together our own Linux solution for the newest member of the processor family. While a "pure vanilla" solution could meet our immediate need, in the longer term, we needed to know the availability from both open and proprietary standpoints.The first step in creating a Linux solution is getting to know the hardware. After gaining a good understanding of the hardware, research available Linux solutions. A "vanilla" solution is necessary if only as a baseline against which to gauge the real-time solutions. Many real-time alternatives, such as RTAI, are patches to existing kernel versions, so the vanilla solution should be the first solution.
The PowerQuicc III: An Introduction The PowerQuicc III is an integrated communications processor with a communications processor module (CPM) that provides multiple integrated I/O channels. The PowerQuicc III CPM module supports up to three fast serial controllers (FCCs), two multichannel controllers (MCCs), four serial communications controllers (SCCs), one serial peripheral interface (SPI), and one I2C interface. The processor uses the e500 Book E compatible core. Along with the e500 core and the CPM are two integrated 10/100/1000 Ethernet controllers, a double-data-rate (DDR) SDRAM controller, a 64-bit PCI controller, and a RapidIO interconnect.The PowerQuicc III has quite a lot of I/O, but what's needed to run Linux on it? It may not be possible to get all of the I/O immediately, yet it may not be necessary anyway. Most cases only require serial I/O and Ethernet. In fact, many operating systems, including Linux, may only support these two processor interfaces.
The PowerQuicc III: Going To The Source Motorola's spinoff company, Freescale Semiconductor, manages the PowerQuicc III. Searching the Freescale site using "software for MPC8560" brings you to the MPC8560 product summary page. The summary pages are a wonderful resource for pulling together all of the chip vendor's information. While by no means complete, they're an excellent resource when starting out. The first evaluation platform we find is, naturally enough, Freescale's MPC8560ADS. This was the only hardware platform listed, though, so a bit more indirect searching is warranted.In the summary page under operating systems, you will find several RTOS companies with appropriate board support. Many of these OS companies don't make hardware, but one familiar source was there-Wind River Systems, one of the aforementioned "big gorilla" embedded software companies. A brief search of its product directory pointed us to its SBC8560/40 page, providing a second potential evaluation platform.
The PowerQuicc III: Gathering The Pieces After completing a basic hardware search, it's time to think about finding the vanilla solution. Our organization has long been involved with the free Embedded Linux Development Kit (ELDK) from DENX Software Engineering. Not only have all our previous BSPs and SDKs been based on that toolset and Linux distribution, many of our partner companies use its solution, too.It then made sense to see what support the Linux community had for the PowerQuicc III. A quick Google search using "Linux on MPC8560" indicated that support for the ADS evaluation board and the Wind River board was added to the kernel sometime in mid-2004. A trip to the main PowerPC page at www.ppckernel.org and http://rsync.ppckernel.org produced the latest 2.4.30 and 2.6.11 kernel versions with support for the PowerQuicc III. There were now two options for a 2.4 kernel: 2.4.25 from DENX v3.1 and 2.4.30 from rsync. There was also a 2.6 version to evaluate against the 2.4 kernel. We were now ready to start building. Then, we hit a snag.
There was no existing support in the ELDK 3.1 for the PowerQuicc III. A 3.1.1 ELDK version provides a firmware solution in UBOOT and the appropriate GNU tools support, but no board support. This meant that the 2.4.25 kernel direct from DENX wasn't sufficient. Finding PowerQuicc III board support other than the rsync versions would require more investigation.
The PowerQuicc III: Building A Solution It was time to go back to the Web to find specific commercial board support for the PowerQuicc III. After some searching, it turned out that both TimeSys and MontaVista had public board support for the ADS evaluation platform. These provided alternative commercial solutions in addition to the public kernels from the rsync site. There are five basic steps to follow to get a final build solution for the ads8560 platform.Step 1: Installation
One of the biggest advantages of working with the ELDK tools is the ease of installation. Simply load the CD and install using the install.sh script. This script deals with every nuance of the delivered RPM packages without requiring the user to be an RPM expert. Many solutions don't provide this level of control. Instead, they require that you extract RPMs, which unnecessarily complicates the whole process. If the user is installing an entire architecture family, such as the complete ELDK for PPC, expect the install process to take a bit of time, especially on older hardware.
In a similar fashion, the MontaVista Preview Kit with the LSP for the ads8560 board also has a script that allows for straightforward installation. The Preview Kit typically installs into the /opt directory within a few minutes. After some searching of the /opt/montavista directory, the user will find the LSP directory containing the actual Linux kernel-in this case, a 2.4.20 version.
Step 2: Configuration
The build process involved using the "make menuconfig" approach. Other developers use "make oldconfig" or "make xconfig" instead, as a matter of personal preference. The arch/ppc/configs directory contains default configuration files for specific hardware platforms. Here, the MontaVista LSP contained a file called mpc8560_ads_defconfig, which was copied to the kernel directory and renamed to .config. The actual configuration process is immediate when using a preexisting script as in this case. Otherwise, in many cases, selecting the appropriate selections for the kernel can take a few hours. Figure 1 shows an example of the menuconfig approach.
Step 3: Building
Usually the first step in creating a build solution for Linux is getting the basic command line build working with the particular tools solution (ELDK 3.1.1) and the particular Linux kernel tree you are using. In this case, it's the MontaVista Linux Support Package (LSP) for the ads8560 board. The two goals of this process are to first create a working "make clean," "make dep," "make" sequence that produces a final executable, and then to work these into an appropriate build script.
When building for embedded or cross-development targets such as PowerPC platforms, the user usually sets the ARCH and CROSS_COMPILE variables in the kernel Makefile. In this example, the Makefile was modified to add the lines "ARCH := ppc" and "CROSS_COMPILE = ppc_85xx-," replacing the default lines. Doing a typical Linux kernel build completely from scratch has been known to take 30 minutes or more. Time depends heavily on the speed of your desktop hardware and any other applications that may be running at the same time.
Step 4: Scripting
Once you have a valid build sequence, it's easiest to create appropriate Linux build scripts for the specific Linux kernel and tools combinations being used. These are especially useful when mixing GNU tools support from one vendor with board support from a different vendor. Figure 2 lists a typical build script. It has always mystified me why such build scripts aren't a standard part of every Linux distribution. Using appropriate build scripts, both the MontaVista 2.4.20 ADS and the TimeSys 2.6.4 ADS kernels were built using the ELDK 3.1.1 GNU tools. Builds were then performed for the rsync 2.4 and 2.6 kernels.
Step 5: Testing
Extensive testing with real hardware can commence. This is a huge area best covered in a separate discussion, but here are the basics. Now that a system has been created with an initial board boot-up solution (UBoot, Board Firmware), it is time to build the first typical Linux kernel. The Linux kernel requires a UNIX-like root file structure to run properly. This step can go in one of two ways. The first employs an NFS-mounted root file system. The ELDK tools already provide this type of file system via the /eldk311/ppc_85xx directory for PowerQuicc III hardware. The NFS-enabled Linux kernel is most commonly used as a downloadable image for application development.
In many embedded scenarios, however, the NFS-mounted kernel isn't the right choice. In these cases, the kernel must be configured to utilize an onboard RAMdisk image. The RAMdisk image, a separately compiled root file system, provides smaller Linux components in a root file system image that's suitable for burning into a flash device. RAMdisk images can vary wildly in size depending on which components are built in. Pre-built RAMdisk images are publicly available from multiple sources, or you can build your own. Combining the RAMdisk image in flash, the RAMdisk-enabled Linux kernel (which also can be stored in flash), and the boot-up solution in flash, you now have a final embedded Linux solution.
The PowerQuicc III: Bundling The Solution In our organization, we provide an Eclipse-based Linux development environment complete with a supported Linux software development kit (SDK) (Fig. 3). We have such a strong preference for the ELDK solution in part because the GNU tools and full Linux distribution all sit in a single directory hierarchy that's easily "tar'ed, zipped, and shipped."
We bundled the Linux 2.4.30-pre1 kernel from rsync with the ELDK 3.1.1 tools from DENX to create a final solution for users of the ads8560 board. Preferring a single directory structure for our SDKs, we moved the 2.4.30-pre1 kernel directory under the "ads8560_sdk/eldk3.1.1/ppc_85xx/usr/src" directory, effectively merging the two directory structures for tools and kernels with board support. The UBoot solution and the RAMdisk image also are provided within the same usr/src directory structure. At this point, we're free to tar the entire directory structure, gzip it, and manufacture the SDK for shipping. While merging the two directories isn't strictly necessary, it presents a simpler structure to the end user.
The Eclipse framework lets users add their own customer scripts into the environment via the external tools menu pull-down (Fig. 4). Our initial product integration using this capability is shown with a completed build in Figure 5.
This article has endeavored to cover a good deal of ground on how to pull together a Linux distribution from the many different possible sources and vendors. It also has called out many of the potential pitfalls that await the unprepared or unwary developer. Ultimately, when the development team has made their respective choices and engineering tradeoffs necessary to settle on a particular solution for their development needs, though, one outstanding question remains: Whose Linux is it anyway? Regardless of whether you're using a proprietary or fully open solution, and regardless of any real-time extensions you may choose, the answer is obvious. It's your Linux!