Typical devices and applications used by multimedia are well suited to high-quality playback. But the underlying resource management provided by standard Linux and other general-purpose operating systems renders it impossible to achieve high quality, unless nothing else is running. Designers, though, can achieve major quality-of-service (QoS) improvements in Linux-hosted multimedia player operation by implementing CPU reservations provided by the operating system.
There's no question that multimedia technologies have become commodities, and that players are now ubiquitous. MPEG-2 video players are available in satellite receivers, delayed television players, portable DVD players, cell phones, and the workstations that spawned the Internet age. Likewise, MP3 audio players are in car radios, PDA devices, cell phones, and workstations.
However, this ubiquity has forcefully exposed the shortcomings of the traditional Ethernet infrastructure and the timeshare scheduling. Low-resolution, small-frame video; clipped and tinny audio; stalled video; lost audio; and poorly synchronized audio and video are the common experience of every Internet user, no matter the processor speed, the amount of memory available, or even the Internet connection speed. To assure adequate performance on both network-sourcing and stored-content players, players are limited to single functions because of the inability to assure QoS in the presence of multiple processing tasks.
Now, a dramatically effective new scheduling policy changes all of that. It enables multimedia players, operating concurrently with arbitrary processor loads, to secure the computational resources they require to operate continuously with a guaranteed high QoS.
Multimedia player implementation:Figure 1 illustrates the devices and processing stages typically present in multimedia generation and presentation devices. The upper pipeline illustrates the content generation process, starting from sampling of the physical environment and leading to persistent storage or network transmission of the recorded event. Likewise, the lower pipeline illustrates the content presentation process, starting with content retrieval from storage or a network, and leading to video and audio recreation. While this article focuses on the presentation pipeline and the example below uses disk-based rather than Internet-sourced streams, the described reservation mechanisms apply to all stages.
The multimedia player discussed below was executed on a 662-MHz Pentium III Dell OptiPlex GX200 with an Intel i810 Analog Devices AD1881 sound device; an nVidia Vanta video graphics controller; ALSA 0.5.10 drivers for sound (with the emulated OSS core); Xfree86 3.3.6 for video; and RealPlayer 8.0.1 for video presentation. TimeSys Linux/RT 1.2, Linux 2.4.2-pre3 was the operating system used. The test multimedia clip, New Line Cinema's 2001 Lord of the Rings film trailer, is an 11.4-Mbyte, 1:46-min, 1000.1-kbit/s RealMedia file with 64-kbit/s stereo audio.
Resource reservations: A reservation consists of a guaranteed-available-utilization duty cycle for a single resource, such as CPU runtime, memory pages, memory bandwidth, network buffers, network bandwidth, disk blocks, or disk buffers. For simplicity, only CPU reservations are discussed in detail here. But other reservations are possible, and some are already available on actual Linux platforms.A reservation has parameters that define its periodicity, the deadline within each period, and the amount of time reserved for use within those parameters. The parameters include "T" for the period, "C" for the uninterrupted computation time, and "D" for the relative deadline in each period.
Additionally, each reservation has a depletion policy parameter that describes whether tasks attached to the reservation are allowed to use the reserved resource when the reservation becomes depleted within each reserved period. "Hard" reservations deny use of reserved resources after completion of C in each period. "Soft" reservations grant usage of reserved resources to attached tasks after completion of C in a period, but in competition with all other tasks in the system.
Figure 2 illustrates the depletion and replenishment behavior of reservations. Another way to view a hard reservation is that in addition to protecting the reserved task or tasks, it protects all other tasks from attempted monopolization by the reserved task.
For instance, an application might reserve 2 ms of processor runtime every 10 ms, with a deadline of 8 ms (effectively allocating itself 20% of the processor). The reservation for this would be characterized as:
\{resource=CPU, T=10 ms, C=2 ms, D=8 ms, depletion=HARD\}
Determining parameters: Several techniques exist to determine appropriate parameters for the resource reservations required by an application. When designing a new application that interacts with physical systems, the system's constraints can often suggest the parameters for appropriate reservations. For example, an application to display video at 33 frames/s might have an appropriate reservation period of 33.33-ms.Once an initial parameter set is defined, the C value can be "guessed," and the reservation can be tested by a successive-approximation process—better known as trial and error. The reservation can be adjusted via a simple utility, or by the application using the provided API. Determining the optimal value for C can be tricky. But in Linux/RT, this can be as easy as reading the /proc/rk files and looking at the minimum, maximum, and average utilization statistics for the reservation after an initial setup. A first rule of thumb is to look for average utilization within the same order of magnitude as the re-served utilization.
Comparative performance: Applications performing streaming multimedia require service quality that's predictable, configurable, and managed on an end-to-end basis—from origination application to destination application through intervening I/O buffers, devices, operating-system tasks, protocol stacks, and across network flows. This service quality is an amalgamation of subjective and objective measures, such as round-trip latency, jitter in latency, bandwidth, stream synchronization, frame size, color depth, compression, image artifacts, sound quality, and so forth.Measuring QoS: The subjectivity of QoS makes its objective measurement difficult. For audio, the traditional approach has been to survey an audience of listeners and compile a Mean Opinion Score (MOS) of the audio quality. Recently, computer techniques have begun to appear: the Perceptual Speech Quality Measurement System (PSQM) and Perceptual Analysis/Measurement System (PAMS) are now in wide use. These use carefully selected speech samples as inputs. They then use signal processing on the outputs to compare the received signal with the original sample and generate a comparative MOS value.A subjective technique, im-plemented here, is simply to listen to the audio playback, as the human ear is a very sensitive detector of poor quality. The reserved utilization can be tuned down until the ear de-tects stuttering in the audio playback. It's then nudged back up, so no stuttering occurs. Likewise, for assessing video quality, a similar technique of looking for pauses, blurring, and lost lip synchronization is used to determine when the video player lacks resources.
Test configuration: Multiple players could be run simultaneously to determine how many of these players can operate at the same time without affecting the quality of their peers. Unfortunately, it turns out that the Advanced Linux Sound Architecture (ALSA) or Open Sound System (OSS) sound drivers in Linux use one or more semaphores to provide mutually exclusive access to the sound hardware.Although it's quite reasonable, this prevents one from running multiple players. (Windows, by the way, makes no such restrictions, and one can start up a true cacophony of audio players.) Therefore, the approach taken is to run a single player in competition with heavy processor loading, and to determine the player's ability to maintain QoS in spite of the competition.
Player performance without reservations: A processor reservation that assigns a specific portion of the processor to an infinite-shell loop is first created. This creates a well-defined load that will demonstrate the effectiveness of the player reservations—by competing against the multimedia player while it operates. This reservation is increased until the quality of the unreserved player becomes noticeably impacted.The RealPlayer and the X server, when unreserved, are able to withstand a constant, competitive load of about 5%—a loss of 500 µs in each 10-ms period. Beyond this, RealPlayer begins to drop frames, and X begins to stutter-paint the requested frames. Therefore, the competing reservation is created with parameters:
\{resource=CPU, T=10 ms, C=500 us, D=10 ms, depletion=HARD, inherit=TRUE\}
As the competitive load is increased, either by increasing the reservation or by adding additional unreserved tasks to the workstation, the quality of the presentation continues to deteriorate. More and more frames are dropped, X repaints begin to stutter, and eventually even audio becomes affected.
Player performance with reservations: To assure that the players are competing against the same environment as experienced by the unreserved players, the infinite-shell-loop reservation is recreated. It's given the maximum amount it had in the unreserved case, or the amount of processor remaining after assuring the players their minimum reservations. In this case, the same 5% reservation can be granted to the competitive load.RealPlayer requires approximately 65% of the processor, i.e., 6.5 ms per each 10-ms period, and a corresponding X server reservation for 30% of the processor, or 3 ms per each 10-ms period. This is required to play the video clip without stuttering frames and to achieve proper lip synchronization. It can be readily provided by using the rkexec script to create a reservation with parameters:
\{resource=CPU, T=10 ms, C=6.5 ms, D=10 ms, depletion=HARD, inherit=TRUE\}
for RealPlayer, and a reservation for X with parameters:
\{resource=CPU, T=10 ms, C=3 ms, D=10 ms, depletion=HARD, inherit=TRUE\}
Having been given these reservations, the RealPlayer player is, for all practical purposes, immune to other activity on the desktop. Additional unreserved tasks on the workstation compete for the scarce remaining resources. Still, the presentation maintains its quality. (With 95% of the processor dedicated to the RealPlayer presentation, little remains for anything else. But one can actually squeeze quite a bit from 5% of a 662-MHz machine running Linux.)
Simple deployment for applications: Because the resource reservations can be created externally to applications and can have tasks dynamically attached to these reservations, employing reservations for the majority of systems today is relatively easy. No code modifications are necessary, and the reservations can be set up through shell scripts and managed via applets.For those applications that need more direct control of scheduling primitives in general and reservations in particular, an API and a system-call library are available to enable resource reservation operations to be performed directly from application code.
The ability to provide this QoS assurance is unique to the Linux/RT operating system. Details on this operating system and APIs for its Resource Kernel are available in online documentation at www.timesys.com.
Doug Locke is vice president of engineering for TimeSys Corp., Pittsburgh, Pa. He received his PhD from Carnegie Mellon University, Pittsburgh. Contact him by phone at (412) 681-6899, ext. 233, or via e-mail at [email protected].
Lonnie VanZandt was a senior systems engineer at TimeSys Corp. when this article was written. He holds a BSCE and an MSCE from the University of Illinois, Urbana.