Digital signal processors (DSPs) are a foundation element of virtually every type of consumer electronics, especially in digital media and communications. The form that these DSPs take is shifting as the standalone processor gives way to multicore and system-on-a-chip (SoC) designs. Vendors are striving, though, to ensure that consumer device developers have as easy a task as possible when utilizing this once-arcane technology.
Open up any electronics device in the home and chances are good that you will find a DSP inside. You may not recognize it as such, however, because the DSP is so deeply embedded within the application. In a cell phone, the DSP lies at the core of the baseband processor, where it handles tasks such as the audio codec. In a PC, DSL interface, or wireless router, the modem devices hide secret DSPs inside.
Every portable multimedia device uses a DSP to handle image processing and compression. This includes both simple media players and multifunction playback devices such as Apple’s iPod, as well as capture devices like digital cameras and camcorders. Many of the DSPs inside these devices are known by other names, though, such as video processors or image encoders.
The demands on these DSPs are growing. Increasing adoption of high-definition video has multiplied the pixel rate that video processors must handle both at the set-top and increasingly among mobile devices. Similarly, high-definition audio with surround sound has increased the precision and performance demands on audio processing DSPs. Even relatively simple systems such as digital cameras are expanding their needs with the advent of features such as image stabilization and automatic face detection.
Internet convergence is providing an additional opportunity for these DSPs to proliferate. Television set-top boxes, Internet Protocol (IP) telephones, cameras, gaming systems, and media players are blending together in various combinations, interchanging information, and collectively connecting to the Internet. As once standalone consumer devices evolve, their core DSP functions need augmentation to handle wireless communications as well as the new media they must now manage.
STANDALONE DSP EVOLVING
To meet this growing array of needs, the traditional numbercrunching DSP has had to evolve. Yet the standalone, generalpurpose programmable device is still in use. The Analog Devices (ADI) Blackfin DSP family, for instance, is available in broadly targeted versions. Similarly, the Texas Instruments (TI) C64x and C674x families have general-purpose members. Increasingly, however, the DSP functionality a consumer device needs is arriving in another form.
Where the number-crunching requirements are small, the DSP functionality may be as simple as a multiply-accumulate (MAC) block or other DSP-like extensions to an otherwise conventional RISC processor. ARM offers the V5TE DSP extensions for several of its processor families including the ARM9. It also incorporates the DSP-based Neon media processor in the CortexA8 processor family. Similarly, MIPS has DSP application-specific extensions for its MIPS32 and MIPS64 architectures. Tensilica can add DSP functions to its Xtensa LX3 and Xtensa8 customizable cores as well as offer the VectraLX DSP engine.
A common form in applications that have relatively stable, well-bounded requirements for DSP functionality is the accelerator coprocessor. In a DVD player, the microcontroller handling the user interface and system control can have coprocessors for handling demanding functions such as video and audio decoding. These engines are typically DSP-based but target specific application requirements, often by the mix of integrated peripherals they include.
For example, Freescale offers its Symphony DSP family for high-definition and surroundsound audio processing. Such coprocessor engines are optimized to handle just the one task, but retain the flexibility for reprogramming to accommodate requirement changes such as increased resolution or to handle new codecs for improved playback quality.
Often, vendors don’t even identify these targeted devices as DSPs. Instead, these products are known by other names. The DSP Group’s XpandR II family, for instance, targets Digital Enhanced Cordless Telecommunications (DECT) and Wi-Fi wireless telephony. The XpandR II incorporates a CEVA Teak- Lite DSP core, but the company literature simply calls it a wireless chipset processor (Fig. 1).
Where bounded functionality has stabilized to the point of being a commodity requirement, the DSP has become a fixedfunction device that’s scarcely distinguishable from hardwired logic. It is preconfigured for a specific operation, with its programming stored in on-chip ROM.
Simple, low-cost devices such as MP3 players and telephone modems use DSPs this way. As with other targeted devices, vendors do not typically identify them as DSPs. Examples include Zoran’s line of digital camera image processors and Vimicro’s line of IP camera processors.
Vimicro’s latest camera processor, the VC034, is dedicated to handling a digital still image and sending it over a USB link to a host. It includes an integrated interface for direct connection to the image sensor, hardware for creating the JPEG header, and a USB slave interface, all surrounding a ROM-based image signal processor that handles the JPEG compression.
The device’s design could also have implemented JPEG compression in hardware. But since it’s firmware-based, it can offer developers some flexibility for the implementation of additional processing, such as face tracking, as well as the addition of new features. Such flexibility is one of the key reasons that DSPs remain in fixed-function devices rather than migrating all the way to hardwired designs for lowest cost. Consumer device developers need the ability to customize their product’s features to remain competitive.
EMBRACING THE MULTICORE TREND
At the other end of the consumer spectrum, DSP requirements are both unbounded and in continual flux. This is the case with Internet convergence devices, which are seeing a proliferation in the types of media they must support. They’re also seeing many changes in the algorithms for handling the media due to evolving compression techniques. Further, consumers are demanding evergreater image resolution and sound quality in the media playback, increasing the performance demands on the DSP.
The answer that has evolved to address these shifting requirements and growing demands is the multicore embedded DSP. In such devices, one or more programmable DSP cores join a RISC CPU core integrated into a single chip. The TI OMAP L-138 combines an ARM9 CPU with a C674x DSP. The devices in the Zoran Quatro 4500 series, which target printers, have an ARM11 CPU and four DSP cores.
These types of combined devices give a consumer product developer both the performance and flexibility needed to keep pace with dynamic design requirements. Often, DSP chip vendors offer a family of combined devices to give developers a selection of cost and performance tradeoffs.
As with single DSPs, these multicore devices come in varying degrees of generality. The TI OMAP-L138 includes a video I/O block but is otherwise undedicated for a specific application. The Toshiba SPURSEngine SE100, on the other hand, is structured specifically to handle streaming media (Fig. 2).
The SE100 integrates four DSPs known as synergistic processing elements (SPEs) with hardware encoders and decoders for MPEG2 and H.264 video streams. The decoders and encoders let the SPEs work with baseband video even though the device’s input and output are compressed. Having the codecs as separate cores offloads those tasks from the other processors and simplifies the implementation of image processing for streaming media.
The popularity of the multicore approach has broadened the market for DSP cores from vendors that offer only cores, not chips. Chip developers targeting the consumer market are increasingly turning away from their once-proprietary DSP design foundations to embrace licensable cores.
The DSP core company CEVA is seeing rising success with its Teak and TeakLite cores in the baseband processing for cellular phones. It reports that many baseband chip vendors, including Broadcom, Infinion, and ST-Ericsson, have moved away from internal, proprietary DSPs to use CEVA cores in what may be as much as 50% of the cell-phone units sold in 2012.
Other consumer application chipset providers are moving away from internal, proprietary DSPs. The STi5189/5197 family of set-top box chipsets from STMicroelectronics uses the DSPenhanced ST40 CPU core along with proprietary graphics and audio processing cores to handle decoding tasks. Recently, however, STMicro announced plans for an HDTV chipset design that steps away from the company’s own cores. These next-generation devices will incorporate the DSP-enhanced ARM Coretex processor along with ARM’s DSP-based Mali-400 graphics processor instead of STMicro’s proprietary cores.
Two factors appear to be prompting such transitions from proprietary to third-party DSP cores. One is the demand to reduce the need for extensive in-house DSP expertise. By incorporating a third-party core instead of maintaining a proprietary core at a competitive level, the chip vendor frees design resources to focus more on the system aspects of SoC design.
The other factor, especially among fabless semiconductor companies and consumer product developers creating their own ASICs, is the desire to have multiple sources of supply. Using a third-party DSP core gives developers a choice of foundries, while a proprietary core may restrict developers to using the core creator’s production line.
BROAD FAMILIES OFFER APPLICATIONS SCALING
Whether offering cores or chips, though, DSP vendors are typically creating products that span the full range of design approaches to address the breadth of consumer application requirements that DSPs must meet. Often, these products form software-compatible families that range from the highly targeted to the broadly generalized.
TI offers the general-purpose C6742 and the multimedia-processing OMAP L-138 in the same family (Fig. 3). The OMAP 35x forms a similar family, sharing upward software compatibility within the family as well as compatibility with other C64x DSPs and ARM9 processors. Such extended compatibility blurs the lines between product offerings, making selection less critical for developers. If a given DSP proves sub-optimal for the application, it is relatively easy to move to a different variation.
The distinctions between individual DSP offerings from vendors are further blurring as the vendors, depending on the markets they are trying to address, point their product development efforts in multiple directions simultaneously.
One development direction vendors are taking is to optimize standalone and multicore DSPs for existing applications by integrating targeted peripherals on chip. This produces products that follow the application requirements as they become increasingly established, slowly trading flexibility for lowered cost and increased performance.
Another direction is to continue expanding the performance of general-purpose DSP offerings within existing product families, addressing the needs of applications still in the developing market stage where flexibility is essential for tracking rapid market changes. Vendors are also continuing to investigate new DSP architectures to handle the unprecedented requirements of emerging applications where a jump in performance is required.
ARCHITECTURES STILL EVOLVING
One such new application is the use of software-defined radios in cell phones and wireless Internet devices. These products need to be compatible with the whole range of wireless communications protocols, including Bluetooth, Wi-Fi, WiMAX, and multiple generations of cellular service, to provide the seamless and ubiquitous connectivity that consumers will demand.
Simply combining the chipsets for each type of connection would result in a design too large, expensive, and power-hungry for the market. The answer that is arising is the use of a highperformance DSP that can be programmed to handle any of the protocols as needed as well as control the radio-frequency selection, modulation, and antenna connections.
New applications sometimes require new architectures, and the DSP community is responding. CEVA’s CEVA-XC DSP architecture offers multiple computational units that blend singleinstruction, multiple-data (SIMD) and very long instruction word (VLIW) behaviors. A single general computational unit joins with as many as four 256-bit vector processing units to provide instruction-level parallelism capable of executing as many as 64 MAC and arithmetic operations in a single cycle (Fig. 4).
Whether developing new architectures or creating optimized combinations of processors and peripherals, however, DSP vendors do seem to be following at least one common trend: simplifying the consumer product developer’s task. DSP technology was once considered arcane, accessible to only a handful of experts. But the huge market for DSPs in consumer designs has made shattering that image a priority for vendors. Their efforts have targeted software as the accessibility barrier to be removed, focusing on simplifying software reuse and easing application development by offering open and often free libraries.
SOFTWARE DEVELOPMENT SIMPLIFYING
To simplify software reuse, DSP vendors have expanded their product offerings by forming families. This has allowed development teams to utilize a common tool set to develop and port software among DSPs that span a wide range of cost and performance points.
The TI OMAP, C64x, and C647x product lines all are or include software-compatible DSPs while targeting different applications. The TI DaVinci family, which specifically targets digital media processing, has more than a dozen software-compatible family members and sees several new introductions each year, most recently the DM365 introduced in March. Similarly, the ADI Blackfin family includes more than a dozen variations.
Core vendors are also striving for software compatibility within families so customers creating their own multicore processors can enjoy software portability across product generations and families. The CEVA Teak and TeakLite families share an instruction set and allow developers to cover the multimedia application range from simple portable players to full set-top boxes with wireless connectivity while reusing their code.
Along with supporting application code reuse by developers, DSP vendors are working to simplify initial application development, especially in emerging markets where rapid introduction of new product variations is key to successful participation. One way they are doing so is to eliminate the once-critical distinctions between fixed-point and floating-point computation in DSPs. Architectures such as the ADI Blackfin and the TI C674x now offer the ability to run either type of computation as needed, simplifying the porting of legacy algorithms.
DSP vendors are further simplifying development by offering extensive core function libraries that developers can use to quickly enter a new market. TI recently introduced the VLIB library for video analytics, which has consumer applications such as automatic face identification for digital camera focusing.
Such libraries also seek to eliminate barriers to their adoption by cost-conscious consumer product developers by reducing their cost to use. TI has indicated that it is moving its DSP libraries to an open-source model so customers can easily build on the foundations it provides. Meanwhile, ADI offers its customers royaltyfree libraries of core DSP functions for a variety of applications.
The cost consciousness of consumer product developers, however, is additionally causing the mix of off-the-shelf offerings from DSP vendors to be ever-changing and creating an ever-more bewildering array of choices. As markets mature and requirements stabilize, consumer product developers pursue cost reduction through high integration as well as by seeking multiple-source options for their components.
These pursuits force the off-the-shelf DSPbased product to yield to a customer-specific SoC design using licensable cores. Chip vendors, therefore, must continually revise their product offerings to remain targeted to applications where design flexibility trumps production cost.
The proliferation of DSP offerings for consumer products, therefore, will continue unabated. At the same time, trends toward scalability in function across the application spaces as well as along price and performance points within an application will likely reduce the diversity of architectures available as the challenge of maintaining software compatibility while increasing performance creates barriers to further development. Developers may well find themselves with more options and fewer choices in applying DSP technology to consumer design.