Electronic Design
Dealing With Multicore Application Testing Issues

Dealing With Multicore Application Testing Issues

Multi-threaded applications are becoming more sophisticated as multicore and many core platforms become the norm even for embedded applications. I recently spoke with Andy Chou is, Co-Founder and Chief Technology Officer of Coverity about development challenges and how static and dynamic analysis tools can address a range of issues including security defects.

Wong: What industry trends are driving multicore development?

Chou: For the past 40 years, Moore's Law has held true, with the number of transistors available in a circuit doubling approximately every two years. Moore's law, however, has reached its performance ceiling. In response, hardware manufacturers have developed new multicore devices to accomplish the speed gains to which the world has grown accustomed to.

Now, the world of software development is presented with a new challenge. To fully leverage this new class of multicore hardware, developers must change the way they create applications. By turning their focus to multi-threaded applications, developers will be able to take full advantage of multicore devices and deliver software that meets the demands of businesses and consumers.

Wong: What are the challenges and considerations for development teams when developing for multicore systems?

Chou: Multi-threaded software adds a new wrinkle of complexity for those of us who care about software quality. Concurrency defects such as race conditions and deadlocks are software defect types that are unique to multi-threaded applications. Complex and hard-to-find, these defects can quickly derail a software project. To avoid software defects that could lead to problems such as or failures and unexpected behaviors in multi-threaded applications, software development organizations must understand how to identify and eliminate these problems early in the software development lifecycle.

Testing software code to identify defects in a scalable, accurate manner is notoriously difficult, even for single-threaded applications. One reason this is a challenge is that properly testing an application requires the simulation of all possible inputs to the program. Many organizations are satisfied when they achieve 95% line coverage in their programs (meaning 95% of all lines of code in their program are executed at least once in their test suite). However, this metric of coverage does not even come close to the total number of possible execution paths through a code base. Each decision point in a program means at least a factor of two in the number of paths though that program. Multi-threaded applications add a new level of complexity to the program that results in an exponential increase in the number of possible run time scenarios.

Wong: Given the rising importance of security, does multicore development pose any security risks and what should development organizations do to minimize these risks? How can static and dynamic analysis address problems like these?

Chou: The challenge of ensuring that applications and software systems are secure and free of easily exploitable security vulnerabilities is present in single threaded software, and is amplified in multi-threaded applications. The reason is because the new level of complexity in multicore development brings with it the potential for introducing more coding defects, which are then exploited by attackers and turned into security vulnerabilities. An example of this vulnerability is a resource leak that gets triggered only when the application is multi-threaded. If these resource leaks are not identified and resolved during development, they become vulnerabilities that can be exploited by attackers to cause Denial of Service (DoS) attacks.

Developer testing technologies like static and dynamic analysis provide an automated and sophisticated way to analyze multi-threaded code, identify defects that can be exploited and provide developers with accurate information to fix them. Both static and dynamic analysis have their advantages and should be used together. Static analysis does a broad analysis of the entire code base, without needing to execute the code and identifies a wide variety of errors by evaluating all the execution paths the program can take. While dynamic analysis, because it executes at runtime, allows a developer to test specific areas of the software and uncover deep, hard-to-find concurrency defects such as race conditions, deadlocks and resource leaks in multi-threaded applications.

Wong: How do developer testing technologies, such as static or dynamic analysis, help developers working on multicore projects?

Chou: Developers moving to multi-threaded applications need sophisticated and automated testing capabilities to help them identify race conditions and deadlock defects during development, when it's easiest and cheapest to find and fix problems. Because these types of errors are nearly impossible to replicate in dynamic testing environments, developer testing solutions, such as static analysis, is uniquely suited to play an important role in eliminating concurrency defects early in the software development lifecycle. However, due to their underlying complexity, some concurrency defect types have historically escaped detection by conventional static analysis tools. Today, sophisticated new technology is advancing the science of static analysis to help developers meet the challenge of creating multi-threaded applications. By arming developers with static analysis capabilities that detect complex concurrency defects early in the development lifecycle, organizations will accelerate their ability to produce and release multi-threaded applications with greater confidence.

Wong: What other problems does Coverity help developers solve?

Chou: Here at Coverity, we are the market leader in developer testing solutions that help development organizations get visibility into the quality, security, and complexity of their software code-and the teams that develop it. Building upon our leadership in static analysis and customer demand, in April we launched Coverity Integrity Control, a new solution for code governance that enables software development organizations to set policies for code quality, security, and complexity, automatically test code against these policies with our developer testing engine, and get early warning into development risk as the code is tested by flagging policy violations. This innovation was a direct response to customer demand for greater visibility into the quality and security of code being developed across the software supply chain. What do we mean by software supply chain? For a given development project, code is being developed across a variety of sources including geographically distributed development teams, third party suppliers, and open- source projects. The software supply chain is many times a business necessity due to shortened time to market windows and increasing customer demand for new products and features, but can also turn into a liability if the code is not properly tested for defects and not flagged early in the development cycle.

Hide comments


  • Allowed HTML tags: <em> <strong> <blockquote> <br> <p>

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.