Risk-Based Testing in Automotive: Essentials for Dodging a Software Crash
Download our Expert Opinion
- English version:
Risk-Based Testing in Automotive: 7 Essentials for Dodging a Software Crash
- French version:
Risk-Based Testing pour l’Industrie Automobile : 7 Clés pour Éviter le Crash Logiciel
Automotive embedded software development is a specific business. In this complex environment, defining a Risk-Based Testing strategy makes perfect sense. With over ten years of experience in software and systems engineering projects quality management, especially among major players of the automotive industry, we identified the key elements to build and lead an efficient and rational testing strategy based on risks.
For 30 years, the share of embedded electronics has been growing in the automotive industry, for the benefit of passenger safety, functionalities reliability, performance and comfort. A trend that will accelerate in the coming years, to see the wide-scale development of ever more 1. electric, 2. automated and 3. interconnected cars.
For each of these development axes, software has become a major and transversal component, and is now indispensable to prevent accidents and malfunction risks. To the point that the number of lines of code in car systems already exceeds that of some software used in the aviation industry. Indeed, a recent car model would embed an average of 10 to 20 million lines of code (generally written in C, C++ or Assembler), compared with 1.7 MLOC in a F-22 Raptor, and 6.5 MLOC for the Boeing 787. A figure that is expected to grow tenfold in a near future, according to Frost & Sullivan.
Specific software constraints to the automotive industry
Moreover, software development in the automotive industry is a singular –even unique– business, because of the complexity of its constraints:
- High volume of distribution: The car is the most widely used mean of transport. Embedded software is logically overexposed to potential exceptions and failures.
- Reliability is not a luxury: Since user safety is involved in the core functions of a vehicle (breaking systems, airbags, etc.), reliability and stability of the software need to be guaranteed by the manufacturer. A defect can seriously damage the brand reputation, and tragically impact the potential costs of a correction, as well as the business performance of a car model. This was recently the case for Toyota.
- Users cannot act on systems: While airline pilots can switch to manual mode and still master the different instruments on board, the driver of a car only has a driver’s license. When it comes to the software, indeed, the developers are the only people in command. Therefore, risks have to be managed upstream.
- A highly competitive market: Even if the industry tends to be more concentrated, the opening-up of the automotive market strongly fostered competition, with the entering of new players and brands. While the part of the software cost increases in car production, the development processes have to be optimized.
- Time-to-Market: Manufacturers must be able to quickly integrate the recent technology evolutions in their systems, especially with regard to connectivity. Apple’s “Carplay”, unveiled at the last Geneva Motor Show, outlines the shape of things to come in enhanced driver comfort.
- Inflexibility of the delivery: The pace of software deliveries is more similar with the mobile device industry than the airline or aerospace industries. Conversely, it is still a difficult and expensive decision to modify a software component that has been already delivered, until cars are not fully connected. So far, test-related costs will always be lesser than the costs of a recall of hundreds of thousands of vehicles.
Risk-Based Testing: A factor of competitiveness
We can easily understand that developing and testing software for the automotive is an extremely complex activity. Indeed, in order to remain competitive without compromising the reliability of the systems, the validation efforts require to be right-balanced and oriented to achieve the « Good Enough » objective. That is to say getting an acceptable level of reliability, stability, performance and security, for reasonable costs and time frames.
This balance can not be achieved without a rational risk assessment, also recommended as a core practice by the Automotive SPICE standard. This assessment is moreover required by the ISO 26262 that defines an approach based on the risks specific to the automotive industry, while allowing some flexibility in the analysis of these risks.
In this context, defining a testing strategy based on risks is about a must. Risk-Based Testing is an approach that enables to define priorities, to execute and control the tests of software components, while taking into consideration the different kinds of risks: functional criticality of the tested features, specific environment and constraints of the project, code complexity and stability, risks associated to development teams experience and technical skills, etc.
Find a detailed presentation of the 7 key elements listed below to build and lead an efficient and rational testing strategy based on risks.