Aurora Autonomous Driving Safety Case Framework

In August 2021, the autonomous driving company Aurora launched the first-ever initial version of the Safety Case Framework for autonomous trucks and passenger cars, addressing the safety issues of autonomous trucks and passenger cars. This makes Aurora the only autonomous driving company in the industry that publicly shares its Safety Case Framework. This article introduces the industry significance, five safety principles, and applications of Aurora's autonomous driving safety framework.

Significance of the Safety Case Framework

Aurora uses a safety case-based approach to evaluate when autonomous vehicles are safe to operate on public roads and assess whether they do not pose an unreasonable risk to motor vehicle safety.

The Safety Case Framework is the most effective way to safely remove the safety driver and is an essential component for any company that hopes to operate and safely deliver large-scale commercial autonomous vehicles without a safety driver. The Aurora Safety Case Framework evaluates the entire development lifecycle of the vehicle, enabling faster deployment and determining when it is acceptable for autonomous vehicles to be safe on public roads.

Aurora views safety as an ongoing process rather than a static to-do list, and an evidence-based approach is critical both internally and externally. Internally, the Safety Case Framework is how we continually review evidence and evaluate the performance and development of the Aurora driver against our internal standards to ensure we have the confidence to put self-driving vehicles on the road with or without a vehicle operator. Externally, the Safety Case Framework allows us to effectively share our approach and progress with partners, customers, regulators, and the public. This transparency helps build trust, which is important when deploying any new technology.

Introduction to the Aurora Safety Case Framework

Aurora has adopted a safety case-based approach because it is the most logical and effective way to present and explain how Aurora determined that our self-driving vehicles are acceptably safe to operate on public roads. At the core of this framework is a structured argument with evidence for why our vehicles are acceptably safe. There are complex interactions and relationships between the many elements in a self-driving vehicle. No single piece of evidence can demonstrate the totality of safety. The safety case-based approach brings these two fundamental concepts together in a logical way to effectively present what we have done to determine that our vehicles are safe to operate on public roads.

Aurora developed the framework to help evaluate the entire development lifecycle of Aurora's trucking and passenger products in order to deliver secure and scalable products to partners and customers.

The Aurora Safety Case Framework combines guidance from government organizations, best practices from safety-critical industries, voluntary industry standards and alliances, academic research, and what organizations have learned in their own work. In the autonomous vehicle industry, it is an important tool for developing autonomous vehicles that can operate safely on public roads and delivering these vehicles to partners, customers, and the public.

Aurora's safety case framework covers different elements that are critical to evaluating the safe development, testing, and operation of autonomous vehicles on public roads. The framework is designed to cover testing with and without vehicle operators. At the same time, it is built to be adaptable so it can be customized for different scenarios and environments. The safety case statement can be adapted to apply to different vehicle platforms, vehicles with operators, vehicles on test tracks, and vehicles on public roads.

Aurora's Safety Case Framework helps evaluate the design and development of the Aurora driver and aligns with the product development roadmap. For each major product milestone, we will examine which claims are relevant and develop corresponding evidence. A claim is an assertion we are making, such as "G3.1 Safety performance metrics are measured, analyzed, and used to monitor safety." Appropriate evidence, which Aurora is actively developing internally, will be tailored to substantiate each individual claim and may include test results, peer reviews, audits, or assessments.

This is just the first release, and Aurora's framework will continue to evolve as we continue to learn and expand our testing operations to new environments and platforms. Aurora is sharing the first 4 levels of the framework because it is important for Aurora's partners, customers, and the public to understand why we are confident in our progress in delivering the Aurora driver. Further development will follow an iterative process, and Aurora will continue to share updates to the framework as it evolves.

Highest level goal

The Aurora Safety Case Framework is built around a top-level claim that “our self-driving vehicles are acceptably safe to operate on public roads.” The entire safety case is used to substantiate this top-level claim and break this claim down into five safety principles or sub-principles.

G1: Proficient

Autonomous vehicles are acceptably safe during normal operation.

It is not safe for automated vehicles to operate on public roads unless appropriate proficiency is achieved. Proficiency includes the design, engineering, and testing required to develop the product. This safety principle contains automated vehicle performance requirements for nominal, non-nominal, and corner case scenarios.

G2: Fail-safe

Autonomous vehicles are acceptably safe in the event of malfunctions and failures.

The fail-safe principle addresses the behavior of an autonomous vehicle in the presence of failures and malfunctions. No system is 100% perfect, and parts sometimes wear out or fail prematurely. The Aurora driver is designed to detect and safely mitigate these failures. This safety principle encompasses all fault detection, mitigation, and notification built into the vehicle.

G3: Continuously improving

All identified potential safety issues that pose unreasonable safety risks are evaluated and addressed with appropriate corrective and preventive actions.

The Continuous Improvement Principle outlines how the concept of continuous improvement is built into the development of the system. Autonomous vehicles are equipped with sensors, and a fleet of autonomous vehicles captures a large amount of data from just one day of operation. We are able to harness the power of this data for continuous improvement. This field data feeds a comprehensive data analysis effort that calculates safety performance indicators and takes into account data collected during design and development. This systematic approach to collecting and analyzing data allows us to discover trends, mean regression, and emergent behaviors. Aurora also takes a proactive approach to continuous improvement, using risk identification techniques to proactively identify risks.

G4: Resilient

Automated driving vehicles are acceptably safe in the event of reasonably foreseeable misuse and unavoidable events.

Autonomous vehicles are designed to operate safely on public roads, but this does not insulate them from malicious actors or unavoidable events. The Resilience Principle demonstrates how the Aurora driver can withstand adverse events and intentional misuse and abuse.

G5: Trustworthy

Autonomous driving companies should be trustworthy.

Aurora's self-driving cars may be proficient, fail-safe, continuously improving, and resilient, but we cannot fully achieve our highest aspirations without the trust of the public and government regulators. The Trustworthy Safety principle addresses how Aurora plans to earn trust through public, government, and stakeholder engagement, safety transparency, safety culture, and external review and consulting activities.

Decomposition of safety principles

The top-level statements are defined based on the security principles covering the scope of security operations, breaking down each security principle using a breadth-first, depth-second approach.

Each safety principle is broken down into a hierarchy of intermediate arguments, context, and strategy. The lowest level statement is ultimately met by evidence provided by our people. This approach allows each safety argument to be traced as a logical decomposition, from broad concepts to specific, tangible evidence supporting the statement.

Security principle decomposition example

Evidence used to support claims comes in two forms – product evidence and process evidence. Product evidence includes deliverables such as technical specifications, test plans, and test results. Process-related evidence shows that product evidence is generated in a systematic manner with sufficient rigor, scrutiny, and independence. This evidence may include informal internal audit reports confirming that established processes are being followed. Both types of evidence are needed to adequately address the claims in the safety case.

Application of the framework

The Safety Case Framework is a tool that Aurora uses to inform the daily activities of hundreds of Aurora employees in the process of developing Aurora drivers.

The Safety Case Framework is designed to adapt to different vehicles, scenarios, and environments. We will use the Safety Case Framework to create a specific safety case, taking care to define its specific context and application in each instance. Think of the framework as a general blueprint for generating a variety of specific safety cases. For example, safety cases are created for specific vehicles and vehicle configurations (truck and passenger car platforms) and for specific operational design domains (e.g., highway). Therefore, there will be multiple separate safety cases covering various configurations, platforms, and operational domains, rather than a single safety case that covers all uses of our autonomous vehicles.