Applied Research

Assessment of an Autonomous Racing Controller: A Case Study From the Indy Autonomous Challenge ’ s Simulation Race

Authors
  • Yangwoo Kim orcid logo (Texas A&M University)
  • Jose Cezares orcid logo (Texas A&M University)
  • Lance Nelson Decker (Texas A&M University)
  • Ivan Damnjanovic (Texas A&M University)

Abstract

The Indy Autonomous Challenge (IAC) was a competition among universities from around the world created to showcase fully autonomous operations under the extreme circumstances encountered in high-performance racing. The pinnacle of the virtual portion of the competition occurred on June 30,2021, during the IAC Simulation Race that consisted of a series of qualifying events followed by a head-to-head race. The objectives of this paper are to present a team’s controller and demonstrate its performance throughout the IAC until the final simulation race. The results presented in this paper were obtained by testing the team’s controller within a simulated environment. The final controller is capable of sustaining speeds of nearly 300 kph (186 mph) with an average speed of 280 kph(174 mph) and a maximum speed of 298 kph (185 mph). The final steering proportional-integral-derivative controller yielded cross-track errors no more than 4.7 meters from the desired waypoint. Analysis of the simulated vehicle’s G-G diagram reveals that the vehicle could sustain operations while experiencing over 2.5 Gs of lateral force as it navigated the turns of the track, and there is evidence to suggest that future work can be performed to tap into the full potential of the tires’ grip capabilities.The results presented in this report are indicative that the race team’s controller can perform safe high-speed operations in the presence of seven additional script-controlled opponents within a simulated environment.

Keywords: simulated car racing, proportional-integral-derivative controller, overtake, case study

How to Cite:

Kim, Y., Cezares, J., Decker, L. N. & Damnjanovic, I., (2023) “Assessment of an Autonomous Racing Controller: A Case Study From the Indy Autonomous Challenge ’ s Simulation Race”, The Journal of Technology, Management, and Applied Engineering 39(2), 1-19. doi: https://doi.org/10.31274/jtmae.15670

Rights: © 2023 The Author(s). This is an open access article published under a Creative Commons Attribution 4.0 International License.

393 Views

101 Downloads

Published on
13 Jul 2023
Peer Reviewed

Introduction

On October 23, 2021, the Indianapolis Motor Speedway (IMS) hosted the Final Race of the Indy Autonomous Challenge (IAC). This competition among universities from around the world was created to showcase the operation of fully autonomous vehicles under the extreme circumstances encountered in high-performance racing. The Texas A&M University Reveille Racing team was formed as a university’s entry into the competition and was among the 28 initial teams representing 40 universities from around the world. The team was composed of undergraduate and graduate students with faculty members and industry partners serving as advisors.

The IAC began in late 2019 and consisted of two parts: a simulation race held on June 30, 2021, and a final (real world) race held on October 23, 2021. Between 2019 and the simulation race, participating teams prepared their racing controllers within a virtual environment using resources provided by Ansys. The simulation race itself served as a means by which teams could measure their performance in terms of balancing competitiveness with safety before transitioning their controllers into the physical world. Because the Reveille Racing team did not participate in the final race due to monetary constraints, the majority of this paper focuses on the team’s performance in the simulation race and in the events leading up to it.

The objective of this study was to present the Reveille Racing team’s controller and demonstrate how its performance throughout the IAC allowed for the team’s third place achievement in the simulation race. The controller was designed to perform safely and efficiently at speeds of up to 313 kph (194 mph) while operating fully autonomously (e.g., full speed and steering control) and receiving information from on-board sensors. The results presented in this paper were obtained by testing the team’s controller within a simulated environment. The results also include the controller’s performance at several benchmark events held throughout the competition.

The contributions of this study are

  • To present a framework that can be used when designing a safe and competitive controller for high-speed racing;

  • To illustrate the performance of a robust proportional-integral-derivative (PID) system for fully autonomous racing in a realistic simulated environment for producing safe yet competitive behavior in a variety of possible racing conditions (e.g., overtaking, emergency braking, and cornering) according to speed, steering wheel angle, and the cross-track error along the track as well as according to the G-G diagram plotting longitudinal and lateral acceleration against one another;

  • To provide the results obtained first-hand from a highly competitive event among universities from around the world using their own unique controllers; and

  • To remark on the challenges of balancing safety and competitiveness by clearly stating the limitations of the proposed controller and how future work can avoid the pitfalls faced by the team.

The remainder of this paper is structured as follows: Section 2 presents a brief review of previous autonomous challenges as well as a literature review pertaining to PID systems and difficulties in designing autonomous driving systems. Section 3 describes the simulation environment provided by Ansys. Section 4 describes the methodology used to create the team’s controller. Section 5 presents the competition and operational results. Finally, Section 6 presents the paper’s conclusion as well as recommendations for similar projects going forward.

Background

Non-racing autonomous challenges

The first notable autonomous driving challenges were hosted by Defense Advanced Research Projects Agency (DARPA) from 2004 through 2007 (DARPA, 2014). Whereas the first two Grand Challenges focused on off-road autonomous driving, DARPA’s 2007 Urban Challenge was the first to be held in an urban setting. Held in Victorville, California, this challenge required 11 participating teams to design their own autonomous vehicles that could navigate through an urban environment containing marked lanes, traffic signs and signals, and other vehicles while obeying the rules of the road (Reinholtz et al., 2007). Following DARPA’s Urban Challenge, a number of organizations began to host their own challenges in Europe (McEachern, 2015), Asia (Hyundai Motor Group, 2014), the United States (SAE International, 2021), and virtually (Shell, 2021).

Autonomous racing challenges

As autonomous driving systems continue to evolve and gain increasing public awareness, numerous autonomous driving competitions have emerged seeking to push autonomous operations to the limit. These competitions require successful navigation of challenging tracks and require autonomous racers to operate at speeds much higher than those found on public roads. Beginning in 2005, Formula Student Germany is an ongoing event that includes the design of autonomous driving systems as part of its evaluation criteria (Formula Student Germany International Design Competition, n.d.). While teams did not race head-to-head, the autonomous performance of each team’s vehicle performance is evaluated through various scenarios and compared to the performance of other teams. Two other competitions, Roborace and Self Racing Cars, both began midway through the 2010s (Yvkoff, 2016;). Self-Racing Cars hosts amateur competitors who have modified their vehicles to navigate autonomously through a racetrack. As with Formula Student Germany, these modified vehicles do not directly compete but instead try to see which vehicle can achieve the lowest lap time. In contrast, Roborace has held a race in which two vehicles directly competed against each other on a real-world track (Tobin, 2019).

Applications of PID controllers for simulated vehicles

Within the industrial automation sector, it has long been estimated that 95% of all operations rely on closed-loop PID controllers (ELPROCUS, n.d.; Åström, 2002; Tejado et al., 2019). The widespread use of these controllers is largely attributed to their straightforward design and robustness that has enabled their usage as the technology used in the industry sector has changed over the past century (Åström, 2002). Concerning vehicles, numerous studies have implemented their own PID controllers for unmanned aerial vehicles (Salih et al., 2010; Kada & Ghazzawi, 2011; Müller et al., 2019, Najm & Ibraheem, 2019), simulated vehicles (Marino et al., 2011; Azar et al., 2019; Baskaran et al., 2019; El Hajjami et al., 2019), and real-world autonomous vehicle systems (Hoffman et al., 2007; Zhao et al, 2012).

One application of interest has been the use of PID controllers for simulated racing vehicles. As noted by Chen et al. (2021), the advantage of PID controllers is their ability to reduce computational load; this particular characteristic is significant for vehicles operating at high speeds. Recent studies have implemented customized PID controllers for high-speed longitudinal control systems (Gomes et al., 2017; Wang et al, 2018; Chen et al., 2020, 2021), lateral (steering) control systems (Armagan & Kumbasar, 2017; Zhang et al., 2018), and at least one previous study by Wang et al. (2021) has utilized two PID controllers for both longitudinal and lateral control. Generally, the results of these works found that PID controllers provided favorable results for simulated racing vehicles. In particular, PID controllers that were integrated with model predictive controllers or fuzzy logic controllers tended to perform the best in terms of balancing lateral and longitudinal vehicle states (Chen et al., 2021) and improving handling stability (Zhang et al., 2018).

Difficulties of designing autonomous driving systems

Designers face numerous challenges when designing autonomous driving systems. For real-world, public road applications, it is imperative to design a system that can make decisions on a level that is at least equivalent to an attentive and responsible driver. Past non-racing challenges assess systems by placing autonomous vehicles in mock environments that contain elements that a human driver would encounter on a typical commute (Bacha et al., 2008; Kammel et al., 2009; Burnett et al., 2019). Such elements include traffic signs and signals, pavement markings, intersections, and other vehicles (static and dynamic). Furthermore, designers must consider the rate at which sensors receive information about the vehicle’s environment while also accounting for any potential software errors that can contribute to a failure of operations.

The IAC and other racing challenges introduce the additional challenge of having an autonomous driving system operate at speeds well above the threshold for most human drivers. Designers must ensure that the selected sensors are capable of reliably collecting accurate information in a timely manner (Wischnewski et al., 2019). Similarly, the selected computational software should be capable of performing all required calculations as quickly as possible, and the software should be streamlined to reduce computational time while remaining flexible enough to promptly develop accommodations for unexpected changes to the vehicle’s surroundings (Andersen et al., 2020; Nekkah et al., 2020).

Materials

Simulation software

The simulation competition was conducted using the Ansys VRXPERIENCE (VRX) Driving Simulator. VRX is a physics-based simulator powered by SCANeR Studio that allows users to modify various aspects of the virtual environment, including the terrain, vehicles, and custom scenarios. The program’s user interface grants the user the ease of editing vehicle configurations as well as creating unique testing scenarios with a built-in pseudo-script editor. Furthermore, VRX can also be run using external scripts written in Python through the Data Distribution Service. This environment facilitated the testing of each team’s unique control systems in a manner that avoided damage to real-world vehicles or the IMS.

Vehicle model

The Dallara IL-15 is the vehicle model that was used by all teams for the simulation competition (see Figure 1). The files for this model were provided to each team since the real-world competition would be using this very same model of racing vehicle. Simulated vehicles were equipped with sensors (cameras, laser imaging detection and ranging [LiDAR], radar, and global navigation satellite system [GNSS]), and each vehicle could be controlled using VRX’s own default controller or using a custom controller. Information regarding the vehicle (e.g., lateral and longitudinal speed and acceleration, tire grip and forces, etc.) could be monitored directly via either VRX or RTI Connext.

Figure 1.
Figure 1.

Illustration of (a) Dallara IL-15 model and (b) IMS recreated within VRX. IMS, Indianapolis Motor Speedway; VRX, VRXPERIENCE.

Racetrack model

The simulated environment consisted of a replication of the IMS. The simulation tracks latitude, longitude, and elevation values that match up with real-world values that were collected onsite by Ansys. Identical track values help ensure the realism of tests and the simulation race. A view of the racetrack modelled in VRX is shown in Figure 1.

Methodology

Test scenarios and control system architecture

In order to ensure a successful performance within the final simulation race, the team’s controller needed to meet two key requirements. Firstly, high speed and low lap times are desirable in order to remain competitive with other racers. Second, the ego vehicle should have an awareness of its environment and should be able to avoid maneuvers that could result in a collision or veering off the track.

In order to design a control system that could meet these two requirements, two main scenarios were designed to test different components of the team’s custom vehicle controller. The first scenario, the Single Ego, contained a single simulated vehicle used to test and evaluate longitudinal and lateral controls in a basic environment. The second scenario, the Multi-Vehicle, contained four vehicles controlled by the custom controller racing against each other in order to test and evaluate obstacle avoidance and path planning portions of the main controller.

To this end, the team’s controller would be required to interpret sensor input data from and transmit commands to the VRX environment. This was accomplished by incorporating five distinct modules into the controller. The first module, Localization, was responsible for determining the global and local positioning of the vehicle, and it was dependent on sensor input data. The second module, Perception, also uses sensor input data to determine the road boundaries as well as for object detection. The next module, Path Planning, was dependent on the first two modules for determining the optimal race line that the ego vehicle will try to adhere to. The fourth module, Decision-making, was dependent on the previous three modules for obstacle avoidance and obstacle movement prediction. Finally, the Longitudinal and Lateral Control module takes the outputs of the four prior modules to determine the required throttle (or braking) and steering angle that will be transmitted to the VRX environment.

Because the longitudinal and lateral control module contains three sub-controllers, it should be noted that the use of the term “controller” from this point on refers to the overarching main control architecture that is presented in Figure 2. The sub-controllers that exist within this main controller are expanded upon in a later section of this paper.

Figure 2.
Figure 2.

Overarching main controller architecture. GNSS, global navigation satellite system; VRX, VRXPERIENCE.

Due to the nature of the IAC’s Simulation Race, the Reveille Racing team did not need to spend time and resources on selecting sensors or computational hardware because these components were predetermined and uniform for all teams. Thus, the Reveille Racing team placed all of its focus on designing effective and robust software modules that could effectively handle a multitude of potential scenarios. To this end, the team’s goal was to develop a control architecture that could operate competitively at high speeds while simultaneously prioritizing safety.

In summary, the compartmentalization of the main control architecture was designed to allow the vehicle to autonomously assess its environment, evaluate its choices of action, and ultimately select the optimal action based on its assessments. An overview of the overarching main control architecture is provided in Figure 2, a list of the longitudinal and lateral controllers within the architecture is provided in Table 1, and the remaining sections discuss how the team’s control architecture and each longitudinal and lateral controller were designed in order to safely operate at high operating speeds.

Table 1.

List of longitudinal and lateral controllers in proposed architecture.

Controller Type Direction
Throttle Proportional-Derivative-Integral (PID) Longitudinal
Brake Proportional-Derivative-Integral (PID) Longitudinal
Steering Angle Proportional-Derivative-Integral (PID) Lateral

Sensors

Each simulated vehicle is equipped with sensors that would be identical to the ones that would be included in the physical vehicle. The data from each sensor could be acquired and read using either a custom script or RTI Connext. Data from each simulated sensor was pre-processed by VRX; consequently, this eliminated the need to create custom code to interpret objects detected by each sensor. A summary of the sensors and their specifications are included in Table 2.

Table 2.

Overview of simulated sensor specifications.

Sensor Name Sensor Type Location FOV Range
MakO G319-C Camera 2 Front 34H × 24V N/A
MakO G319-C Camera 1 Left, 1 Right, 1 Rear Left, 1 Rear Right 102.8H × 77V N/A
Aptive ESR 2.5 Radar Front/Rear 90H × 4.4V (Mid-Range)
20H × 4.4V (Long-Range)
60 m (Mid-Range)
175 m (Long-Range)
Aptive MRR Radar Left/Right 90H × 90V 40 m (Mid-Range)
160 m (Long-Range)
ibeo ScaLa LiDAR 1 Front, 1 Left, 1 Right 120H × 30V <2 m (Minimum)
250 m (10% Reflectivity)
500 m (Maximum)
PwrPak7-E1 GNSS Front, Left, Right, Top
  • FOV, field of view; H, horizontal; LiDAR, laser imaging detection and ranging; GNSS, global navigation satellite system; N/A, not applicable; V, vertical.

Perception

Camera, LiDAR, and radar sensors were used for perception. Each sensor provided pre-processed data, thus reducing the effort required by each team to interpret objects and track boundaries. After fusing the data from these sensors, the location of the track boundaries and all objects around the ego vehicle could be determined by the team’s control system.

For this competition, lane and object detection is handled by the Reveille Racing team’s perception module. VRX provides third-degree polynomial coefficients that can be used to identify the track boundaries that will be used for path planning. For objects (e.g., other vehicles), the camera sensor provides X and Y local coordinates originating from the point of installation on the simulated ego vehicle, and these local coordinates are then converted into VRX’s world coordinates as part of the localization process.

Localization

The coordinates of pre-processed sensor data provided by VRX have different origin points; each sensor is installed at a different location on the vehicle and with its own unique heading. In order to obtain accurate track boundaries and object positions, the local coordinates for each sensor had to be converted to match with the simulated ego vehicle’s VRX world coordinates. This conversion was performed by utilizing the simulated vehicle’s known position and the sensors’ local properties provided by VRX. By doing this, the local coordinates of detected boundaries and objects could be placed into a common world coordinate system with the ego vehicle.

Path planning

In order to be competitive with other teams, it was important to develop a controller that could determine potential race lines to follow. Furthermore, the designed control system would need to be able to handle areas where the main track would split off to the pit lane; these locations are referred to as blind spots because the camera sensors had difficulty correctly identifying the track boundaries at these locations. Figure 3 provides an overview of the path planning module.

Figure 3.
Figure 3.

Overview of the path planning module.

The path planner takes in the ego vehicle’s location and track boundaries that were previously calculated with the localization and perception modules. The output of the path planner is a set of 10 potential waypoints or race lines. Out of those 10, the left- and rightmost candidates are immediately removed as potential race lines due to their closeness to the left and right boundaries, respectively. The decision to omit these race lines stems from the fact that following these two lines would result in the ego vehicle driving onto either the grass or into a wall. An illustration of the candidate race lines, as well as the prohibited race lines, is presented in Figure 4.

Figure 4.
Figure 4.

Script generated candidate and prohibited race lines.

Decision-making

As the penultimate component in the Reveille Racing team’s control system, the decision-making module is tasked with processing the outputs of all the path planning, localization, and perception modules in order to determine the best course of action. The proposed module is designed to perform competitively while simultaneously maintaining safety as the highest priority. An overview of the decision-making process is shown in Figure 5.

Figure 5.
Figure 5.

Overview of the decision-making module.

Once path-planning determines a set of 10 candidate race lines, the ego vehicle’s position is determined via localization, and the environment is assessed via perception (Step 1); the decision-making module considers the ego vehicle’s location on the track and determines an initial optimal target race line and target speed Vmax (Steps 2–4). This part of the process is illustrated in Figure 5 where the target race line initially depends on whether the ego vehicle is in a straightaway or curved section. Next, the module considers whether or not any objects have been detected (Step 5). In the case in which no objects have been detected, the module outputs the final target race line and target speed (Steps 8–9).

On the other hand, if an object is detected, then the module next considers the object’s location in relation to the ego vehicle’s own location (Step 6). In the event that the detected object is not directly ahead of the ego but is located on the target race line and is located relatively nearby, then the ego vehicle adjusts the target race line to prevent colliding with the object (Step 6 ➔ Step 7b). If the object is on the target race line but is considerably far from the ego, then the ego vehicle makes no changes to its target race line (Step 6 ➔ Step 8). In either case, no changes are made to the target speed.

If the detected object is directly ahead of the ego and the ego vehicle is already on the target waypoint, then the module considers the object’s distance from the ego. If the object is considerably far from the ego and if the object’s speed Vobst is greater than the ego vehicle’s speed Vactual, then the ego vehicle is not prompted to adjust its race line (Step 6 ➔ Step 8). If the object is instead moving slower than the ego vehicle, or if the object is determined to be too close, then the decision-maker must evaluate its options for obstacle avoidance (Step 7a ➔ Step 8).

To further elaborate on obstacle avoidance (Step 7a), there are several checks that need to be performed in order to assess all possible maneuvers at any given instance. First, the controller determines the ego vehicle’s current race line. Next, the positions of all detected obstacles are processed, and the status of each race line is determined; each race line can be considered to be available or blocked depending on the position of detected obstacles or the distance to the track boundaries. Following this, the controller checks the heading of each obstacle, and all race lines that the obstacle appears to be driving toward are marked as undesirable to prevent the ego vehicle from a potential collision. After all potential race lines have been evaluated, the controller determines whether there are any suitable race lines that can be used to perform a takeover maneuver. In the event that a takeover is not feasible, the controller will prompt the ego vehicle to slow down to prevent a collision. If either the ego vehicle has performed a takeover and does not detect any other obstacles or it does not detect any obstacles to begin with, the controller will direct the ego vehicle to make its way around the track using the optimal race line.

Longitudinal and lateral controls

Three separate PID controllers designed to provide throttle, brake, and steering inputs control the ego vehicle’s longitudinal and lateral motions. Due to robustness and ease of implementation, PID controllers were selected to reduce the complexity of the control architecture while simultaneously providing a simple means for fine-tuning. A simple block diagram of the PID controller structure is provided in Figure 6 whereas an overview of how longitudinal and lateral control was handled by the Reveille Racing team’s controller is provided in Figure 7.

Figure 6.
Figure 6.

Block diagram of PID controller structure. PID, proportional-integral-derivative.

Figure 7.
Figure 7.

Longitudinal and lateral control module. PID, proportional-integral-derivative; VRX, VRXPERIENCE.

In order to obtain a command throttle or braking value, the speed’s PID controller requires both the target speed, Vtarget, and the ego vehicle’s current or actual speed, Vactual. Whereas Vactual can easily be obtained directly from VRX, Vtarget is dependent on whether the decision-making module has determined the need to slow down or to continue at the maximum speed, Vmax. After assessing the simulated environment information (e.g., speed, steering wheel angle, cross-track error, GNSS positioning, LiDAR points), the decision-making module determines the need to slow down. More specifically, the action of slowing down was performed either by reducing throttle input or by braking. In instances in which braking was deemed unnecessary, then the Vtarget was set to 90% of the Vactual value; this value was selected based on empirical observations made while testing the controller. Although reducing Vtarget by more than 10% offered a greater safety buffer between the ego vehicle and nearby obstacles, this reduction to 90% of the Vactual resulted in the ego vehicle falling farther behind and negatively affected the observed lap times. This 10% reduction in speed offered a practical compromise that let the ego vehicle create a buffer between itself and the target while simultaneously preventing the ego vehicle from increasing its lap times. An overview of this framework is provided in Figure 8.

Figure 8.
Figure 8.

Process for determining target speed. VRX, VRXPERIENCE.

Once both the Vactual and Vtarget are known, the controller selects Throttle if Vactual < Vtarget or the Brake PID if Vactual > Vtarget. The PID controller for the throttle and brake inputs both use the following models:

acommand=KPe(t)+KIe(t)dt+KDddte(t)
(1)

bcommand=KPe(t)+KIe(t)dt+KDddte(t)
(2)

where the commanded output will be the value of acommand (throttle input command) or the value of bcommand (brake input command); KP, KD, and KI are proportional, derivative, and integral adjustment factors, respectively; and e(t) is the difference between Vactual and Vtarget. The values for the PID constants are provided in Table 2.

A PID controller is utilized for determining a steering angle δcommand and is presented in the following equation:

δcommand=KPeCT(t)+KIeCT(t)dt+KDddteCT(t)
(3)

where eCT(t) is the cross-track error between the ego vehicle’s projected position and its desired waypoint’s position. The eCT(t) can be found by the following equation:

eCT(t)=Lasin(απ180°)
(4)

where La is the lookahead distance to the desired waypoint. Figure 9 illustrates the logic behind the eCT(t) calculation.

Figure 9.
Figure 9.

Cross-track error measurement.

The final variables for Equations 13 were fine-tuned using a modified Ziegler-Nichols procedure (Ziegler & Nichols, 1942). Initially, KP, KI, and KD were set to 0. KP was increased until a steady oscillation was obtained; at this point, the critical gain KU and peak-to-peak oscillation period TU can both be obtained. Several simulation trials were run, and each variable was further tuned based on the vehicle’s performance. The final PID constants for Equations 13 are provided in Table 3.

Table 3.

Throttle, brake, and steering PID parameters.

Action KU KP TU KD KI
Throttle (a) 18 10Ku 1.6 15KuTu 12Ku/Tu
Brake (b) 6.0 0.5Ku 1.25 0.25KuTu 1.2Ku/Tu
Steering (δcommand) 6.0 0.6Ku 1.6 0.25KuTu 1.2Ku/Tu
  • PID, proportional-integral-derivative.

Results and Lessons Learned

Ansys hackathons

Ansys hosted three hackathon events in order for teams to assess their progress against one another before the simulation race. The first hackathon served as an introduction to the VRX program that all teams would be using to test their controllers. Competitive elements were not present for this particular event. This early event made it clear that there would be a significant amount of work to perform in the nine-month span between the first hackathon and the final simulation race.

The second hackathon required all teams to test their controllers to complete two tasks. The first task was simply to complete a lap as quickly as possible without exceeding track boundaries or colliding with the walls. This required teams to design controllers that could compute and run along an optimal race line. The second task was to avoid colliding with a reckless VRX-controlled vehicle that would suddenly swerve into the path of each team’s vehicle. Each team was scored based on their performance while completing the two tasks; the Reveille Racing team finished in 9th place out of 19 competitors. At this point, it was realized that the Reveille Racing team’s determined optimal race line needed to be updated because the selected race line failed to perform as well as the lines followed by other teams.

The third hackathon divided all of the remaining teams into groups of two. Each group would compete head-to-head in two sets of races with the objective of safely completing five laps as quickly as possible. While the Reveille Racing team was able to outperform its competitor in the first race, the team’s controller received a penalty due to a collision with the competing team in the second race. During the second race, the opposing team’s vehicle swerved frequently on its race line, and the race team’s vehicle drove too closely when overtaking the other team on a straight section. Ultimately, the race team finished in 12th place out of 18 competitors.

Final simulation challenge

All teams submitted their final controllers on or before May 19, 2021. At that point, Ansys spent the next month running the simulations using the submitted controllers and obtained the rankings for each team. The final challenge consisted of three parts: a qualifying lap, two head-to-head semi-final races between the qualifying teams, and a head-to-head race among the top seven teams based on the results from the semi-finals. On June 30, 2021, Ansys announced the results of the final simulation challenge.

For the qualifying lap portion of the challenge, each team was required to complete one full lap (approximately 2.5 miles) successfully. No other vehicles were present during each team’s qualifying lap. Penalties were awarded to teams that collided with walls or exceeded acceptable track boundaries. The Reveille Racing team placed 13th out of 16 competing teams with a lap time of 52.152 seconds, less than half a second behind the fastest lap time of 51.848 seconds.

The two semi-final races both hosted five teams as they competed for 10 laps (approximately 25 miles). Teams that collided with walls or other vehicles due to negligent maneuvers would be disqualified. During the first semi-final race, the race team began from the fifth-place position and performed a successful early overtake of the fourth-place team, maintaining this position for the remainder of the race. The race team finished in fourth place with a time of 508.58 seconds, 3.592 seconds slower than the first-place team.

For the final race, the race team began from the sixth-place position and performed a successful overtake in the last half of the first lap and can be seen from the opponent’s perspective in Figure 10. Throughout the remainder of the race, three of the seven teams would become disqualified due to collisions; one of these collisions occurred when another team’s vehicle collided with the Reveille Racing team’s car. Due to the circumstances of the collision, the Reveille Racing team was found not to be at fault and was allowed to continue the race. Ultimately, the race team completed the final 10 laps and finished in third place with a time of 458.74 seconds, 3.816 seconds slower than the first-place team.

Figure 10.
Figure 10.

Opponent’s perspective of the racing team’s overtake maneuver (courtesy of Ansys).

Operational performance of control system

By the end of the third and final hackathon, it became clear that the race team’s control system would require substantial updates to its various components, especially the scripts pertaining to obstacle avoidance. In particular, obstacle avoidance at this point was more focused on avoiding objects on straightaways that were not making significant movements to the left or right; this became problematic when dealing with vehicles that behaved erratically. It was soon realized that the controller should behave more conservatively when deciding how to avoid obstacles and how much space to keep between itself and other vehicles. These changes are reflected in the controller architecture that was presented in the previous section of this paper. Performance data from two laps was collected directly from VRX. The performance along the length of the two laps was assessed according to the following metrics:

  • Speed: The speed of the vehicle (kph)

  • Steering wheel angle: The angle of the vehicle’s steering wheel (°)

  • Cross-track error: The error between the vehicle’s current position and the target race line (m)

  • G-G diagram: The performance envelope of the vehicle represented by the plotting of its longitudinal acceleration against its lateral acceleration (Gs)

To further assess the vehicle’s performance, the first three metrics are measured against the bank angle (°) of the track. It should be noted that the bank angle of the track is 0° in straight sections and negative at the corners. Furthermore, positive and negative bank angles correspond to right and left turns, respectively. Thus, because the track exclusively contains left turns, all bank angles at corners are negative.

The final speed performance for a single vehicle completing two laps is visualized in Figure 11.

Figure 11.
Figure 11.

Speed performance for a single vehicle over two laps.

Figure 11 demonstrates that the final controller design is capable of sustaining speeds of nearly 300 kph (186 mph) with an average speed of 280 kph (174 mph) and a maximum speed of 298 kph (185 mph). In comparison, the top qualifier at the 2019 Indy Lights race held at the IMS had a maximum speed of 320 kph (198 mph) and an average speed of 312 kph (194 mph). The final controller’s performance in contrast to the top qualifying team indicates that there is still substantial room for improvement in terms of speed. This difference between performances is attributed to the final controller being more conservatively designed in contrast to other teams. More specifically, other teams placed more emphasis on following an optimal race line around the track as well as achieving more aggressive speeds when entering corners. The final lateral controller proposed in this paper instead focused on following a more conservative race line and opted for less aggressive speed choice at curves; this design choice is due to the aforementioned overtaking incident during Hackathon 3.

The system’s lateral control module was also evaluated over the distance it took a single vehicle to complete two laps. The steering wheel angle and the cross-track error produced by the steering PID controller are presented in Figure 12.

Figure 12.
Figure 12.

(a) Steering wheel angle and (b) cross-track error for a single vehicle over two laps.

The final lateral control module proved to be effective at yielding cross-track error of no more than 4.7 meters from the desired waypoint. The largest cross-track error values were produced at points immediately after the ego vehicle exits turns 2 and 4 with a rapid correction and stabilization once the vehicle is within the straight sections of the track. As observed with the speed performance, the cross-track error plot indicates that there exists room to improve the vehicle’s lateral module. Deficiencies in the lateral module’s performance are due to the nature of the module being similar to a pure pursuit controller. With the exception of the command steering angle δcommand, the remainder of the lateral controller operates similarly to a simple pure pursuit controller (e.g., finding a desired point, calculating the angle to the desired point, and steering toward the desired point). For this reason, the lateral control module is vulnerable to oversteering at the corners of the track. Lastly, comparing the steering wheel angle with the cross-track error over the same distance reveals that the controller was more effective at steering into corners in contrast to steering out. This is supported by the fact that steer-in behavior corresponded to lower cross-track error when entering the corner versus the more abrupt steer-out behavior contributing to larger cross-track error at corner exit points.

In order to evaluate the final controller’s performance with respect to the theoretical limits of the vehicle, a G-G diagram was produced using the longitudinal and lateral acceleration values collected from the two laps. G-G diagrams are useful for visualizing the operational stability of a vehicle. More specifically, G-G diagrams illustrate how grip on the road is distributed. Generally, values within the envelope (boundaries of the diagram) are considered to be indicative of stable operation, values on the envelope are indicative of operations utilizing all available grip, and values outside of the envelope indicate instances of unstable conditions in which the tires exceed available grip. G-G diagrams can be further divided into regimes based on the actions being performed:

  • -

    Pure acceleration: All grip dedicated to longitudinal acceleration

  • -

    Pure braking: All grip dedicated to longitudinal braking

  • -

    Pure right cornering: All grip dedicated to turning right

  • -

    Pure left cornering: All grip dedicated to turning left

  • -

    Combined acceleration out of a left turn (Quadrant I): Grip divided between acceleration and turning out of left

  • -

    Combined acceleration out of a right turn (Quadrant II): Grip divided between acceleration and turning out of right

  • -

    Trail braking going into right corner (Quadrant III): Grip divided between braking and turning right

  • -

    Trail braking going into left corner (Quadrant IV): Grip divided between braking and turning left

The G-G diagram generated from the collected data is shown in Figure 13.

Figure 13.
Figure 13.

G-G diagram of a single vehicle over two laps (format courtesy of OptimumG, 2019).

As indicated by the G-G diagram, the vehicle would experience over 2.5 Gs of lateral force while maneuvering the left turns of the track. Instances of Gs greater than 1 were experienced as the vehicle accelerated from its rolling start of 100 kph and most instances of pure longitudinal acceleration sustained values below 0.5 Gs. In the absence of right turns, lateral acceleration values that fall into the threshold of right turn cornering are instances in which the vehicle attempted to move to the right outer bound of the track. The absence of points in the pure braking regime was attributed to the decision-making process used by the vehicle. In particular, in the absence of any emergency situations, the vehicle will avoid braking in order to reduce speed loss and will instead slow down by reducing input to the throttle. Because the results in Figure 13 were obtained from a single-vehicle simulation, the vehicle did not encounter other moving vehicles that would need to be avoided by braking. In particular, data from the final race could not be accessed due to the nature of the simulation event being primarily handled by Ansys.

The G-G diagram supports the earlier claims that both the longitudinal and lateral control schemes could be improved. Specifically, the current controller does not make full use of the tires’ grip when attempting to accelerate out of a turn as indicated by the scarcity of points located in Quadrant I. This is indicative of less-than-optimal performance of the vehicle when attempting to re-enter the straight sections of the racetrack, which could be remedied by adjusting the controller to behave more aggressively when selecting speed at curve exit points. Furthermore, the lack of any points on the envelope’s boundaries indicates that some available grip goes unused. For instance, lack of points on the boundary within the pure left cornering regime is indicative that our vehicle could have operated safely at higher speeds while cornering but did not do so due to the controller’s conservative nature. Future work should attempt to tap into the unused grip to maximize the efficiency of the controller.

Despite not operating on its performance boundaries, the controller demonstrated safe high-speed operations in the presence of seven additional script-controlled opponents. Its conservative actions allowed for the simulated vehicle to safely navigate through curves with the aforementioned force of over 2.5 Gs while never reducing to less than 282 kph (175 mph). Figure 14 provides a still image taken during a test run to evaluate the controller’s performance.

Figure 14.
Figure 14.

Eight script-controlled vehicles safely operating at high speeds while entering a corner.

Conclusion and Recommendations

The Reveille Racing team’s performance in the IAC Simulation Challenge demonstrates that it is effective at competing at high speeds while also being able to avoid collisions with other vehicles. During the final simulation race, the Reveille Racing team’s controller displayed that it could effectively follow a near-optimal race line while also being able to keep a safe distance from other vehicles and perform safe overtaking maneuvers at high speeds. The architecture of the Reveille Racing team’s controller is robust yet flexible enough to adapt to the vehicle’s perceived environment. Whereas this is notable for the fact that the work was done in approximately nine months, it was advantageous that the information provided by the simulated sensors was pre-processed.

The proposed controller is not without its limitations. The results presented in this paper are indicative that the controller’s longitudinal and lateral scripts can continue to be improved to fully push the vehicle to its operational limits. More specifically, the controller had room to be more aggressive with regard to speed choice and race line selection. Additionally, the controller’s lateral performance required further refinement to enhance operations at corner exit points by improving both the speed choice and the rate at which the steering wheel is adjusted at said points.

The IAC Simulation Challenge served as a preview for the high-speed autonomous vehicle race that took place in October 2021. The outcome of the simulation race and the performance of teams that participated in the real-world final race serve as indicators that teams that can learn how to balance the desire for the highest speed with the self-awareness required for safe operations for the ability to operate both safely and competitively. However, the two crashes that occurred in the real-world race denote that room exists for all parties to improve upon their current controllers. In particular, most teams that experienced collisions in the final race mentioned that GPS signals were blocked as a result of system errors. This emphasizes the need for a racing control system that can depend on its cognitive sensors whenever external systems (e.g., GNSS) cannot be accessed. In other words, racing vehicles must not heavily depend on external systems, and designers should place priority on improving the operational capacity of the vehicle itself.

The IAC in general has helped stimulate high-performance autonomous vehicle research by involving educational institutions from around the world. Similar future competitions can ensure that there is a steady development for autonomous high-performance systems. If this competition is held again, the advancement of autonomous control systems could continue to be improved for use in non-competitive, public road settings as has been done after past competitions held at the IMS (Leporati, 2021).

References

Andresen, L., Brandemuehl, A., Hönger, A., Kuan, B., Vödisch, N., Blum, H., Reijgwart, V., Bernreiter, L., Schaupp, L., Chung, J. J., Bürki, M., Oswald, M. R., Siegwart, R., & Gawel, A. (2020). Accurate mapping and planning for autonomous racing. 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4743–4749.

Armagan, E., & Kumbasar, T. (2017, November 30). A fuzzy logic based autonomous vehicle control system design in the TORCS environment [conference proceedings]. In 2017 10th International Conference on Electrical and Electronics Engineering (ELECO), Bursa, Turkey. IEEE. pp. 737–741.

Åström, K. J. (2002). Control system design lecture notes for ME 155A. Department of Mechanical and Environmental Engineering University of California Santa Barbara, California. pp. 216.

Azar, A. T., Ammar, H. H., Ibrahim, Z. F., Ibrahim, H. A., Mohamed, N. A., & Taha, M. A. (2019). Implementation of PID controller with PSO tuning for autonomous vehicle. In International Conference on Advanced Intelligent Systems and Informatics (pp. 288–299). Springer, Cham.

Bacha, A., Bauman, C., Faruque, R., Fleming, M., Terwelp, C., Reinholtz, C., Hong, D., Wicks, A., Alberi, T., Anderson, D., Cacciola, S., Currier, P., Dalton, A., Farmer, J., Hurdus, J., Kimmel, S., King, P., Taylor, A., Van Covern, D., & Webster, M. (2008). Odin: Team VictorTango’s entry in the DARPA Urban Challenge. Journal of Field Robotics, 25(8), 467–492.

Baskaran, A., Talebpour, A., & Bhattacharyya, S. (2019, October). End-to-end drive by-wire PID lateral control of an autonomous vehicle. In Proceedings of the Future Technologies Conference. Springer, Cham. pp. 365–376.

Burnett, K., Schimpe, A., Samavi, S., Gridseth, M., Liu, C. W., Li, Q., Kroeze, Z., & Schoellig, A. P. (2019). Building a winning self-driving car in six months. In 2019 International Conference on Robotics and Automation (ICRA). pp. 9583–9589.

Chen, S., Chen, H., Pletta, A., & Negrut, D. (2021, August 17). Coupled lateral and longitudinal control for trajectory tracking, lateral stability, and rollover prevention of high-speed automated vehicles using minimum-time model predictive control [conference proceedings]. In International Design Engineering Technical Conferences and Computers and Information in Engineering Conference [virtual/online]. American Society of Mechanical Engineers. Vol.  85468, p. V009T09A024.

Chen, S. P., Xiong, G. M., Chen, H. Y., & Negrut, D. (2020). MPC-based path tracking with PID speed control for high-speed autonomous vehicles considering time-optimal travel. Journal of Central South University, 27(12), 3702–3720.

DARPA (2014, March 13). The DARPA Grand Challenge: Ten years later. Defense Advanced Research Projects Agency. https://www.darpa.mil/news-events/2014-03-13

El Hajjami, L., Mellouli, E. M., & Berrada, M. (2019, October). Optimal PID control of an autonomous vehicle using Butterfly Optimization Algorithm BOA. In Proceedings of the 4th International Conference on Big Data and Internet of Things. pp. 1–5.

ELPROCUS (n.d.). What is a PID controller: Working & its applications. https://www.elprocus.com/the-working-of-a-pid-controller/

Formula Student Germany International Design Competition. (n.d.). What is the Formula Student Germany competition? https://www.formulastudent.de/about/concept/

Gomes, S., Dias, J., & Martinho, C. (2017, September). Iterative parallel sampling RRT for racing car simulation. In EPIA Conference on Artificial Intelligence. Springer, Cham. pp. 111–122.

Hoffmann, G. M., Tomlin, C. J., Montemerlo, M., & Thrun, S. (2007, July 9). Autonomous automobile trajectory tracking for off-road driving: Controller design, experimental validation and racing [conference proceedings]. In 2007 American Control Conference, New York, New York. IEEE. pp. 2296–2301.

Hyundai Motor Group (2014). HMG to host autonomous vehicle contest. Hyundai Motor Group Newsroom. news.hyundaimotorgroup.com/MediaCenter/News/Press-Releases/HMG-autonomous-vehicle-contest-141002

Kada, B., & Ghazzawi, Y. (2011, October 19). Robust PID controller design for an UAV flight control system [conference proceedings]. In Proceedings of the World Congress on Engineering and Computer Science (WCECS), San Francisco, California. Vol.  2, No. 1–6, pp. 1–6.

Kammel, S., Ziegler, J., Pitzer, B., Werling, M., Gindele, T., Jagszent, D., Schröder, J., Thuy, M., Goebl, M., von Hundelhausen, F., Pink, O., Frese, C., & Stiller, C. (2009). Team AnnieWAY’s autonomous system for the DARPA Urban Challenge 2007. Journal of Field Robotics, 25(9), 615–639.

Leporati, G. (2021, July 7). No driver? No problem—this is the Indy Autonomous Challenge. Ars Technica. https://arstechnica.com/cars/2021/07/a-science-fair-or-the-future-of-racing-the-indy-autonomous-challenge/

Marino, R., Scalzi, S., & Netto, M. (2011). Nested PID steering control for lane keeping in autonomous vehicles. Control Engineering Practice, 19(12), 1459–1467.

McEachern, S. (2015, March 12). Audi to host first ever autonomous driving cup for students. Motrolix.

Müller, M., Li, G., Casser, V., Smith, N., Michels, D. L., & Ghanem, B. (2019, June). Learning a controller fusion network by online trajectory filtering for vision-based UAV racing [conference proceedings]. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, California.

Najm, A. A., & Ibraheem, I. K. (2019). Nonlinear PID controller design for a 6-DOF UAV quadrotor system. Engineering Science and Technology, an International Journal, 22(4), 1087–1097.

Nekkah, S., Janus, J., Boxheimer, M., Ohnemus, L., Hirsch, S., Schmidt, B., Liu, Y., Borbély, D., Keck, F., Bachmann, K., & Bleszynski, L. (2020). The autonomous racing software stack of the KIT19d [preprint]. arXiv. https://doi.org/10.48550/arXiv.2010.02828

OptimumG. (2019). G-G diagram. https://optimumg.com/wp-content/uploads/2019/09/RCE-Assymetrical-Setup-Part1.pdf

Reinholtz, C., Alberi, T., Anderson, D., Bacha, A., Bauman, C., Cacciola, S., Currier, P., Dalton, A., Farmer, J., Faruque, R., Fleming, M., Frash, S., Gothing, G., Hurdus, J., Kimmel, S., Sharkey, C., Taylor, A., Terwelp, C., Van Covern, D., Webster, M., & Wicks., A. (2007). DARPA Urban Challenge Technical Paper.

SAE International. (2021). AutoDrive Challenge. https://www.sae.org/attend/student-events/autodrive-challenge/

Salih, A. L., Moghavvemi, M., Mohamed, H. A., & Gaeid, K. S. (2010). Flight PID controller design for a UAV quadrotor. Scientific Research and Essays, 5(23), 3660–3667.

Shell. (2021). Autonomous Programming Challenge. Shell Eco-Marathon. ecomarathon.swri.org/

Tejado, I., Vinagre, B. M., Traver, J. E., Prieto-Arranz, J., & Nuevo-Gallardo, C. (2019). Back to basics: Meaning of the parameters of fractional order PID controllers. mathematics, 7(6), 530.

Tobin, D. (2019). First Roborace driverless car race held in Spain. MotorSport. https://www.motorsportmagazine.com/articles/sports-cars/first-roborace-driverless-car-race-held-spain

Wang, J., Cheng, Y., & Yu, L. (2021). Racing driver modeling based on driving behavior. In International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (Vol.  85369, p. V001T01A011). American Society of Mechanical Engineers.

Wang, J., Li, W., Li, J., Liu, Y., Song, B., & Gao, H. (2018). Modeling a driver’s directional and longitudinal speed control based on racing track features. Shock and Vibration, 2018, 1–12.

Wischnewski, A., Stahl, T., Betz, J., & Lohmann, B. (2019). Vehicle dynamics state estimation and localization for high performance race cars [conference proceedings]. In 10th IFAC Symposium on Intelligent Autonomous Vehicles, Gdańsk, Poland. pp. 154–161.

Yvkoff, L. (2016). An autonomous racing series kicks off in California. Forbes. https://www.forbes.com/sites/lianeyvkoff/2016/05/28/an-autonomous-racing-series-kicks-off-in-california/?sh=6c6405441c8a

Zhang, S., Yu, M., Lu, Y., & Fan, Q. (2018). Research on control strategy of handling stability for formula SAE (FSAE) pure electric racing car. Australian Journal of Mechanical Engineering, 16(sup1), 61–67.

Zhao, P., Chen, J., Song, Y., Tao, X., Xu, T., & Mei, T. (2012). Design of a control system for an autonomous vehicle based on adaptive-PID. International Journal of Advanced Robotic Systems, 9(2), 44.

Ziegler, J. G., & Nichols, N. B. (1942). Optimum settings for automatic controllers. ASME, 64(11).