Sterling Anderson, a Ph.D. candidate at MIT, continues his discussion of semi-autonomous driver assistance systems in this post. If you haven’t read his previous posts (Part 1, Part 2) you might want to start with them.
In my last article, I described existing driver assistance systems and introduced the notion of threat assessment. Near the end of the post, I described how the choice of threat assessment metrics is critical to the performance of any advanced driver assistance or semi-autonomous system. This article explains how existing systems assess threat and describes two significant problems caused by their approach. It then describes our alternative approach to threat assessment and semi-autonomous control. My next and final post will then demonstrate our system’s performance in both simulated and experimental results.
Significant Challenges Facing the Current State-of-the-Art
The first challenge facing existing systems is their inability to accurately capture the threat inherent to a particular scenario in a meaningful way. Because assimilating multiple sources of threat into a single, actionable metric is difficult, the majority of ADAS systems in existence today consider only one hazard at a time and limit their assistance to one dimension or another. For example, adaptive cruise controllers focus on the closest obstacle within the ego vehicle’s line of sight (longitudinal dimension) and apply the brakes as necessary to avoid rear-end collisions. Lane keeping controllers focus instead on the lateral dimension, neglecting longitudinal vehicle dynamics and collision hazards in order to maintain a desired position and/or heading within the lane. Like the child playing checkers for the first time, the tactical, near-term focus of these systems often fails to consider the effect of current evasive actions on future threat scenarios. For example, Figure 1 illustrates the predictions of an emergency braking system (solid green), along with those of a lateral collision avoidance controller (dashed blue). Without considering Obstacle B or the maneuver required to straighten out after it has passed Obstacle C, the obstacle avoidance system might consider a pass to the left of Obstacle C preferable to an emergency braking maneuver. Such a decision could be disastrous if either 1) Vehicle A cannot straighten out in time to avoid running off the road or 2) Vehicle A cannot avoid an oncoming Vehicle B once it has passed Vehicle C.
The second major problem facing existing driver assistance systems is closely related to the first: the myopic focus of existing systems on a single source of threat requires that a vehicle be equipped with multiple safety systems. For example, comprehensive assistance might require that the same vehicle be equipped with warning devices to alert the driver, anti-lock brakes to prevent skidding, yaw stability control to prevent loss of control, adaptive cruise control to prevent rear-ending the vehicle ahead, lane-keeping control to prevent wandering out of one’s lane, etc. While each of these systems may perform adequately by itself, their combined output when placed together on the same vehicle can lead to vehicle performance that is suboptimal at best and unpredictable or inconsistent at worst.
For instance, imagine a vehicle traveling quickly along a curve in road (illustrated in Figure 2). Wanting to avoid the large truck that is passing you on the inside of the turn, you drift toward the outer edge of your lane. As the lane departure warning begins to sound (adding to your already-heightened apprehension), the lane assist system takes control of the steering wheel, turning it hard left. As you swerve left, the car begins to roll right, whereupon the rollover alarm lights begin to flash and the roll stability controller engages, turning you back toward the outer edge of the road and again engaging the lane keeping controller. You can imagine how this scenario might continue. The point is that when distinct systems with disparate goals try to control the same vehicle, conflicts in their warnings and their steering, acceleration, and braking commands are bound to result. Figure 3 shows a (simplistic) schematic of how multiple systems arranged on the same vehicle can sometimes interact.
The Alternative: An Integrated, Semi-Autonomous Active Safety Framework
To summarize the above, existing active safety systems are both incomplete in their assessment of threat and ineffective at sharing control with the human driver. This is where we come in.
When I began working with Karl Iagnemma and Steve Peters at MIT’s Robotic Mobility Group a couple of years ago, we began exploring a promising new idea for semi-autonomous control. We based our system on the simple observation that most human drivers tend to operate within a field of safe travel as opposed to along a predetermined path. Thus, instead of starting with a simple avoidance path, we chose to define a corridor through the environment that avoids obstacles, road edges, and other hazards. Then rather than selectively replace the driver when s/he strayed from the automation-desired path, we chose to gradually blend the driver’s inputs with the controller’s – giving the driver free reign while s/he remained within the safe corridor and only intervening as the likelihood of his or her leaving that safe corridor or losing control, increased.
The framework we developed (illustrated in Figure 4) storyboards as follows:
As a human driver navigates the vehicle, forward-looking sensors detect road edges, identify and terrain features (slopes, holes, etc.), and localize obstacles. Based on this information, a safe corridor that avoids these hazards is defined. A mathematical model of the vehicle is then forward-simulated to determine the safest or most stable path through the corridor given the vehicle’s current state (position, velocity, roll angle, etc.) and the current state of the environment. Because this optimal trajectory can be considered the best maneuver that can possibly be performed given the current circumstance, it is then used to assess the (best case) threat posed to the vehicle. In other words, no matter how skilled the human driver is, s/he will be unable to perform any better than the “best case” avoidance maneuver. As this “best case” maneuver becomes more dangerous, so does any maneuver that the human might attempt. In low-threat scenarios, the human retains full control of the vehicle (K=0 for you sharp-eyed readers). As threat increases owing, for example, to the driver’s failing to make adequate preparations to avoid an obstacle, so does the level of control authority given to the autonomous controller (gain K in the Figure 4 below). In extreme scenarios, when the avoidance maneuver required to keep the vehicle safe becomes so severe that only an optimal maneuver can be expected to pull it off, gain K becomes one and the system effectively acts as an autonomous controller until threat is reduced to human-manageable levels.
Figure 5 illustrates what the predicted avoidance maneuver might look when the vehicle is at position 1 (low threat, full human control), as well as how that prediction might appear when the vehicle reaches position 2 (high threat, nearly-autonomous operation).
The video below illustrates the semi-autonomous controller in action. Corridor constraints are shown in black and green, with the vehicle’s trail in blue and the predicted trajectory in red. In this simulation, the human driver fails to see or respond to a hazard. As the system’s prediction begins to predict a more and more severe avoidance maneuver, it gradually asserts only enough control necessary to avoid the hazard before giving control back to the human driver.
To conclude, the system we’ve developed combines the environment’s many hazards into a single, safe corridor and moderates the driver’s steering and braking inputs as necessary to keep the vehicle within that corridor. In low threat scenarios, the driver maintains full control of the vehicle. As threat increases, the controller shares control with the human driver to ensure that the vehicle does not leave the safe corridor. By inherently considering multiple sources of threat in a single, unified framework, this approach provides a significant advantage over existing driver assistance systems.
In my next post, I’ll show how this framework performed in the over 800 experimental trials with 8 different human drivers on a Jaguar S-Type. Stay tuned.Posted: February 2nd, 2011 | Filed under: Design, Optimization, Transportation |