Liability for a defectively-designed algorithm: Wickersham v. Ford
By Zane Muller, CLS ’20
The past few years have witnessed a dramatic increase in the prevalence and sophistication of algorithms. Advances in machine learning (sometimes called artificial intelligence) have delivered new applications in areas as diverse as credit risk evaluation, criminal sentencing and winning the ancient Chinese strategy game Go. While they have long been incorporated into software and web interfaces, machine learning algorithms are increasingly used to improve consumer products, and consumers increasingly encounter them in the physical world. As algorithms further permeate our everyday lives, the law will increasingly have to decide how to handle losses that arise when algorithms fail.
Design defects are intuitive in the case of, say, a lawnmower; but how is a machine learning algorithm “designed”? In broad terms, machine learning refers to an automated process for identifying relationships between variables in a data set and making predictions based on those relationships.[1] Those relationships accumulate into a “model”, or algorithm, which can then be used to make predictions or decisions based on new data.[2] Their design involves two stages: “playing with the data” and “running the model.”[3] In the first stage, designers choose a set of data, determine an outcome goal (ie, “identify the likelihood that a given borrower will default”), and then train the model through various iterations until it independently delivers predictions in line with empirical results. In the second stage, designers “set it loose” in the world to interpret newly-gathered data and use it to deliver predictions or decisions, periodically refining or adjusting it based on the accuracy of results.
Will the law recognize and remedy injuries caused by a “defective” algorithm? This question arose in Wickersham v. Ford.[4] In that case, the plaintiff’s husband committed suicide in the wake of an automobile accident that left him with continuous, extreme pain and debilitating injuries, including the loss of an eye.[5] One of the plaintiff’s expert witnesses stated that the cause of this injury was a 146-millisecond delay in the deployment of the seatbelt pre-tensioner and side airbag. A second expert witness testified that the cause of this delay was a defect in the design of the car’s Restraint Control Module (RCM), an electronic component that receives sensor data, processes it with an algorithm, and then determines whether and when to pre-tension seat belts and deploy airbags in anticipation of a collision.
In the instant case, the plaintiff’s expert witness alleged that Ford was negligent in designing its algorithm. More specifically, he claimed that the RCM was not properly calibrated for the type of crash the plaintiff’s husband experienced, and that his injuries could have been avoided if Ford had conducted more thorough testing.[6]
One challenge facing plaintiffs is that algorithms are “black boxes” whose workings are often opaque even to their designers. Here, the plaintiff was able to overcome this hurdle because her expert had experience working with a similar algorithm for General Motors. Furthermore, the court held that the plaintiff alleged sufficiently particular and concrete facts to sustain a claim and denied Ford’s motion for summary judgment.[7]Wickersham presents a case where the causation of an injury by an algorithm’s failure is fairly straightforward; other plaintiffs, whose injuries may be less traceable to algorithmic design, may have a harder time overcoming summary judgment. Not all algorithm design flaws will be as clear-cut as the failure to quickly deploy an airbag, but could nonetheless cause equally or more serious harms. As algorithms further penetrate the physical world, these issues will only become more prominent and challenging for courts and lawmakers to resolve.
[1]Kevin P. Murphy, Machine Learning: A Probabilistic Perspective, 1 (2012).
[2]Michael Berry & Gordon Linoff, Data Mining Techniques: For Marketing, Sales, and Customer Relationship Management, 8-11 (2004).
[3]David Lehr & Paul Ohm, Playing With the Data: What Legal Scholars Should Learn about Machine Learning.51 U.C. Davis L. Rev. 653, 670 (2017).
[4]Wickersham v. Ford Motor Company, 194 F.Supp.3d 434 (2016).
[5]Id.at 435.
[6]Id. at 438.
[7]Id. at 436.