Humans decisions are affected by all manner of biases.  Many of these biases are quirks of the human mind.  A common bias, for example, is to assign more weight to recent data than to older data.  Another involves seeing patterns from random clusterings of events.  Some biases are malignant and the result of human prejudice based on race, ethnicity, gender, or sexual orientation.  Relying on machines to make decisions would seemingly portend more rational and less biased decision making, or will it?

"I know I've made some very poor decisions recently, but I can give you my complete assurance that my work will be back to normal."

A bill now pending before the currently recessed California Legislature cites concerns about the use of algorithmic or automated decision systems to make hiring and other workplace decisions, eligibility decisions, insurance eligibility, lending decisions, and marketing decisions quickly, automatically, and fairly.  This bill, AB 2269, would require a business in California that provides a person, as defined, with a program or device that uses an automated decision system (ADS) to:

• take affirmative steps to ensure that there are processes in place to continually test for biases during the development and usage of the ADS,
• conduct an ADS impact assessment on its program or device to determine whether the ADS has a disproportionate adverse impact on a protected class,
• examine if the ADS in question serves reasonable objectives and furthers a legitimate interest, and

"You know of course though he's right about the 9000 series having a perfect operational record. They do."

AB 2269 would also require a business beginning March 1, 2022 to submit a report to the Department of Business Oversight summarizing the results of its ADS impact assessment for each program or device that uses an ADS. 

The bill is extremely vague.  For example, it is unclear what would make a business "in California".  Further, the definition of ADS would seem to cover just about anything that uses a microprocessor.  As defined, ADs means a "computational process, including one derived from machine learning, statistics, or other data processing or artificial intelligence techniques, that makes a decision or facilitates human decision making, that impacts persons".