Updated blog on November 6, 2019.
The EU General Data Protection Regulation (GDPR) went into effect last year, on May 25, 2018. Following in the EU’s footsteps California has now enacted a similar policy, the California Consumer Privacy Act (CCPA), which will go into effect Jan. 1, 2020.
These regulations are the first of many that have forced companies to change how they handle customer data. GDPR and CCPA include numerous provisions regarding data protection, privacy, a right to know how data is being used and a right to opt-out of having their data sold. CCPA includes a “right to erasure,” right to be informed, right to object and more. Unlike GDPR it has yet to enact the “right to explanation.” However other states and countries looking to enact similar legislature may follow the GDPR footsteps in enacting a “right to explanation.” Several sections of the GDPR have led to a debate among (artificial intelligence) AI industry professionals about the “right to explanation” mandate included in the regulation.
GDPR explainability clauses
GDPR Articles 13-15 and 21-22 outline requirements related to automated data processing and decision making. The basic concept is when a decision is generated solely from automated processing (no human intervention), including profiling, the data subject has the right to receive an explanation of how the decision was rendered. This clause applies when a company is using automated processing on personal data to evaluate an individual (who resides in the European Union) based on the individual’s attributes.
Automated data processing and decision systems typically use machine learning, a subset of AI. The intent of the “right to explanation” clauses in GDPR when it comes to AI algorithms and models are a subject of debate among AI industry professionals.
The explainability debate
When it comes to AI, “explanation” could mean several things: how an algorithm works or how the system functions, or the factors or data that resulted in a decision by the algorithm or system that impacted an individual (a data subject).
AI industry professionals disagree about whether “explanation” in the context of GDPR is referring to how the technologies work or the factors that led to the automated decision.
Dr. Sandra Wachter, research fellow at the Oxford Internet Institute, University of Oxford, has written about GDPR and AI. In a blog post, she said “the GDPR is likely to only grant individuals information about the existence of automated decision-making and about “system functionality,” but no explanation about the rationale of the decision.” For example, a bank could use automated data processing for online credit card applications. If an applicant is denied approval of a credit card, it is likely that the bank would not be required to provide an explanation as to the rationale of that automated decision under GDPR.
Andrew Burt, chief privacy officer and legal engineer at Immuta, explains in an article for IAPP what GDPR “in practice” means for the AI community. In the article, he said the GDPR text “suggests that a data subject is entitled to enough information about the automated system that she or he could make an informed decision to opt out.”
A few months before GDPR went into effect, Pedro Domingos, professor of computer science at UW and author of “The Master Algorithm,” published a controversial tweet that started a heated debate among the AI community:
Domingos received quite a bit of pushback on the idea that GDPR will make deep learning illegal. And now, more than a year later, many pundits are talking about “striking the balance between AI and GDPR” and “What GDPR Means for an AI Strategy.” It’s not all doom and gloom.
GDPR and AI
The GDPR explanation requirements may not be cut and dry when it comes to AI. It may take legal cases to determine the correct interpretation of the explainability clauses in GDPR as it pertains to AI. Although CCPA has yet to enact legislature similar to the “right to explanation” they have started the trend for U.S states to start regulating data protection, some of which may be more comprehensive than the CCPA and include the question “right to explanation.” GDPR, CCPA and Data Protection Laws should be a major consideration to U.S and global companies and how they handle customer data in 2020 and the future.
If you liked this post, check out our recent blog post: nuances of data enrichment.