Some of the greatest legal AI risks arise after an AI tool produces its answer, and often, they’re not immediately recognizable.
From unintentional discriminatory outcomes and malicious generation to unlicensed copyrighted content that can increase organizational liability, AI outputs can create five distinct areas of additional legal risk.
This blog walks through the implications of bias and hallucinations in AI outputs and how they can create legal and reputational exposure. If you’d prefer to listen to my analysis, tune in to episode seven of ZeroDay Law’s Decoding Cyber Law podcast series.
Every AI model is trained on massive quantities of data. Most training data comes from sources the AI vendor may have only partially cataloged or disclosed. So when an employee from your organization enters a prompt into an AI tool, the AI model’s response may include sensitive information drawn from the model’s training data (unrelated to your company), even though the employee did not provide that information as part of the AI prompt.
When protected information surfaces in AI output, organizations may face privacy violations. Even though your organization was not the source of the protected information and does not have a relationship with it, using the AI tool’s output can constitute a privacy-violation disclosure.
Learn how to stop an AI vendor from using your data for training purposes in our blog post, “AI Vendor Due Diligence: What AI Tools Can Do With Your Business Data.”
One of the most significant legal AI output risk categories involved discrimination in automated decision-making. Or, putting it another way, the automated ability to influence decisions affecting someone's ability to earn a living.
Within an organization, this risk category can include:
Consumer-based decisions, from housing applications and credit line decisions to loan approvals and financing applications, can also be influenced by automated AI decision-making.
Important note: Liability for discriminatory AI output arises whether or not the discrimination is intentional.
Learn more about AI bias in our blog post, “Responsible AI: Bias, Transparency and Accountability.”
The current wave of litigation against social media companies is focused on the use of AI-powered algorithms to produce outputs and content alleged to be false, misleading or dangerous. Plaintiffs in these cases are claiming that AI-generated content is resulting in users experiencing significant mental health problems, including suicide and death.
Additionally, this risk area can include false AI outputs resulting in product liability and negligence claims. In these types of claims, the AI vendor and the deploying organization can be exposed.
Bad actors are already using AI to help support actions that vendors didn't intend. Documented use cases include planning nefarious activity, generating disinformation, producing illegitimate content, creating deep fakes and malicious code, and more.
Weaponizing AI outputs through disinformation is a massive issue. AI can make false or misleading content easier to create at scale and more difficult to distinguish from legitimate information. For organizations, weaponized AI outputs can create risks ranging from reputational harm to fraud and other legal exposure.
Full disclosure: this one is a pet peeve of mine. If you've used AI research tools as open-source, professional resources, you've probably encountered seemingly legitimate, but hallucinated resources in the AI output.
The specific patterns I've personally seen include:
Organizational output based on AI-invented sources damages your credibility with clients, and in some circumstances, relying on hallucinated AI information can result in legal liability.
Though AI can certainly make workflows more efficient, these tools also introduce risk via AI output accuracy.
AI outputs aren’t a true shortcut, at least not a reliable one…yet.
You can build workflow controls, pick AI tools with better citation practices and train your team on where to be most skeptical when integrating AI outputs into deliverables. But human review is a mandatory requirement for using AI tools.
For help evaluating your AI output risks and building the controls to manage them, reach out to ZeroDay Law. Listen to the Decoding Cyber Law podcast for the full discussion on bias and hallucinations, and what to specifically look for in your AI outputs.