The leading artificial intelligence models might've met their match: Gen Alpha.
A new study — which you can find at the ACM Digital Library — found that four leading AI models (GPT-4, Claude, Gemini, and Llama 3) all struggled to fully understand slang from Gen Alpha, defined as young folks born between 2010 and 2024.
The study goes into detail about the complicating factors of Gen Alpha slang, which is often born out online spaces, most notably gaming. One phrase can mean totally different things. For instance, they used the example of "Fr fr let him cook" — that's someone supporting another person — and "Let him cook lmaoo," which is mocking. Such delicate differences between language can be difficult to trace, especially since young folks often used coded language to hide their true meaning. And apparently LLMs struggle with it. In particular, the researchers noted, it struggled with identifying "masked harassment," which would be troubling for AI-powered moderation systems.
"The findings highlight an urgent need for improved AI safety systems to better protect young users, especially given Gen Alpha’s tendency to avoid seeking help due to perceived adult incomprehension of their digital world," the study read.
To be fair to AI models, understanding young folks' slang — especially Gen Alpha, which has grown up in digital spaces — is difficult for humans, too. The study looked at parents' understanding of slang, too, and that group came in at 68 percent for having a basic understanding. That was about the same mark as the top-performing LLM, Claude. Though, to be fair, LLMs did seem to have a slight edge over parents in identifying context and safety risks in the language — though all parties performed pretty poorly. Only Gen Alpha itself was reliable at understanding the slang, its context, and potential risks.
The TL;DR of the study seems to be that AI can't reliably understand Gen Alpha and that could result in poor content moderation. That perhaps tracks, since other studies have shown that AI has struggled with complex comprehension.
"This research provides the first systematic evaluation of how AI safety systems interpret Gen Alpha’s unique digital communication patterns," the study's conclusion read. "By incorporating Gen Alpha users directly in the research process, we’ve quantified critical comprehension gaps between these young users and their protectors—both human and AI."
Topics Artificial Intelligence
