By this point, we’d narrowed down the affected users to a single email client - Yahoo Mail, which is where we got suspicious. Had Yahoo Mail introduced any features lately that might be causing this…?
As it turns out, yes, yes they had. A quick Google search revealed that a few months ago Yahoo jumped on the AI craze with the launch of ”AI-generated, one-line email summaries”.
At this point, the penny dropped. Just like Apple AI generating fake news summaries, Yahoo AI was hallucinating the fake winner messages, presumably as a result of training their model on our old emails. Worse, they were putting an untrustworthy AI summary in the exact place that users expect to see an email subject, with no mention of it being AI-generated 🤯
I have no issue with AI summaries but they need to be clearly marked as AI and not replace the goddamn subject line.
iOS does a decent job of showing it - on the normal list of messages, you get the sender, the first 45 letter or so of the subject, and then their “summarize” icon followed by two smaller-font lines (about 75 characters) of the summarized body
Just imagine how much energy it must cost to provide this garbage to every mail going through Yahoos servers.
You do not want to know how much energy it takes to run all of Ymail’s spam cannons. Remember they also own and operate AOL.
I think that using large language models to summarize email (especially marketing), news, social media posts or any type of content that uses a lot of formulaic writing is going to generate lots of errors.
The way I understand large language models, they create chains of words statistically, based on “what is this most likely to say based on my training material”?
In marketing emails, the same boilerplate language is used to say very different things. “You have been selected” emails have similar wording to “sorry this time you have not won but…”. Same cheery “thanks for being such a wonderful sucker” tone and 99% similar verbiage except for a crucial “NOT” here and there.
That’s insane that they apparently replace the subject line. Completely irresponsible.
Should also be labeled as AI regardless. (Or ideally disabled, especially after incidents like this)
