I have a relative that was told flat out that they were going to be replaced by AI, at a major insurance company.

The company poured hundreds of millions into the effort, but unfortunately, the early trials were completely disastrous. It turned out that it's nearly impossible for large language models to navigate a conversation with a human that is emotionally distraught after a disaster.

Ofc, they rolled it out live anyway, with the idea being that they'd fix it as they go.

1/

It's now a year and some change later, and despite the "improvements" in the models, the AI remained disastrous. Not sure what happened, but they're now yanking the cord on the entire project.

The result? My relative now has four times as much work to do. The AI not only didn't replace them, the reality is that the company lost money on the whole adventure.

Sadly, the staff at this place are salaried, sooooo yay for doing the work of the AI for free? Technological innovation, amiright?

2/

What I thought was very interesting was the type of failures that occurred during this company's attempt to deploy AI. These weren't things that you could engineer around.

Large language models are fundamentally unable to deal with human interaction - a human might say something wrong with or without intent, might provide inaccurate information, or might ramble about something completely unrelated.

There's no magic that lets a language model navigate this. It's just a token generator.

3/

@scarlet "with the idea being that they'd fix it as they go," as though literally no one has ever learned that technical debt can't be paid off, it only accumulates.

(The joke is that the vast majority of those within management have not nor can learn this.)

@itspomf Yup - at this point, I'm certain that the eventual end of humanity will come about due to the cumulative tech debt of the prior millennia, lol.

Just imagine trying to debug multiple layers of LLM written code abstractions, a hundred years in the future.

They're going to "move fast and break things" right into a ditch, lol.

@scarlet I think they've surpassed the ditch and found the underground light rail lines at this rate. Or maybe that's just the old catacombs.

@itspomf It's probably the city under the city, or the underground caves. At this point, they're probably just spelunking in the ninth circle of hell - they don't realize its warm yet because the sheer power of denial is just that powerful.

@scarlet "who's that goat guy who keeps staring at us?" asks Chad from marketing.

Everyone tells him it's probably "one of the poors in a Halloween outfit" and to try to grift them for facial recognition data.

@scarlet taking a small and largely irrelevant detail from part of the problem and making it the focus of your whole response? sounds like this ai is also a redditor