I’ve recently figured out that interactions with a particular human have taken on a curiously stilted, disjointed flavor because they’re using an LLM for coaching on how to navigate situations. Alison says she’s been watching someone’s emails at work become increasingly demanding and vaguely paranoid, and she knows it’s LLM-driven because the “author” screwed up and left a reference to a prompt intact in one email.
For me the tell isn’t em-dashes or particular rhetorical tics, or even extreme changes. A human plainly remains somewhere in the loop. It’s a shift in the tone and what I guess we could call shared agreements or previously understood ground truths.
On that last, I suppose it’s a matter of taking the subjectivity we have to assume about every shared agreement or understanding—the simple wisdom of knowing that nobody ever really sees things exactly as we do—and sensing that the previous delta in understanding is the starting point for an LLM’s stochastic narrative-building and elaboration. A predictable and acceptable standard deviation is stretching to several standard deviations.
There are plenty of stories of people using LLMs to descend into a widening gyre of elaborate and dysfunctional subjectivity. As always, given how mass media works, the stories are lurid and extreme. But they always seem to start with a belief that the LLM managed to crystallize something for the person—seemed to speak to something they suspected but didn’t have language for, or support to believe—and then began to spin out from that initial earned trust into statistically average delusion, and then decidedly abnormal madness, where an NYT or Guardian reporter eagerly awaits.
I don’t think I’m watching someone go mad. I think I’m watching someone set aside their native competence and fundamental epistemic agency in favor of a defective cognitive prosthetic that has no mechanism or feature to mediate its own oscillations: An understanding goes in, the delta in understanding is part of a prompt, the feedback pulls further in some random direction, the delta in understanding widens, the next prompt is even further adrift, the oscillations widen.
But like I said, nobody’s going mad. I’m not concerned for anyone’s safety. I’m just concerned about a relationship, and contemplating my own responsibility in the matter, because I can’t help but wonder if more care with clarity or more patience on my part would have made an LLM a less attractive problem-solving partner.
I would prefer someone trying to find their way to understanding imperfectly but authentically to this.
I’m not sure what to do.