Challenge accepted. I will read them both and email you my "vote." What is fascinating to me is that I was on a similar quest in July, training Claude in a "writing project" to sound like me as much as possible. It made a writing skill, etc. I used that for a project I've been co-working on (assessment reform in higher education). I'll email you the result. It took a long time. I had to substantially rewrite much of the prose, but for me, the point was to generate something that is of higher quality than I could do alone. AI is used for decision aid in healthcare because the diagnostic accuracy exceeds that of the individual clinician (e.g., detecting cancer), at the risk of some deskilling when the AI tool is not available. AI has made a chemistry dissertation focused on determining protein structure obsolete (AlphaFold), so clearly it can do amazing things. For me, the hard part is figuring out what to do that's better than what I can already do. For writing, there were some surprises along the way, and they were very specific to my case. One of the constructs my paper was proposing was already described in the Francophone literature (I can't read French and never would have found it, and it is not indexed in any of my library's search engines). I had been using certain terms in a way that was a category error, which I didn't know. However, this was also present in the literature, so I wasn't "hopelessly naive," but was working from a smaller sample of papers that were also committing the category error (and the literature is inconsistent). Generative AI was a great "reverse dictionary" for finding terms and synonyms for constructs I needed to name and define until I found the right words that were more accurate, which sometimes had a historical precedent in an adjacent literature, and which didn't "sound bad" to me. I know I have some flaws in my writing. Some I keep, because (to quote John Candy, "I like me."). Some I use AI to clean up. I enjoy these posts on AI-assisted writing.
I initially approached it the way you suggested—trying to determine which paper was "better." But I quickly realized I couldn't answer that question until I first asked, "Better according to what?" I worked with my AI to develop a simple framework based on values that matter to me as an educator: learning, dialogue, human judgment, ethical responsibility, and the extent to which scholarship invites others into meaningful engagement.
I then did something I hadn't planned. I had two different AI systems independently evaluate the papers, compared their reasoning, and then asked each AI to critique the other's evaluation. What surprised me wasn't that they produced different scores—it was that their disagreements exposed ambiguities in my framework.
Ironically, I ended up spending less time deciding which paper was "best" and more time thinking about a different question: If AI is becoming increasingly capable of producing competent scholarly writing, what qualities should distinguish scholarship that is genuinely valuable?
Your experiment shifted my thinking beyond authorship and disclosure and toward the purpose of scholarship itself. AI gave me an opportunity to ask a much deeper question about what we value in scholarly work.
So thank you for sending me down that rabbit hole! I have a feeling it's going to keep me thinking for quite a while.
Love this Amy - thanks! It's uses like this, I think, where AI becomes a genuine catalyst for thinking and exploration, but you do have to start with curiosity and the skills/ability to gnaw at a question 😊. btw I don't know if you saw my musings on scholarship here: https://soyouwantaphd.wtf/viewer.html?slug=fnd_scholarship (and through this set of AI-focused resurces)
I only read the abstract of these, and found the (obviously) LLM-generated one better. I had to read several sentences multiple times in your abstract, and I still didn't feel I understood what was going on. Found this kinda curious, really, because I both enjoy and easily parse your non-academic writing. (I'm not curious enough about the topic to read the rest, beyond the abstract, sorry. ;p)
It would be great to run an experiment like this at scale, but I'm not sure how to adjust for the obvious confounds of em-dashes and other stylistic tells.
Very interesting - looks like you correctly identified Marmoset as the LLM generated one (and interestingly I didn’t intentionally remove the em dashes - that’s just how things panned out) if you did end up reading a few paragraphs of the introduction I’d be interested in whether the comparison persists, as abstracts are notorious for their compression
One of the things I increasingly wonder about is how many of the characteristics and personas of the various models are due to the underlying model itself (ie the initial weights) vs the various post training strategies that are used (ie someone steered the model in one way or another). Or similarly how many of the clear limitations of AI writing are solvable with different post training strategies. Basically are these things fundamental to the AI models themselves, or some layer on top of it that has made the models think this is right. It’s interesting that some things are consistent across model providers and other things aren’t, eg ChatGPT and Claude have very different voices. With time as more models come out, esp open weight ones, I suspect there will be more variation in the post training that allows us to see. I suspect any domain that’s subjective and “taste” heavy will get disproportionate weight from post training, which makes the role of those people doing the tuning especially impactful. And of course as more of the training data becomes AI generated, then the two become harder to distinguish
Uh oh ... did you redact the em-dashes? Or use them as a red herring? It's like the scene from Princess Bride. "He deliberately used em-dashes in Marmoset to fool us into thinking Rabbit was human. But then he knew we were going to assume this so he did the opposite and guided the LLM to liberally use m-dashes. But of course he knew we would assume he was trying to deceive us by thinking he was deceiving us when he wasn't ...."
"Never go in against a Sicilian when death is on the line!"
Ha - perfect Princess Bride. Connection! And I use em dashes and have for decades, and am not stopping. If it means someone thinks AI wrote my work, their loss 😊
Uncertainty is a lovely human quality, one of many. Unequivocal certainty, on the other hand, is hubris or ignorance or both. AI, in my limited one-person experience, professes certainty until you point out something it has written that's provably wrong, and then it's politely apologetic. AI isn't embarrassed about its mistakes the way I would be. It hasn't had my awkward adolescence or the striving for excellence instilled in me by my parents, or the trial-and-error learning that has given me opinions about how I want to sound now, which is quite different from how I would have wanted to sound twenty years ago. Uncertainty is but one of the qualities that comes with a beating human heart.
LLMs owe everything they "know" to the collective writings of a vast array of humans (without giving those humans credit or compensation, by the way.) We humans may be intellectual and logical (at our best) but we are also deeply and unavoidably emotional. LLMs, like vampires, suck the blood from the writing of living humans and their writing sometimes appears to be alive, but it's not alive in the way of something human-written. (Don't get me wrong; LLMs are often very useful vampires.) With their encyclopedic access to info and improving grasp of syntax and rules of argumentation, LLMs can be immensely helpful in shaping and refining what we humans write but ultimately, as Claude admitted to me, "I don't have a heart for things to land in."
Our human language is a unique adaptation in the animal world, but it's our animal nature that makes good writing exhilarating. Reading and writing, at their best, are profound emotional experiences; exhilaration is an emotion! Okay, maybe AI can generate a short story that wins an award, even makes you smile like "sunrise over a sink" (wtf?) or maybe human judges are sometimes wrong, and their decisions falter like sunset over a kitchen sink disposal. Mr. Maynard, I'd trust your writing, with or without uncertainty, over LLM output, regardless of how attractive the undead may sometimes be.
It's a well known bias of LLMs that they prefer their own outputs to those of other models and to those of humans: https://arxiv.org/pdf/2404.13076. so I wouldn't put too much credence in an LLM telling you that a paper written by an LLM is better to a human written paper.
Thanks - hadn’t seen this paper. Of course this is a moving target with the sophistication of agentic frontier AI, but clearly from my experience still an issue. That said there is a rapidly growing community of people who take LLM critique as the gold standard …
Nice post. I’ve used AI assisted writing of my academic papers for quite a while. The feeling you have that the models are grating (quite a choice of a word, btw) is likely because you start to see the deeper patterns in a model’s use of language. I’m not talking about the obvious tells everybody is referring to. I’m talking about the general voice of the model. After a while you arrive at something of an aesthetic exhaustion. It becomes annoying not because it is bad or sloppy, but because there is just so much out there that all has the same voice.
Thanks Michael. Your note reminded me that I wanted to include something in the article on skills and voice-training, but it slipped my mind - just added as an update.
I suspect the LLM style and the monotony of it is part of this. But I think it goes deeper, and touches on how words lead to understanding in our evolved biological brains. At the same time there’s a chance that to many people AI writing is just fine. And this in itself raises some quite deep questions around the process of meaning making through reading …
Challenge accepted. I will read them both and email you my "vote." What is fascinating to me is that I was on a similar quest in July, training Claude in a "writing project" to sound like me as much as possible. It made a writing skill, etc. I used that for a project I've been co-working on (assessment reform in higher education). I'll email you the result. It took a long time. I had to substantially rewrite much of the prose, but for me, the point was to generate something that is of higher quality than I could do alone. AI is used for decision aid in healthcare because the diagnostic accuracy exceeds that of the individual clinician (e.g., detecting cancer), at the risk of some deskilling when the AI tool is not available. AI has made a chemistry dissertation focused on determining protein structure obsolete (AlphaFold), so clearly it can do amazing things. For me, the hard part is figuring out what to do that's better than what I can already do. For writing, there were some surprises along the way, and they were very specific to my case. One of the constructs my paper was proposing was already described in the Francophone literature (I can't read French and never would have found it, and it is not indexed in any of my library's search engines). I had been using certain terms in a way that was a category error, which I didn't know. However, this was also present in the literature, so I wasn't "hopelessly naive," but was working from a smaller sample of papers that were also committing the category error (and the literature is inconsistent). Generative AI was a great "reverse dictionary" for finding terms and synonyms for constructs I needed to name and define until I found the right words that were more accurate, which sometimes had a historical precedent in an adjacent literature, and which didn't "sound bad" to me. I know I have some flaws in my writing. Some I keep, because (to quote John Candy, "I like me."). Some I use AI to clean up. I enjoy these posts on AI-assisted writing.
Thanks Joel! Assessment on sources and critique of both spot on!
Andrew, thanks for the challenge.
I initially approached it the way you suggested—trying to determine which paper was "better." But I quickly realized I couldn't answer that question until I first asked, "Better according to what?" I worked with my AI to develop a simple framework based on values that matter to me as an educator: learning, dialogue, human judgment, ethical responsibility, and the extent to which scholarship invites others into meaningful engagement.
I then did something I hadn't planned. I had two different AI systems independently evaluate the papers, compared their reasoning, and then asked each AI to critique the other's evaluation. What surprised me wasn't that they produced different scores—it was that their disagreements exposed ambiguities in my framework.
Ironically, I ended up spending less time deciding which paper was "best" and more time thinking about a different question: If AI is becoming increasingly capable of producing competent scholarly writing, what qualities should distinguish scholarship that is genuinely valuable?
Your experiment shifted my thinking beyond authorship and disclosure and toward the purpose of scholarship itself. AI gave me an opportunity to ask a much deeper question about what we value in scholarly work.
So thank you for sending me down that rabbit hole! I have a feeling it's going to keep me thinking for quite a while.
Love this Amy - thanks! It's uses like this, I think, where AI becomes a genuine catalyst for thinking and exploration, but you do have to start with curiosity and the skills/ability to gnaw at a question 😊. btw I don't know if you saw my musings on scholarship here: https://soyouwantaphd.wtf/viewer.html?slug=fnd_scholarship (and through this set of AI-focused resurces)
I only read the abstract of these, and found the (obviously) LLM-generated one better. I had to read several sentences multiple times in your abstract, and I still didn't feel I understood what was going on. Found this kinda curious, really, because I both enjoy and easily parse your non-academic writing. (I'm not curious enough about the topic to read the rest, beyond the abstract, sorry. ;p)
It would be great to run an experiment like this at scale, but I'm not sure how to adjust for the obvious confounds of em-dashes and other stylistic tells.
Very interesting - looks like you correctly identified Marmoset as the LLM generated one (and interestingly I didn’t intentionally remove the em dashes - that’s just how things panned out) if you did end up reading a few paragraphs of the introduction I’d be interested in whether the comparison persists, as abstracts are notorious for their compression
One of the things I increasingly wonder about is how many of the characteristics and personas of the various models are due to the underlying model itself (ie the initial weights) vs the various post training strategies that are used (ie someone steered the model in one way or another). Or similarly how many of the clear limitations of AI writing are solvable with different post training strategies. Basically are these things fundamental to the AI models themselves, or some layer on top of it that has made the models think this is right. It’s interesting that some things are consistent across model providers and other things aren’t, eg ChatGPT and Claude have very different voices. With time as more models come out, esp open weight ones, I suspect there will be more variation in the post training that allows us to see. I suspect any domain that’s subjective and “taste” heavy will get disproportionate weight from post training, which makes the role of those people doing the tuning especially impactful. And of course as more of the training data becomes AI generated, then the two become harder to distinguish
Uh oh ... did you redact the em-dashes? Or use them as a red herring? It's like the scene from Princess Bride. "He deliberately used em-dashes in Marmoset to fool us into thinking Rabbit was human. But then he knew we were going to assume this so he did the opposite and guided the LLM to liberally use m-dashes. But of course he knew we would assume he was trying to deceive us by thinking he was deceiving us when he wasn't ...."
"Never go in against a Sicilian when death is on the line!"
Ha - perfect Princess Bride. Connection! And I use em dashes and have for decades, and am not stopping. If it means someone thinks AI wrote my work, their loss 😊
Be that all as it may, my vote is that you wrote Rabbit. Wait, how do we find out the answer?
Yep - Rabbit is mine 😁
Yay! What's my prize?
Uncertainty is a lovely human quality, one of many. Unequivocal certainty, on the other hand, is hubris or ignorance or both. AI, in my limited one-person experience, professes certainty until you point out something it has written that's provably wrong, and then it's politely apologetic. AI isn't embarrassed about its mistakes the way I would be. It hasn't had my awkward adolescence or the striving for excellence instilled in me by my parents, or the trial-and-error learning that has given me opinions about how I want to sound now, which is quite different from how I would have wanted to sound twenty years ago. Uncertainty is but one of the qualities that comes with a beating human heart.
LLMs owe everything they "know" to the collective writings of a vast array of humans (without giving those humans credit or compensation, by the way.) We humans may be intellectual and logical (at our best) but we are also deeply and unavoidably emotional. LLMs, like vampires, suck the blood from the writing of living humans and their writing sometimes appears to be alive, but it's not alive in the way of something human-written. (Don't get me wrong; LLMs are often very useful vampires.) With their encyclopedic access to info and improving grasp of syntax and rules of argumentation, LLMs can be immensely helpful in shaping and refining what we humans write but ultimately, as Claude admitted to me, "I don't have a heart for things to land in."
Our human language is a unique adaptation in the animal world, but it's our animal nature that makes good writing exhilarating. Reading and writing, at their best, are profound emotional experiences; exhilaration is an emotion! Okay, maybe AI can generate a short story that wins an award, even makes you smile like "sunrise over a sink" (wtf?) or maybe human judges are sometimes wrong, and their decisions falter like sunset over a kitchen sink disposal. Mr. Maynard, I'd trust your writing, with or without uncertainty, over LLM output, regardless of how attractive the undead may sometimes be.
Thanks - and well said!
It's a well known bias of LLMs that they prefer their own outputs to those of other models and to those of humans: https://arxiv.org/pdf/2404.13076. so I wouldn't put too much credence in an LLM telling you that a paper written by an LLM is better to a human written paper.
Thanks - hadn’t seen this paper. Of course this is a moving target with the sophistication of agentic frontier AI, but clearly from my experience still an issue. That said there is a rapidly growing community of people who take LLM critique as the gold standard …
Nice post. I’ve used AI assisted writing of my academic papers for quite a while. The feeling you have that the models are grating (quite a choice of a word, btw) is likely because you start to see the deeper patterns in a model’s use of language. I’m not talking about the obvious tells everybody is referring to. I’m talking about the general voice of the model. After a while you arrive at something of an aesthetic exhaustion. It becomes annoying not because it is bad or sloppy, but because there is just so much out there that all has the same voice.
Thanks Michael. Your note reminded me that I wanted to include something in the article on skills and voice-training, but it slipped my mind - just added as an update.
I suspect the LLM style and the monotony of it is part of this. But I think it goes deeper, and touches on how words lead to understanding in our evolved biological brains. At the same time there’s a chance that to many people AI writing is just fine. And this in itself raises some quite deep questions around the process of meaning making through reading …