5 Comments
User's avatar
Robert Diab's avatar

Thanks for endeavoring this and sharing the paper, Andrew. I found it much better than 'not bad.' I was especially struck by the quality of the writing. In another example from Fable 5 posted yesterday, I drew the inference that because all the usual tell-tale signs of AI were there, no matter how powerful it gets, LLM's probably won't overcome the fingers-on-the-chalkboard cringiness you mention. But I come away from this doubting I should be so complacent.

The signs are certainly here, but there's also more subtlety and range in the writing. I suspect that with a simple prompt, you could get it to tone down the em-dashes, colons, not-this-but-thats, and so on. And if so, I think we're heading into a terrain where my operating assumption thus far — that it's easily to tell if something is written with AI — will no longer be true.

The doc I'd like to see? The one where you feed this paper back into Fable 5 and say: re-write it but with only 10% of the em-dashes, colons, etc.

Derek Hendy's avatar

Yeah, we’re fucked.

Jonathan Jackson's avatar

Thank you for sharing! I had a brief look through parts of the paper and some bits of the abstract feel a bit “off”. Some potentially useful stuff in there though.

The risk of process-based assessments being undermined (moreso) is an interesting one. Something I think about quite a bit. I think when process-based activities are scaffolded in class and there are structural constraints that enforce “going slow”, I don’t think it really matters if there are crazy capable tools that can one-shot significant tasks.

Michael Mallett's avatar

Appreciate the exercise here, Andrew.

It's quite fascinating to think about a different kind of augmented learning which could or could not hurt the formulation of independent thinking, logic, writing, speaking, and decision-making.

I recall writing my own thesis on the impact of social media on communities and relationships after reading over a dozen books and citing works from the ancient Greeks to the modern philosophers.

It was difficult but rewarding. I feel better setup for success because of that exercise, before my brain was impacting by the technologies which promise shortcuts.

I'm not so naive to recognize we need to let technology into the process and affect how institutions center around the person. Bans are a mistake. Complete takeovers are too.

The question of how the educational system truly prepare the student for this society, which is inclusive of technology for learning and doing, will teach them to discern, build, and lead the future.

All of my jobs, I've received because more for my double concentration in Philosophy than my degree in Finance. Thinking isn't going to go out of style...

Peter Buck's avatar

Andrew — useful provocation, but I wonder if the more interesting academic move is not to grade the machine against the old artifact, but to redesign the artifact.

The impressive part is not simply that Fable produced a plausible academic paper quickly. That now feels like the expected direction of travel. The harder question is whether the academic paper, as an artifact, is still doing the work we need it to do.

The paper is no longer the proof.

So the question I’d put to you and others is: what are the elements of the next scholarly article design when a “paper” is increasingly an artifact of another century? As evidence, look at how you refactored your book at https://spoileralert.wtf. I hope to hear more.