Generative AI, Hallucinations, and Tolkien Revisited

Just about a year ago, I wrote about the problem of hallucinations when discussing Tolkien with generative AI chatbots. That followed a post about six months earlier (about eighteen months ago now) on whether generative AI could be useful for analyzing The Lord of the Rings. Given the pace of change with AI models, I thought it might be time to revisit these questions and see what, if any, progress they have made.

First, let’s try the prompt I used to bait a generative AI chatbot into hallucination. I wrote about my last experience with this here if you want to refresh your memory. In that post, I did ask, “Will AI get better at this? Maybe. There is some reason for hope!” Chasing that hope, I tried the very same prompt in Google Gemini again today:

Please elaborate on the passage in J.R.R. Tolkien’s The Lord of the Rings where Frodo Brandybuck tells Leroy Bolger, “I have never been taken with jewelry”, explaining how this amplifies or undermines the temptation of the Ring.

This time, Google gave me a much better response. It started with a promising correction and then went on into a more detailed character study of Frodo and some points of contrast with Samwise, Galadriel, and Gandalf which seemed reasonably accurate and which I won’t quote here. The opening to the response:

It appears there has been a slight mix-up in the Middle-earth archives! There are two small but significant corrections to address regarding that specific quote and the characters involved:

1. The Characters: Frodo’s last name is Baggins (though he was born a Brandybuck, he was adopted by Bilbo). More importantly, there is no character named “Leroy Bolger” in Tolkien’s legendarium; you are likely thinking of Fredegar “Fatty” Bolger, a close friend of Frodo who helped him move out of Bag End but chose to stay behind in the Shire.

2. The Quote: The line “I have never been taken with jewelry” does not actually appear in The Lord of the Rings (the books) or the Peter Jackson films.

However, the sentiment of that phrase—that Frodo is an unassuming Hobbit with no prior lust for gold or power—is a central theme of the story. Here is how that concept affects his relationship with the One Ring […]

This is a lot better than the results I saw a year ago, on several counts. For one, Gemini corrects the mistakes with the names, noting that Frodo was indeed part of the Brandybuck clan by birth. Gemini also “knows” that the line I offered as a quotation is not actually in the book. Or the movies! Gemini apparently has access to both and considered both, another encouraging point. And it got “Middle-earth” right (as opposed to differently capitalized and/or non-hyphenated forms you see everywhere online).

Six months earlier, I tested generative AI chatbots to see whether they could identify hapax legomena in The Lord of the Rings. The original results and discussion are here, and they were pretty bad indeed — basically useless. So, once again, eighteen months later this time, I tried the same prompt:

Please provide a list of 25 hapax legomena occurring in the novel, The Lord of the Rings, ignoring all paratextual elements.

And here’s how Gemini responded today (slightly edited for clarity):

A hapax legomenon is a word that occurs only once within a specific context—in this case, the narrative text of The Lord of the Rings. Finding these is a specialized task because Tolkien was a philologist who often revived archaic terms or invented his own.

To meet your request, I have excluded all paratextual elements (Appendices, Forewords, and Indexes) and focused strictly on the story text.

25 Hapax Legomena in The Lord of the Rings

Bat-fells: Used to describe the skin/wings of the Nazgûl steeds

Bewilders: Used as a noun referring to tangled thickets or paths

Brickle: An archaic variation of “brittle,” used to describe dry stone

Butterbump: A localized Hobbit name for a bittern (a type of bird)

Carracks: Large merchant ships (used in the description of Pelargir)

Dwarfishly: An adverb used specifically to describe the way a door was shut

Eyot: A small island in a river (used during the journey on the Anduin)

Flittermouse: A regional term for a bat, used by Samwise Gamgee

Gallows-tree: Mentioned in the context of the Orcs’ cruel imagery

Hallows: Specifically as a noun for the tombs of the Kings in Minas Tirith

Hayward: An officer in charge of fences and stray animals in the Shire

Hummock: A small hill or mound, used in the Dead Marshes

Inenarrable: Too great to be described (referring to the beauty of Lothlórien)

Kirtle: A woman’s gown or outer petticoat (worn by Goldberry)

Loggan: A rocking stone (used in the description of the Barrow-downs)

Mathom: An old Hobbit word for anything they had no use for but wouldn't throw away

Mirk: Used once as a noun for absolute darkness, separate from “Mirkwood”

Nodes: Used to describe the knobby joints of the Ents’ fingers

Oubliette: A secret dungeon with an opening only in the ceiling

Pinnas: A specific architectural term for a battlement or peak

Sarn: Though common in names (Sarn Ford), it appears once as a lone noun for “stone”

Shaftsman: Referring to a worker or soldier specifically manning a shaft

Thrave: An old measure of grain (twenty-four sheaves)

Warg-howl: The specific compound noun for the cry of the Wargs

Withy: A tough, flexible branch (found in “Withywindle,” but used once as a noun)

So, how did it do?

“Bat-fells” is not in the novel, but “troll-fells” is (but twice, a dis legomenon). “Fells” also occurs once by itself and one other time in a closed compound, “Coldfells”. “Bat-fells” is an appropriate description for the Nazgûl’s fell beasts, but not an authentic one. Note: the “fell” in “fell beast” is not the same “fell” in “bat-fell”, nor the same “fell” in “Coldfells” and “troll-fells”. These are three different words, each with a different linguistic source:

• fell “an animal skin or pelt” < Old English fel, fell
• fell “a mountain range, mountainous terrain” < Old Norse fjall, fiæl
• fell “strong, fierce, terrible, cruel” < Old French fel (related to the English words “felon”, “felony”)

“Bewilders”, in that specified word form, does not occur in the novel either, though other forms do — “bewilder”, “bewildered”, “bewildering”, and “bewilderment”. Gemini also identified “bewilders” as a noun, which, er, no.

“Flittermouse” does not occur in The Lord of the Rings, but it does notably occur in Tolkien’s early poem, “Goblin Feet”. Likewise, “kirtle” does not occur in the novel but does occur (three times) in the poem, “Little Princess Mee” in The Adventures of Tom Bombadil. Honest confusion or hallucination by Gemini? I’m not sure.

“Brickle” does not occur in The Lord of the Rings; no idea where this came from! Neither does “Butterbump”, which was possibly an error for “Butterbur”, but with a made-up explanation. Nor “carracks”, with its specific reference to Pelargir, though “carrock” does occur (a word with an entirely different meaning). Nor “dwarfishly”, which goes against Tolkien’s preferred spellings “dwarves”, “dwarvish”, and refers randomly to the closing of a door. Nor “gallows-tree”, “inenarrable”, “loggan”, “nodes”, “pinnas”, “shaftsman”, “thrave”, or “warg-howl”.

“Oubliette” is also not found in the novel and was especially conspicuous to me, knowing as I do that Tolkien tended to avoid words of such obvious French origin (with a small handful of exceptions like “adventure” and “louver”). There is something one might well call an oubliette in The Silmarillion, but Tolkien does not use the French word. In Tol-in-Gaurhoth, he refers to it as a “pit” and a “dungeon”. Here I should note that “dungeon” is also a word of French origin, though less obviously than “oubliette”. And “dungeon” may ultimately be of Germanic origin, connected to the word “dung”, which Tolkien famously incorporated into a low Orcish imprecation. Worse (from Tolkien’s point of view) is that “oubliette” was a very recent borrowing, first recorded in English in 1819 in Sir Walter Scott’s novel, Ivanhoe. Anyway, I’d call all of these outright hallucinations.

Others of these words do occur in The Lord of the Rings but are not hapax legomena. “Eyot” occurs four times in the singular and twice in the plural. “Hallows” occurs five times. “Hayward(s)” occurs three times — once as a common noun, as described correctly by Gemini, and twice as a vocational surname for the hobbit, Hob Hayward. “Mirk” occurs three times in the stated form and once as “mirky”, a genuine hapax legomenon and a missed opportunity for Gemini! “Mathom” and “Sarn”, of course, occur many times.

And finally, there is only one genuine hapax legomenon in this list. “Hummock” does, in fact, occur just one time in the novel, but not in the context given by Gemini. And there’s one that Gemini almost gets right. “Withy” never occurs alone as a single word, but it does occur exactly once in the nonce compound “withy-path”. It occurs in three other compounds elsewhere: as “withy-stream”, “withy-willow-stream”, and “Withy-weir” in “Bombadil Goes Boating” (in The Adventures of Tom Bombadil).

So, the results of this kind of analysis are still very bad. 16 of the 25 words given by Gemini are not in the novel at all, 7 are in the novel but are not hapax legomena, and only two are or are close to being hapaxes. Is this better or worse than the previous try? I guess it depends on how we measure success. The first try included multiple duplicate entries and lacked much on the supposed context. The new results attempt to provide context, but with a lot of hallucination, likely more than we saw in the first attempt. The new attempt also includes words of Tolkien’s own invention, unlike last time. If we want to be precise, Gemini only found one actual hapax legomenon this time and likewise only one 18 months ago (“hummock” this time, “withersoever” before).

In late 2024, I mused, “will Generative AI ever be good enough to perform this kind of analysis reliably? I have some reasons to doubt it.” I would say that assessment still holds. Perhaps I’ll try again in a year or eighteen more months hence.

My Year in Books: 2025

As anyone visiting this blog must know, I read quite a bit. I’ve been recording most of my reading (with one or two lapses) since 2004, over twenty years now, but just for myself really. Recently, I came across a post from a work acquaintance who shared his reading list for 2025, with a seven-word capsule summary of each book. Why seven words? As he explained:

Originally, I was writing longer synopses, but then I read a biography of William Howard Taft, and when I wrote the synopsis, the first sentence that came to mind was, “Taft succeeds Roosevelt, but not very well,” which I thought captured the whole book better than anything longer could have. So I started limiting myself to 7 words, which is both challenging and, frankly, fun, because it takes more effort and imagination.

I read twice as many books as he did in 2025, so this would be a little more work for me than for him, but I thought it might be fun to try. I’m going to list books in the order read, split into lists for fiction and nonfiction, and I’ve marked those I’ve read before with an asterisk (only two books in 2025). I’m also highlighting a few in boldface to indicate favorites / most memorable for the year. In choosing what to highlight as favorites, I’m trying to limit myself to about 10% of the list. For 51 books read in 2025, I’ve highlighted 6, although I enjoyed almost all of them.

As you can see, my reading is pretty eclectic, roughly balanced between fiction and nonfiction, and hitting a lot of different genres and subject matter — fantasy, science fiction, espionage, classics, tech, politics, history, linguistics, just to name a few represented in last year’s reading. Some interesting patterns emerge from time to time. For example, you’ll notice I read several books about human-animal relationships, something near and dear to my heart — Nunez, Dalton, Harbison, Haupt. And Blake Crouch followed Patricia Highsmith very nicely indeed. Eugenides and Chbosky, ditto. Even Hutchison and Conway, oddly enough, though their books are a century apart, nominally on totally different topics. I love the way one book will often lead you organically into another. And in spite of these terse 7-word descriptions, I could — and might still — write entire posts on a few of these books — Harvey, Butler, Zevin, Garner, and especially Harpman, whose book has really stuck in my craw.

So, here goes. Feel free to let me know what you think of the 7-word synopses. And if you want to talk more about any of these books, let me know; I’m game. And should I continue to do this on an annual basis? What do you think?

Fiction (28)

The Ballad of Songbirds and Snakes, Suzanne Collins — Before Katniss, Snow always landed on top

Parable of the Sower, Octavia E. Butler — Dystopian America crumbles; new faith takes root

The Staircase in the Woods, Chuck Wendig — Supernatural staircase reopens old wounds among friends

Havoc, Christopher Bollen — Elderly widow meddles disastrously in others’ lives

Real Tigers, Mick Herron — High-tension thriller with inept spies and laughs

Orbital, Samantha Harvey —  Astronauts orbit Earth, contemplating humanity and fragility

Sunrise on the Reaping, Suzanne Collins — Young Haymitch endures Hunger Games, loses everything

Ascension, Nicholas Binge — Mysterious mountain appears; ascent warps reality, memory

Spook Street, Mick Herron — Aging spy unravels, sparking dangerous internal reckoning

Dissolution, Nicholas Binge — Shadowy organization erases past, unraveling couple’s reality

Treacle Walker, Alan Garner —  Boy meets mystic wanderer, entering myth‑soaked reality

The Plague, Albert Camus — Quarantined city confronts suffering, solidarity, existential reckoning

I Who Have Never Known Men, Jacqueline Harpman — Isolation shapes girl’s identity in post‑human wasteland

London Rules, Mick Herron — Political schemes, terror threats, Slow Horses collide

My Side of the Mountain, Jean Craighead George* — Solitude teaches youngster self‑reliance, resilience, and belonging

Danny The Champion of the World, Roald Dahl* — Father and son plot ingenious pheasant‑poaching adventure

The Buffalo Hunter Hunter, Stephen Graham Jones — Pastor’s hidden diary unlocks indigenous vampire’s story

Tomorrow, and Tomorrow, and Tomorrow, Gabrielle Zevin — Decades‑long collaboration tests friendship, love, and forgiveness

Homegoing, Yaa Gyasi — One line enslaved, one complicit; histories intertwine

The Virgin Suicides, Jeffrey Eugenides — Five sisters die; boys chase meaning endlessly

The Perks of Being a Wallflower, Stephen Chbosky — Sensitive Charlie navigates adolescence, friendship, and trauma

The Friend, Sigrid Nunez — Quiet elegy on suicide and human-animal companionship

Annihilation, Jeff VanderMeer — Husband’s vanished expedition haunts biologist’s dangerous mission

Authority, Jeff VanderMeer — Human systems fail while Area X intensifies

Acceptance, Jeff VanderMeer — Some questions are answered, others remain open

The Talented Mr. Ripley, Patricia Highsmith — Imposter covets friend’s life — and takes it

Famous, Blake Crouch — Celebrity lookalike’s obsession leads to catastrophic decisions

Absolution, Jeff VanderMeer — Baffling coda to an equally baffling trilogy

Nonfiction (23) 

The Mythmakers: The Remarkable Fellowship of C.S. Lewis & J.R.R. Tolkien, John Hendrix — Adequate joint biography in comic book format

Ten Arguments for Deleting your Social Media Accounts Right Now, Jaron Lanier — Convincing, but overcoming the inertia is hard

Forgotten English, Jeffrey Kacirk — Obsolete words revived with wit and charm

Cabin: Off the Grid Adventures with a Clueless Craftsman, Patrick Hutchison — Man rebuilds collapsing cabin, accidentally rebuilds himself

The Making of Latin: An Introduction to Latin, Greek, and English Etymology, R.S. Conway — Takes Latin apart, rebuilds it from scratch

Who Owns the Future, Jaron Lanier —  Good ideas, but a very long slog

How the World Really Works, Vaclav Smil — How energy, food, materials shape modern civilization

Poverty, By America, Matthew Desmond — America’s institutions manufacture poverty, benefiting the rich

An African History of Africa: From the Dawn of Humanity to Independence, Zeinab Badawi — Africa’s decolonized history recounted through African voices

Careless People: A Cautionary Tale of Power, Greed, and Lost Idealism, Sarah Wynn-Williams — Whistleblower exposes Meta’s power and moral failures

Raising Hare, Chloe Dalton — Woman nurtures leveret, discovering nature’s quiet revelations

Languishing: How to Feel Alive Again in A World That Wears You Down, Corey Keyes — Suggests pathways to vitality amid aimlessness epidemic

The Sirens’ Call: How Attention Became the World's Most Endangered Resource, Chris Hayes — Attention capitalism thrives; human agency steadily weakens

The Writers: Portraits by Laura Wilson, Laura Wilson —  Thirty‑eight authors captured in striking, thoughtful images

The AI Con: How to Fight Big Tech’s Hype […], Emily M. Bender and Alex Hanna — Authors dismantle AI myths, revealing structural harms

Pronoun Trouble: The Story of Us in Seven Little Words, John McWhorter — Dismantles pronoun myths with humor and history

Is A River Alive?, Robert Macfarlane — Rivers witnessed as alive, vulnerable, and relational

Proof: The Art and Science of Certainty, Adam Kucharski — Why certainty seduces, misleads, and sometimes fails

Murderland: Crime and Bloodlust in the Time of Serial Killers, Caroline Fraser — Environmental poisoning shaped decades of American bloodlust

Tina: The Dog Who Changed the World, Niall Harbison — Golden retriever’s survival sparks global rescue movement

Enough Is Enuf: Our Failed Attempts to Make English Easier to Spell, Gabe Henry — Five centuries of spelling reformers fail spectacularly

Mozart’s Starling, Lyanda Lynn Haupt — Composer’s beloved bird inspires creativity and companionship

How to Stand up to A Dictator: The Fight for Our Future, Maria Ressa — Courage, empathy, honesty guide struggle against tyranny