Welcome to an all-new digital dystopia

Ray Bradbury’s dystopic science fiction novel Fahrenheit 451, a reference to the temperature at which paper burns, was based on one of the most defining images of cultural destruction in the 20th century — the spectacular and performative burning of books by Nazis. In the novel, people resist the wanton destruction of books by turning to an all too human skill, memory, to preserve books by learning them by heart.

While fire has been the favoured tool of political ideology in its battle against books, there are indeed many other ways humans have destroyed books, and the latest version of this entails large AI companies like Anthropic destroying old books to ingest them as raw data for training large language models (LLM).

Industrial scale blades guillotine the spines of physical books, unbinding the pages to be fed into high-end scanners that convert text into digital data while the physical remains are shredded. It’s hard to imagine a book lover who is able to watch a video of this form of book destruction without a sense of horror that echoes the nightmare conceived by Ray Bradbury.

And yet, often, the only thing that outdoes the dystopia of science fiction is the feral reality of science and technology. If the bibliophiles of Fahrenheit 451 turned to human memory, what is being assembled today is a vast memory machine that has no use or regard for imperfect human memory.

Unlike book purges of the past that took place in the name of political ideology and the cleansing of culture, the contemporary destruction of books is ironically being carried out in the name of knowledge production and preservation.

This textual cannibalism craves a lot of text, but treats the physical carrier, the book, merely as a container of raw data. If books were earlier the primary artefacts of human memory and consciousness, now they are merely vehicles that carry data sets.

Interestingly, this is not the first time the physical book and its contents have been on a collision course. In the early days of print history, book binders took an opposite route to arrive at the same destination. They routinely destroyed old manuscripts to use them in the spines of new books, valuing the parchment while discarding the text. History does repeat itself, as farce and tragedy, not necessarily in any order.

Writing in 1953, the Polish poet Czeslaw Milosz hauntingly declared that whatever does not exist on paper does not exist at all. As someone who lived through a communist dictatorship at the height of the cold war, Milosz would have known better than most the importance of pitting political memory against enforced amnesia. His Czech counterpart Milan Kundera echoed this sentiment in his famous lines ‘the struggle of man against power is the struggle of memory against forgetting’.

Why should it matter, cynics may ask, whether paper survives, when the word itself is being preserved.

Consider a story that circulates among librarians. A researcher working on an arcane subject goes to a library with little hope of finding a rare book essential to their work. To their delight, the book is on the shelf, but to their surprise, the borrowing card is blank: in over fifty years nobody has ever issued it. They go to the librarian and ask if no one has ever taken this book out, who was it bought for? The librarian replies: why, you, sir.

Books have always been a wager on the future, on readers who may not yet exist. But AI companies have no use or appetite to invest in unknown futures. Replacing human readers with memory machines reverses Bradbury’s consolation at the end of his book where the book people preserve memory by ingesting books into their beings. In the contemporary version, the machine will remember while we will forget because we have no paper.

Digitisation is not a bad thing. Given the fragility of paper, it is important to digitise, but there is a big difference between a small library digitising its fragile books and multibillion dollar companies choosing that route because non-destructive scanning is much slower and more expensive. But can we lay all the blame entirely on the actions of companies like Anthropic, or are we too complicit in some ways with the book burning of our times?

One of the sharpest, and most controversial indictments of ordinary complicity during the Nazi regime came in the form of Daniel Goldhagen’s book Hitler’s Willing Executioners, which argued that the Holocaust was perpetrated not just by a group of SS fanatics but also by ordinary Germans who willingly and often zealously participated in the extermination of Jews.

I certainly do not want to equate the destruction of books with the scale of violence and crime that he was speaking about, but the question nonetheless remains. How do we think about complicity in the contemporary destruction of books by AI companies?

The enthusiastic embrace of all things AI by just about everyone comes at a cost. We have been informed of the huge ecological cost of even a single query on AI models but we choose, as we often do with ecology, to ignore it because the convenience is great.

A college student querying ChatGPT for an easy fix to an inconvenient homework may not be thinking of their action as contributing to ecological or cultural damage, but if there is any lesson to be learnt from ecological disaster, it is that even when there is no single actor to hold accountable, and even if we plead innocence as individuals, we are all complicit and culpable.

In our embrace of AI as the next great achievement of human civilisation, let us not forget Walter Benjamin’s dire warning that there is no document of civilisation that is not simultaneously a document of barbarity.

Lawrence Liang is a professor at the school of law, BML Munjal University and director of the Centre for Law, Science and Technology