BERTIFIER

[msterslev]

I want to investigate the classic BERT model from a literary perpsective and NaNoGenMo seems like the ideal place to do so. I'll be using an approach I'm developing elsewhere, which I call BERTIFIER, to iteratively turn any text into another, BERTified, version of that text. It's a conceptually simple but (to me at least) fun experiment with the conditions of language and semiotics under LLMs.

The main project for my NaNoGenMo participation is the question of how to operate BERTIFIER within the context of the 50.000 words framework. If I generate text, is it then still interesting from the BERTIFIER perpsective? Alternatively, how do I even navigate it if I DON'T generate new text? Do I source it from Project Gutenberg or somehwere else? There's a nontrivial influence of the selected (or generated) source text over the finished novel, so this may be at the crux of the NeNoGenMo-version of BERTIFIER.

There's also something about the process of BERTification itself that can easily be lost if the finished model looks too much like a traditional novel. How do I preserve the process in the product? Is it possible to situate a generative palimpsest?

[msterslev]

I'm (ALMOST-ALMOST!) done programming the book, but ran into an annoying bug that is not difficult to resolve, but since it is now late in my time zone, I won't have time to generate the full book before November ends... I'll be back to upload book and code VERY SOON (hopefully tomorrow), but since this is a nights-and-weekends-project to me, I can't say for sure when it'll be possible for me to finish the project. I hope that's ok, otherwise I'll put it somewhere else on github.

[msterslev]

NOW DONE!!!

The full repo including code and data is here: https://github.com/msterslev/BERTIFIER_NaNoGenMo_2024

The updated description of the project is in the README: https://github.com/msterslev/BERTIFIER_NaNoGenMo_2024/blob/master/README.md

And the full book is here: https://github.com/msterslev/BERTIFIER_NaNoGenMo_2024/blob/master/When_Bert_met_Markov.pdf
(Of course, running the code again would result in a new book, so this is just one possible output.)