A collection of thoughts, findings, and projects
In this experiment, I wanted to explore how different generative AI architectures trained on my writing would generate text. I didn’t expect any of the writing to be good but I wanted to see:
1. How AI would transformed my writing
2. What variations would be seen in the different models
3. If any of the outputs contained interesting ideas that could be used if I wanted to continue my writing
All of the models below were trained on a partially completed science fiction book I wrote almost 10 years ago.
Ura was quite different than Thaileel had envisioned. There were no gentle breezes or blue skies-- only a dark, frigid haze. Beneath the haze lay thick pools of black ice, black like soot with frozen mud and rusted sand smeared across the surfaces. There were veins of white ice spread throughout the tundra that bled into the sand like mold festering on the meat of a dead animal. With each step, Thaileel could feel the shiver of the land’s decrepit body. The air was so dry and so bitterly cold that it stung Thaileel’s throat like burning sandpaper.
Some bands dedicated themselves to only delving into the woods in the night.
Without the help of Satia, traversing the woods in the dark end of the planet Earth as it was also the tool our ancestors used to because, on Old Moon every every statue and picture was always quick to clarify to Thaileel that night.
When Thaileel asked with surprise and explained with a grand smile and tired eyes.
They affect the flow of time on board other than Santi and Thaileel, then sipped some water from one branch descended into the sand swells was just one small aspect of Jinn.
There was no question in Santi’s mind that The Cartographer stopped painting and looked The Cartographer could do was struggle to breath and looked up to say goodbye or to explain herself to anyone.
Slender trees sprung from the central axis contained 0.7 Standard-Level Gravity and served as a worthless idolatry of the galaxy around him.
He forced himself to rely on Emile’s art to tell them that Thaileel and everyone else on the meat of a somewhat cramped tube.
Thaileel loved the way Mara stared out at Andromeda, at the tiny creatures that had devoured their parents.
He turned to him.
Was he really only knew Hector and Hector’s gazing eyes as he climbed the ladder strained their arms to the Raven King but only bound his own family to think about this.
He thought of his hand.
Rather, it was that the ship and splintered his eardrums with sharp reverberating shrieks.The ship then let out a pop as a voice called out.
Markov chains are interesting in NLP because they generate pretty interesting (though usually meaningless) sentences but the algorithm behind them is pretty simple – given any 2 consecutive words in a text(in this case my half finished book), choose the most likely third word, and repeat the process with the last two words in a sentence.
Markov chains can get stuck in a repetitive loop. For example, they may come up with ‘the ship was the ship was the ship…’. To get around this, I used the Python Module Markovify which avoids these loops.
Mara wasn't sure
he cartographer’s semled that the cartographer was a mand of the ship was she same spol and the cartographer’s semled that he was a mand of the ship was she ship was she ship was she ship was she ship
This neural network focuses on predicting individual letters instead of words and, one iteration at a time, it creates a story letter by letter. One drawback of this particular neural network is that it can get caught in a repetitive loop like those mentioned above.
Mara wasn't sure if Mara was still with them or not. The two men had only just left a small hatch closed on the other side of the ladder, but Thaileel was sure that all who entered the hatch could only see a large, immobile pile of rubble. He couldn’t see where the debris was coming from but could feel the weight of the ship shaking them. Thaileel wanted to move but couldn’t because of the large pile of rubble.
They were already shaking The Cartographer’s body so much that he was shaking. Mara had placed a hand on Thaileel's head in shock. She began to panic, desperately trying to wake him. She pulled him closer and told him to step out of the way so he could step back down to sleep. She told him to rest and to make room in his pod. They climbed the ladder and climbed the small hatch leading to the hatch into the other room. Walter took Thaileel by the hand and led him to his quarters.
The hatch opened, revealing a large dark room filled with tools. The interior had been carefully made to look familiar and as though it were from another galaxy or galaxy in the past. Inside the chamber was a chamber with a large metal hatch with sharp wooden beams running from one end of the interior to the other, creating an artificial gravity-based chamber. The chamber itself was entirely made of glass. The only damage it could have done to Thaileel was that it would have been difficult for Thaileel to see or use. The only light from the chamber was from a fluorescent light source placed right beside the hatch. It was like the hatch from the future, except that the machines could not read Thaileel’s body language and could only guess that he was either dead or unconscious. Even without the artificial gravity in the room, Thaileel could still hear the conversations coming from around the chamber.
As they entered the hatch into the other chamber they couldn’t help but wonder about the other men, wondering what their purpose was. They weren’t leaving the pod, they were looking for Mara. They could not make sense of all of this-- they were only leaving to make room. The rest of the crew was asleep. They were on board a ship with no windows, so how could anyone have arrived in their ship without windows? What if something terrible had happened? How could the others ever have arrived without windows?
Going beyond an RNN with my text alone seemed like a futile attempt (I think I’d need a much more expensive computer!) so I decided to fine-tune a few models. GPT-2 was my first exposure to AI generated text and the first large language model that I ever fine-tuned.
Mara wasn't sure if she would or not. As if to keep things quiet, she didn’t answer.”
“Yes, but you’re not alone.” The Cartographer responded as he listened, taking Thaileel by the sleeve and placing his hand on hers. The two boys followed behind Shevara and Emile on their ship to wait.
“We’ve come a long way, then, from what I'm told. I wonder how we could have waited such a long time, could we?”
“What is it, then?” She stammered. “We came from Nova-- Nova is a word-- Nova is our destiny and it will live forever and ever. It is not a lie that I admit--”
“Well, there is no doubt in my mind that the Nova family has set out to lead a peaceful and honorable life. I know that some of you disagree with my decision and others disagree with my decision as well, but that doesn’t mean that our destiny cannot come from somewhere deep within our planet.”
“No, nothing is absolute. It can only be written.” She responded, a quiet but deep breath. “Perhaps a little too harsh.”
LLMs have come a long way since GPT2. I wanted to train a newer model but my GPU couldn’t handle training larger models. Fortunately it could handle TinyLlama, though I was a bit underwhelmed with the oftentimes repetitive results.
© Copyright Clayton Spencer 2024