Story writer

So a little bit of changes of plans. So my project got rejected cause the Demo couldn’t be a google colab page, but I never saved my my model (kinda dumb, right😁😭). So the only solution was to train it again. And I also changed some things with this occasion: Not much but modified learning rate and Input size and number of epochs (sessions of training). After all the training and modifications my AI did better in testing than the previous version:
Like the sentences now are meaningful (but not so related to each other, like in a real story), but with adjustments and training it more times I guess it will make good stories ^_____^
I also launched it on Hugging Face (find link in readme). There you can talk to my SLM (small language model)

Use: Give him a sentence without period and he will continue in a story (if it can be called story….More like random propositions, with a slightly relation between them).

Also look at release to find a portable app, kinda of a offline version.

I officially finished it I trained the AI.
Here I won’t get a lot in detail, cause it is too advanced, but I can explain some things:

1. The GPU is faster than the CPU at small tasks. So using CUDA a platform for parallel computing made by NVIDIA, I can send most of the training part to the GPU.
2. I declared the optimizer. The optimizer modifies weights in the Neuronal Network. Weights are learnable, numerical parameters that connect neurons, regulating the strength of signal transmission between them (importance, in normal language).
3. I also declared some warmups steps for my AI. Also my AI will have a higher learning rate at start and a smaller one at finish. That helps the AI to fine-tune when he already has a lot of info.

That’s all hope you learned something about AI!!!🥳😝
P.S. I also published the code on Google Colab and Github repo.

I declared my GPT style arhitecture for my NN (neuronal network).

Let me explain you how it works:
Sorry cause I couldn’t write all here cause of the character limit:
You can see it here: https://pastebin.com/vs8KZyJd

See you when the model will be trained!!!

I added Input-Output pairs. You might wonder what that is. Let me explain: Input-Output pairs are sequences of words that the AI uses to learn what comes next. The AI doesn’t “think” like a human—it looks at all the sequences it has seen during training and learns patterns to predict the next word.

Let’s take a closer look:
a) Given the dataset:

Anna lives with her mother and her father.

Joe lives with his grandma.

Maria lives with her cousins.

b) If the AI sees the sentence:
“Joey lives”

c) It will try to predict the next word based on patterns it learned:

It has seen sequences like “lives with her mother” and “lives with his grandma.”

Based on these patterns, it predicts probable next words (“with his grandma”) rather than combining all sentences into one.

So the AI might continue the sentence like:
“Joey lives with his grandma”

Over many predictions, it can generate longer sentences that follow the patterns it has learned from the dataset.

This way, the AI uses Input-Output pairs to learn likely continuations instead of memorizing or merging every sentence. 😁

P.S.This is especially used in calculating the loss.

I registered my database from HuggingFace (TinyStories) and I tokenized the splits.
Tokenizing is the main reason AI can understand words. It splits the word in smaller words, based on the frequency of the in the given dataset:
Like if we have the names:
Joe
Joey
Joseph

It takes this three tokens: “Joe” and “Joey” and “Joseph” and “Jo”

“Jo” is there cause it has a high frequency.