Activity

Wasn't Me

Shipped this project!

Hours: 13.71
Cookies: 🍪 122
Multiplier: 8.87 cookies/hr

My first time making and training an LLM. Definitely fun to see it improve day by day, but I ran into performance issues due to its small size. I also wanted to give it other capabilities, but TensorFlow is somewhat limited in terms of what I can do with it. I learned a lot from this project.

I want to redo this project, but with a custom NN library and a better device when I have more time.

Wasn't Me
  • added a feature where the llm generates 2 responses and has the user pick their favourite response
  • currently data is being stored but may potentially be used to train a “teacher” model which will allow for unsupervised learning

Probably gonna end the project here because of size restrictions. I might write a custom NN library from scratch and run it on a beefier device next time!

Attachment
0
Wasn't Me

Changes/Updates:

  • Updated LLM architecture (expanded capacity)
  • found and parsed new training data
  • found new testing data
  • increased training speed (active cooler on the pi!)

Notes:

  • had to retrain due to an architecture change
  • it might be a bit schizophrenic (sentient perhaps)
Attachment
0
Wasn't Me

Created the basic transformer architecture, added some basic Discord commands allowing me to interact and monitor with the bot. Currently training the bot on a Pi locally. (It is not very good at writing yet)

Attachment
0
Wasn't Me

I’m working on my first project! This is so exciting. I can’t wait to share more updates as I build.

Attachment
0