14 devlogs
59h 45m 4s
redux.chat is an “actually good” AI chat app, with dozens of models, deep search, RAG. Competitor to t3.chat/chatgpt.
I spent forever fixing a bug that I introduced in a previous commit that was meant to FIX bugs lmfao
0
Log in to leave a comment
I’ve added a stats display to the llm’s response message. (tokens per second, time to first token, etc)
Also made a bunch of minor ui changes/fixes
0
Log in to leave a comment
Made stream resume more robust and also made the optimistic message sending even faster
0
Log in to leave a comment
Finally tracked down the source of the error over to better-auth… This marks the fourth issue i’ve encountered with better-auth in my time using it that’s taken over 4h to debug.
(The record is https://github.com/better-auth/better-auth/pull/4724, which took well over 10 hours)
0
Log in to leave a comment
I’ve decided to migrate to tanstack start. This is because the app is mostly run fully client-side, and I want a better client-side routing experience. The only real next features I use are server actions and api endpoints.
Currently dealing with a weird bug: https://github.com/TanStack/router/issues/5196
0
Log in to leave a comment
I actually managed to do it… I optimized the send message latency from ~750ms to ~130ms! And on top of that, I implemented optimistic updates. Little writeup on what I actually did (copy-pasted from discord):
i rewrote the backend, the bottleneck was I was inserting sequentially as it’s basically a tree, and i need a parent node’s id before i can insert a child node
the solution i came up with is so dumb
basically I don’t want the client (browser) to control the id I give to threads and messages, as that allows for the user to supply basically whatever string they want
so I instead generate 3 ids, and I cryptographically sign them so I can “attest” to them being truly random. These ids are then passed to the client on page load (and regenerated on-demand)
then when the message is sent to the database (convex), it validates the signature and inserts the required rows with the ids that the client already knows about
all of this is so that when enter is pressed, the client already knows the ids of the thread and the message, so it can optimistically route to /chat/ and add their message onto the screen, making it look faster
tl;dr: I did some dumb sh*t with pregenerating ids on the server to make the browser redirect and show stuff faster before the rows are inserted in the database
1
Log in to leave a comment
Comments
*I also shaved about 100ms off from the latency by modifying my auth system for convex functions to instead check the user’s jwt and not try to read their user record from the db on every request
Everything works, but it isn’t fast enough. At least it doesn’t look fast enough. Apps like t3.chat optimistically route the user to a new chat before it’s even created, so i’m going to do that too. However you can’t trust data created by the client, so i’m having the server generate and sign a id on page load, and that signed id is validated by convex when a message is sent, so the client and optimistically route itself to the new chat page before its even created.
PS. Antigravity sucks cursor GAPS it
0
Log in to leave a comment
My initial implementation of resumable streams was flawed and did not properly work. I spent two entire days debugging it and fixing it. Now it works!!
0
Log in to leave a comment
I’ve implemented a chat message input bar. It has token estimation, and a feature that highlights tokens. Additionally, any important chat state (i.e submitted, generating, error) is shown in the border of the chat input.
0
Log in to leave a comment
Did a bunch of work on basically everything, including the database schema, auth, and implemented the backend functions and routes, with resumable streams!
https://github.com/vercel/resumable-stream
0
Log in to leave a comment
I’ve implemented a core layout, and spent most of the time building the sidebar. It uses tanstack virtual lists (https://tanstack.com/virtual/latest) to dynamically load the chat history from convex.
0
Log in to leave a comment
Spent a bunch of time debugging weird issues with turborepo, and getting convex to work properly with better-auth (https://labs.convex.dev/better-auth/features/local-install)
Starting to think I should have gone with what i’m usually used to working with (postgres), or using something like clerk/workos
0
Log in to leave a comment
Spent nearly 4 hours figuring out convex and setting it up in a turborepo with better-auth. It finally works!
Currently trying to figure out whether to use convex ents
0
Log in to leave a comment