Silo

I’ve made the server router be more like a builder pattern, and added support for (async) callbacks for expiry and public ACL

I’ve begun polishing up this project to be shipped. I’ve redesigned the projects page, made project slugs unique, and added it to the create project dialog, and also added the project list to the main sidebar.
Also made a bunch of other misc changes

Added file expiry/TTL for uploaded files. File validity is checked on download, and they’re lazily deleted by a cron job on the cf worker every 30 minutes.

I just implemented the SDK and did some major refactoring of the TUS implementation. There are 4 packages: @silo-storage/sdk-core, @silo-storage/sdk-next, @silo-storage/sdk-react, and @silo-storage/sdk-server

The Core SDK implements the core functions, like URL signing, different API request helpers etc…
The Server SDK implements the uploadthing-like file router ergonomics, including handling callbacks etc…
The Next SDK helps adapt the Server SDK to specifically nextjs, like creating the route handlers, etc…
The React SDK implements the React hooks like useUpload() and unstyled upload buttons/dropzones.

I did use AI to partially generate some of the SDK code, specifically the server SDK. This is because writing the typescript types would be very hard and cumbersome (see image 4)
I also used AI to quickly create an example nextjs app to demo the SDK. I plan on rewriting this part later.

Finally, while testing I ran into issues with my TUS implementation. Specifically with the upload resuming. Before, the worker stored all the metadata into KV, but after looking at Signal’s TUS worker implementation, I decided to refactor the TUS handler routes to instead use Durable Objects instead of storing state in a KV. This helps make recovery and keeping things tied together easier. Did use some AI to help with the migration.

Right now, it uses TUS chunked uploads, but i’m looking into streaming it

Built out the webhooks. It uses vercel queues to dispatch the webhooks with retries etc…
Webhook events are signed with a signing secret provided in the ui when creating the webhook.
(the ui is bad right now, but i’ll work on it later)

I’ve revamped the environment system. Now each developer gets their own dev env, makes stuff easier. Also added a environment selector in the sidebar.
Finally, to handle deletion of environments with possibly tens of thousands of files, the worker offloads the deletion task onto cloudflare queues, which provides a durable way of ‘queueing’ the deletion of these objects. Not sure if that makes sense, it’s very late and I want to go to bed :p

Revamped the dashboard, and also added a time range filter to the analytics page.
Also a bunch of other stuff that I forgot about. Spent some time implementing a thing I forgot I already implemented on another computer :/

I’ve made it so that client SDKs can self-sign upload URLs without needing to hit /api/v1/upload for a presigned url.
Also API keys are no longer req’d to upload a file via the ui. It’s handled behind the scenes.

Added file/bandwidth analytics, and also revamped the sidebar + mobile support

The TUS protocol is now properly implemented, had to go around fixing some bugs etc…

file info ui

I’ve implemented the TUS protocol. It’s essentially a protocol for resumable file uploads, it’s pretty cool!

I’ve bootstrapped more of the project, added scoped API keys, etc

I’ve bootstraped a new project using turbo-kit. I’ve also set up a cloudflare worker microservice, and set up organization provisioning