alexxino

I’m finally implementing queries. I wrote a query builder interface for it to feel modern and comfortable for users and I’m trying to correctly handle encrypted indexes. I implemented the entire query operation client-side.
I also refactored SCAN’s inputs and outputs and how tokens are encrypted and generated.

I’ve been trying to implement queries. They work by querying the indexes table with a client-side encrypted token, then reverse get the object through the object ID reference saved in the index document. I implemented the first type of query (hash-only index, plain equality), even though it still doesn’t completely work.
I also refactored SCAN’s inputs and outputs and how token are handled server-side (now a byte representation of the object ID is appended to avoid PK-SK pairs duplication issues).

I added a DELETE route (which works by soft-deleting items and tagging them with a 30 days TTL) and also a RECOVER route to recover deleted objects before the 30 days.
The DELETE operation soft deletes the object (by changing its status), but deletes its indexes (so that it can’t be searched), so RECOVER can only prevent the final object deletion, but won’t restore the indexes. In the future, they could be restored either by going through the enclave or by re-encrypting the indexes client-side.

I added a DELETE route (which works by soft-deleting items and tagging them with a 30 days TTL) and also a RECOVER route to recover deleted objects before the 30 days.
The DELETE operation soft deletes the object (by changing its status), but deletes its indexes (so that it can’t be searched), so RECOVER can only prevent the final object deletion, but won’t restore the indexes. In the future, they could be restored either by going through the enclave or by re-encrypting the indexes client-side.

I just found out that the JSON encode function can actually handle []byte slices (lol), so I just replaced all B64-encoded strings in requests (both standard client-server and server-enclave requests) with []byte and remove all the unnecessary B64 encoding/decoding. All tests are still passing!!!

I just found out that the JSON encode function can actually handle []byte slices (lol), so I just replaced all B64-encoded strings in requests (both standard client-server and server-enclave requests) with []byte and remove all the unnecessary B64 encoding/decoding. All tests are still passing!!!

I refactored how indexes are structured and, therefore, how schemas are initialized and handled under the hood. I made users able to create composite HASH+RANGE composite indexes (inspired by DynamoDB design) (before this, users just couldn’t query for multiple indexes at the same time without manual result aggregation).
Now, at schema creation, users have to pass both a gardb.Model struct, which includes the basic schema definitions, and a new gardb.Indexes that contains all the indexes that will be created alongside the object itself in DynamoDB.

I started actually working on the indexes and I refactored how they will work. Since DynamoDB and the zero-trust principle wouldn’t have made users able to query for multiple indexes at the same time without manual aggregation, I had to change how indexes are created in the first place to enable the creation of DynamoDB-inspired composite Hash(fixed length DET ENC)+Range(OPE->todo make it safe) indexes to grant a little more usability.

After doing completely rework of the PUT operation, I used the SDK’s integration tests to make sure everything was working fine and, obviously, it wasn’t. However I only had to fix some minor issues with versioning and dynamodb operations, and now all tests pass! (look at 08 and 09)

After the big PUT rework, I continued implementing the update functionality both with the PUT method and with a new UPDATE method (which, under the hood, just combines a GET with a PUT with some safety enhancements, since everything is encrypted client-side anyways). I implemented the actual UPDATE method and I had to fix some problems with versioning. Now all tests pass again (focus on tests 07 and 08)

I did a complete refactoring of how the PUT operation works.
Before, the flow was similar for small and large objects:

• Client sends object’s metadata
• Server creates metadata document in DynamoDB with TTL, sends back a presigned PUT url (directing to S3 if it’s a large object, or to another route if it’s a small object)
• Client uploads encrypted blob
• Server removes TTL
  However, this common approach was not optimized, as most of the objects will be small in size and could be uploaded with a single request. Therefore, I changed it to:
  Small objects:
• Client sends everything to Server
• Server uploads everything to DynamoDB
  Large objects:
• Client asks for a presigned put url (S3)
• Server generates the url
• Client uploads object
• Client sends to Server all the object’s metadata

Also, removing the PENDING state enables me to handle object versioning in an easier way, as I only need to put an object with a different version now to update it, while before I had to work with multiple pending upload requests and had to filter objects by state. This enables me to complete the PUT route (I wanted to add the possibility to replace an existing object through PUT) and fully create an UPDATE route.

I was able to make all tests pass after the PUT rework

First of all, sorry for the 6h devlog, I couldn’t have devlogged sooner and I’ll explain the reasons here.
I was trying to add a way to update existing objects and I wanted to take inspiration from how DynamoDB does it (PutItem to create or replace existing objects, UpdateItem to partially update objects). To enable versioning, I completely refactored how the PUT operation works, also optimizing how lightweight objects are uploaded (before, common flow of: client sends metadata -> server creates pending object and sends a presigned put url -> client uploads the object; now, for lightweight objects, everything is done in one backend call, while for large objects the client first asks for a presigned put url, then confirms the upload to the server and only later the metadata is stored in DynamoDB).
However, while I was figuring out how indexes should be updated on an object update, I noticed that they were currently encrypted (with det. enc.) using the object’s DEK as key. Obviously, this completely broke how index should work, since a user should be able to det. encrypt an index token and search for it across the entire table. To solve this problem, I had to create a new TableConfig table, which stores, among other table metadata, the WrappedIEK field, which represents a KMS-wrapped table-wide Index Encryption Key to use for consistent and deterministic index encryption.
This way, I solved both the versioning issue and the index encryption consistency issue.
To complete the PUT operation refactoring, I refactored how indexes are stored, adding a DynamoDB Global Secondary Index to enable a new Object ID access pattern (I needed to search for every index of a specific object).

Finally, after all these problems, I completed the refactoring, enabling users to update their objects and indexes, and in the picture attached there’s the proof of it passing all tests!

I completely refactored how schemas are created. Now they use generics to directly reference the struct from which they are generated, so that users now have compile-time type safety. Also, this way, I refactored the way GET and SCAN operations return their results, replacing unmarshalling data into a pointer parameter to just returning an array of structs.

I added more integration tests to the SDK, so I spent some time fixing all the backend-side issues with the existing code. I also refactored the SCAN operation so that it correctly implements pagination now (heavily inspired by DynamoDB).

First of all, I refactored how the SCAN operation works so that now paginating the results is way more straightforward (I took inspiration from how DynamoDB does it). I also added a lot more integration tests, even for edge cases and I’m fixing everything to make them all pass little by little. I also fixed a very bad issue with time values and integers that was happening while unmarshalling the unwrapped data into the object pointer after GET or SCAN and I fixed the SDK’s behavior when scanning empty tables or with 0 as limit.

I had to fix some issues with the GET route and I also added a SCAN route and all the tests for it on the SDK. I reorganized the object API structure, with these changes (inspired by DynamoDB operations):

• POST “/:table/:id” -> POST “/put”
• GET “/:table/:id” -> POST “/get”
• POST “/:table/scan” -> POST “/scan”
  Right now all the integration tests on the SDK are passing!

I had to fix some issues with the GET route and I also added a SCAN route and all the tests for it. I reorganized the object API structure on the backend, so I had to make some changes to the SDK too. Here are the changes for the API structure (inspired by DynamoDB operations):

• POST “/:table/:id” -> POST “/put”
• GET “/:table/:id” -> POST “/get”
• POST “/:table/scan” -> POST “/scan”
  Right now all the tests are passing!

After refactoring how the objects are stored on the backend (S3 -> hybrid DynamoDB + S3), I had to rewrite some parts of the API client so that everything connects as it should. Something is still not working, so the GET test is currently failing

I was thinking about adding some more operations like SCAN and then QUERY, but I came across a problem. In the GET operations, objects metadata was fetched from DynamoDB, then the server created a get presigned url for S3 and the client would have just downloaded the encrypted object from there. However, scaling up, this meant eventually creating hundreds of thousands of presigned url, creating both performance issues and security issues. So I decided to refactor how objects are stored: now, the client first declares the encrypted blob’s size; if it’s a large object (>100KB) it the server will send a put presigned url for s3; if it’s a small object (<100KB), then it will generate the metadata in dynamodb and send a url for the upload of the blob to DynamoDB inline. This way, everything becomes way easier to scale, without creating very big performance or cost issues.

I’m still fixing how everything connects, so the GET operation is currently failing

After the big refactoring, I started fixing all the bugs I could find by debugging the Go SDK tests. I fixed a few bugs, like an error regarding how hashing was handled, or some stupid mistake like writing all API keys to the tenants table instead of their own table, but I managed to fix everything and now all the tests pass again!

After refactoring, I went through all the unit and integration tests again to fix them. I first had to completely rewrite the tests, then I had to debug a little to fix all those minor code bugs I did while refactoring (e.g., I was writing all API keys to the tenants table instead of the keys table…). The screenshots show all the tests passing… again.

I spent a LOT of time figuring out how the SDK should work. First of all, since I refactored the entire Gardbase’s API structure, I had to refactor the PUT and GET flows.
I then refactored the relationship between the Client and the Schema objects, moving Schema to the gardb package and only leaving the Field struct in the schema package: this ensures that I can easily integrate the Schema struct within the gardb package, and leads to the biggest change I made - I moved the GET and PUT methods from the Client struct to the Schema struct. I figured that the Schema struct should’ve had the responsibility of handling PUT and GET, and not the Client (since this SDK is more similar to a mongoose-like interface than to a DynamoDB-like one).
I consequently had to change how the client and the schemas are initialized and the whole validation flow.
I also added a simple cache to persist table hashes and not having to call the backend to hash table names each time the client and schemas are initialized.

I refactored the API (again…) and extracted all requests’ and responses’ structs into a different package, so that I can import everything from the Go SDK repo. I also refactored how health checks work and spent some time fixing problems with the infrastructure in terraform. I added the logic to validate tenants and api keys with a middleware for secure access and refactored S3 key generation for consistency.

I completed the API refactoring by re-organizing handlers in internal/handlers, so I clarified all the requests and responses structs and the crypto and enclaveproto packages accordingly. I also fixed an issue regarding KMS’s policy in the infrastructure’s configuration, making it safer.

I did a complete refactoring of the API structure, moving all the enclave functions from the enclave proxy (which, in fact, was not a proxy anymore) to a service-like struct that I can reuse across all handlers, making communicating with the enclave much simpler and more straightforward. While doing the refactoring, I also completed the table hash generation logic, making use of the shared session key with the enclave.
I also refactored how KMS is called, moving recurring logic to a service struct in order to make the code easier to read and less error-prone.

I refactored the object’s model to not include EncryptedTableName (since it can be easily referred from context) and changed how the PUT flow works. Specifically, I added an enclave handler to hash table names so that the client just needs to send the table name encrypted with the shared session key to the backend and nothing gets leaked in the process

I spent a few hours solving some structural issues and figuring out how to authenticate tenant for the application to work as expected. I reflected on how to handle key hierarchy and came out with a 4 level approach:

• Level 1: KMS Key (AWS-managed) - one key per environment, used to encrypt tenant master keys
• Level 2: Tenant Master Key (TMK) - Random 32 bytes per tenant, stored KMS-wrapped in DynamoDB, used to encrypt DEKs
• Level 3: Data Encryption Keys (DEKs) - Random 32 bytes per object, stored KMS-wrapped and TMK-wrapped in DynamoDB, used to encrypt actual object data
• Level 4: Data, encrypted with DEK
  I also added tenant registration and API key handling

I figured out the project wasn’t really fully ready for shipment, especially from the usability side. Therefore, I set up a Docker image (so I added a way to run the app from Docker, without needing to install all the required dependencies) and I quickly coded a Gradio server to display a demo of the project. I also uploaded the repository to Hugging Face and set up Hugging Face Spaces to have a hosted demo online for users to check it out quickly.

Seeing the results on TikTok, I made a few adjustments to all the parameters for the video generator and now it has become a full blown generator. I also made some structural changes to make the code more clean and readable and fixed a few issues regarding subtitles timing. Below a cropped example of the generator’s final result!

I’ve been trying to implement how tenants are handled: so tenant registration, authentication, etc, following my application’s zero-trust model. I created a tenant config object in DynamoDB that stores tenant information such as encrypted master key (that, for now, I think will be provided to the user for requests authentication), a wrapped table salt, used to encrypt table names too, metadata like timestamps, version, IDs, key rotation information, recovery contacts. I’m trying to find the optimal way of handling tenant registration and then use through the SDK

I’m still working on making all the basic integration tests pass for the SDK and today I worked on the GET operation. I noticed there was an error linked to encryption, so I re-analyzed and debugged the flow to find out my crypto library wasn’t working as expected.

I had to debug and rewrite the section that integrated the library, update it and now it seems to work!

I’m still working on making all the basic integration tests pass for the SDK and today I worked on the GET operation. I noticed there was an error linked to encryption, so I re-analyzed and debugged the flow to find out my crypto library wasn’t working as expected and wasn’t opening sealed DEKs decrypted through the enclave service and re-encrypted with the derived session key.
Now everything seems to work as expected!

I’m seeing the first results on TikTok and I’m making changes to the format of the video according to all the analytics. I set up a new funny intro with an explosion sound effect and edited how audios are handled. I also implemented a function to add fading to the start and ending segments of an audio using Pydub.

I wrote a README and added a .env.example file for helping others to quickly set up everything.

I’ve been continuing to add more tweaks and quality-of-life features. For example, I added a way to build multiple videos with one call, or automatically fetch stories from Reddit just from the link. I also added support for some more abbreviations and replacements for curse words, so that the algorithm doesn’t punish the video and I don’t have to edit them manually.

I spent some more time adjusting the video’s parameters and I added a way to handle abbreviations (I had to work it out because of the TTS model). Now everything looks good and I already started publishing some stories on TikTok! If you want to see the results, check out https://www.tiktok.com/@massive_ideas

I made some more changes to the way the different parts of the videos are displayed and added support for abbreviations.

I added the RedditFrameImage class to scrape postfully.app tool (free, no login) and generate an initial fake reddit story image to display at the start of videos. I also enhanced audio generation by removing leading and trailing silence (to perfectly match all the timings) and I was able to sync every component. I spent some time adjusting all the required parameters and the styles of the video’s components and refactor some of the application’s structure.

I did some research for figuring out a way to generate automatic subtitles and I found out OpenAI actually has an Open Source model for speech recognition! I also looked for specific forced alignment models (audio + transcript -> timestamps), but I stuck with OpenAI’s Whisper model (the base version, to keep it fairly lightweight) as I did not find any good alternatives and it actually performed pretty well. Also, I added WhisperX to get word-for-word timestamps (to keep the engagement as high as possible!)

Now that I was able to generate CapCut drafts, I needed a way to turn some text (the stories) into spoken audio. I did some research (I looked for opensource, lightweight TTS models) and found the model Kokoro-82M, which satisfied all my needs and can be ran directly on my machine. So now I’m able to start from some text to generate the video’s audio and then create a draft on CapCut for a full video. I ultimately set up Kokoro TTS and adjusted some parameters for CapCut’s open API. Next step would be generate subtitles for the video, maybe through automatic speech recognition or forced alignment?

I finally found a library to work with CapCut video editor programmatically, so I created a first part of the program to generate a draft, add the background video, a voice audio and some subtitles and export it to CapCut. I set up the foundation for creating the actual videos and got my first draft working!

I created some integration tests to test the actual working APIs and check that everything works as expected… and it doesn’t, obviously. So I spent some time figuring out what was wrong, I rewrote some parts of the SDK that handled communication with the enclave and fixed some bugs with the actual API handling (like how multitenancy was handled client-side).

I continued going through the SDK’s tests to verify if every route worked as expected and fixed some errors regarding the creation of objects. Now objects are successfully created as they should and those tests are passing! I had to fix how multitenancy was handled and a bug regarding the enclave’s attestation document.

I applied the same fix to both decrypt and session_unwrap handlers. Now they both call the enclave’s route for get_attestation and use the attestation for decrypting with KMS. Then they return KMS’s CiphertextForRecipient field to the enclave, that in turn decrypts it with OpenSSL and re-encrypts it with the session key/client ephemeral public key, so that the proxy NEVER sees the plaintext DEK.

Note: the code highlighted in the images is not the complete updated code - the full commits are:

• https://github.com/QodeSrl/Gardbase/commit/cf668e8b980c0de19d5c603989fb2d594dacb38c
• https://github.com/QodeSrl/Gardbase/commit/ca481d41fcd59ccdcc820fb4247debcbdcbe55bc
• https://github.com/QodeSrl/Gardbase/commit/f53e6bbe8be6b535288cb3a661bf636fcdfd6754
• https://github.com/QodeSrl/Gardbase/commit/a285a5043b43dcc934b606e827cccf7e53b5d0d5

I fixed the way the server handles key generation by moving the logic from the isolated enclave to the proxy, using the enclave’s cryptographic attestation document to guarantee zero-trust.

• I SPEND 5 EXCRUCIATING HOURS TESTING HOW THIS KMS ENCRYPTION SYSTEM WORKS BUT FINALLY I GOT IT!!!!! Credits to this super helpful answer on stackoverflow:
  https://stackoverflow.com/a/77985003/21306952 -

The proxy then sends the results to the enclave, which securely decrypts the new keys and re-encrypts them with a public key provided by the client, so that the keys are cyphertext to the proxy, but can be decrypted locally by the client.

The enclave-proxy system now finally works as it should!!!!!

While also working on the SDK I’m noticing a few errors regarding the vsock communication between the main server (the API) and the Enclave service running inside of it. Also, I’ve been trying to fix how the enclave handles calls to AWS KMS to generate and handle Data Encryption Keys (DEKs), so I’m trying to figure out how I can make calls like those from an isolated environment. I’m studying the official AWS Nitro Enclaves SDK (only written in,,, C 😭) to figure out how to do the same thing in Go.

I’ve been adding more tests to the SDK and I’m trying to solve some issues related to the vsock communication between the parent application (my server) and the enclave app. Also, I’ve been enhancing the way errors are structured in order for users to have little to no trouble figuring out what is actually wrong.