Lattice: Devlog 1

Introduction

In the end, Lattice will have be a full-featured learning system.

The core ideas behind Lattice are:

• Engaging, interactive lessons
• Fraud/cheat detection & redirection
• Social learning via cohorts
• Gamification via collectibles & achievements

Current Progress

Course Content

I have created types for progression through modules & sections.

In addition, I added simple rich media support (see in the screenshot below).

There is support for getting assets from bucket storage, which should be useful later on.

“Knowledge” System

The idea behind the “knowledge” system is to track what a learner has learned.

In doing so, we also track all the content in the world, and what relationships exist between content.

I have typed out the basic structure for this.

Additionally, I made components & API routes for querying & creating knowledge attributes. Attributes are the different pieces of information attached to a knowledge item (e.g. ISBN for a book, or birthdate for a person).

Usability

Although not yet ready for use, all of Lattice has been designed with future usability in mind.

Lattice is not meant to be a standalone product. It is fully embeddable within other Svelte applications.

This means that Lattice’s UI is minimal & non-intrusive.

Screenshot

Current State

The game works!

LLM’s are told that one of them is a human imposter.

Well, there really is no human yet, as I still need to add the ability for a human to play the game too!

What I Need To Do

• Human interaction
• Improved TTS (considering using web GPU?)
• Add security to prevent misuse of the API endpoint
• Improve design

Echo Kit: Dev Log 1

Progress so far

Currently, I’m putting everything into a Svelte component called GeminiLive.

It’s currently capable of:

• Recording audio from the user’s microphone
• Sending audio data to the Gemini Live API

Wow, that’s not much!

Behind the scenes, though, a lot is happening:

• An AudioWorkletProcessor to capture audio data without blocking the main thread
• Downsampling and encoding audio data into the required format (Gemini Live is strict about this)
• A queue to send audio (and other data in the future) to Gemini in a controlled manner

And while you can’t hear Gemini’s response yet, when I speak into the microphone, I see beautiful base64 data coming back from the API, which I can only hope is a wonderful “hello” back from Gemini.

Design problems

The hardest part of this project seems to be designing the library itself. I’ve never really made a reusable component or library before.

1. Why am I using a component? Surely I could just make a class in a .svelte.ts file and export that?
2. Props are too intertwined
3. Empty audio data
   • Gemini wont respond if no audio data is sent. I need to handle this case.
4. Input & response modalities
   • Gemini Live is more than just audio. It can handle text I/O and video input too.

The UI…

The component itself has no UI yet, but I’ve added some basic checkboxes to the demo page. It’s real ugly though:

The Logo

Honestly, I had a lot of fun making the logo. I used Lunacy.

It was originally going to be a horizontally centered audio wave. But as I copied the rounded rectangles and made them each a different size, I liked the top-aligned look.

So, I went with it.