DONEE

Refactored the whole script into a proper package structure. Wrote the pyproject.toml and set up the freq console script entrypoint so it boots directly into FreqApp().run(). Hardcoded the Hugging Face SERVER_URL straight into app.py so users can instantly connect. Deployed the server to hugging face spaces via docker.

🪲🪲🪲

Left the app running for a bit and noticed memory climbing. The audio deques were staying alive forever even when a user disconnected. Added a garbage collector in the pull_mixed read path. If a deque gets fully drained, the sender’s id is appended to an empty_keys list and then deleted from the _streams dict.

Textual has some weird ui desyncs. If someone changed their nick via /nick, the status bar wouldn’t update immediately. So had to force immediate UI refreshes.

🦗🦗🦗

Ran into a super annoying echo bug. Since the relay blindly broadcasts to the room, you hear your own voice a second later. Instead of making the server track users, I generated a random 4-byte _session_id using os.urandom on startup. Injected that into the encrypted payload envelope [type][session_id][nick_len][nick][data]. Now clients just drop any packets that match their own session ID.

Also, Noticed terrible latency spikes if someone lagged out on their connection. I wrote a custom jitter buffer safeguard in the push_chunk method. It checks if any sender’s deque exceeds MAX_BUFFER_BYTES (which I capped at 500ms, or 16000 bytes). If it does, it aggressively drops chunks via popleft() until it catches up. Keeps the real-time feel intact.

Tested the audio with two clients and it sounded like a broken robot. You can’t just play overlapping audio streams sequentially or the buffers crash into each other and delay indefinitely. I realized I needed an actual mathematical mixer to handle multiple people talking at once. Wrote the skeleton for an AudioMixer class that stores incoming chunks in a dictionary of deque objects keyed by the sender’s session id.

Also, Brought in numpy to handle the actual audio mixing. I pull the raw bytes from the deques, convert them to int16 arrays, but then cast to np.int32 before summing them up. This prevents digital clipping when loud streams overlap. After the accumulation, I run an np.clip between -32768 and 32767 and cast it back down to int16.

Audio BS

Tested the audio with two clients and it sounded like a broken robot. You can’t just play overlapping audio streams sequentially or the buffers crash into each other and delay indefinitely. I realized I needed an actual mathematical mixer to handle multiple people talking at once. Wrote the skeleton for an AudioMixer class that stores incoming chunks in a dictionary of deque objects keyed by the sender’s session id.

Async

Getting async websockets to play nice with blocking audio streams took wayy too long. miniaudio fires its _on_audio_chunk callback from a background C thread, which obviously crashed when i tried to await a websocket send. Had to wrap the wire transmission in asyncio.run_coroutine_threadsafe and pass it the main event loop reference to finally get raw bytes transmitting across the relay.

Audio

The audio situation was a massive headache. Looked at a bunch of libs and went with miniaudio cause cross-platform I/O is slightly less broken there. Configured a miniaudio.CaptureDevice to pull 16kHz, mono, SIGNED16 samples. Hardcoded buffersize_msec=100 so it streams in small 3.2KB chunks to bypass standard websocket payload limits.

Server

Built the server side with websockets and asyncio. Its just a zero knowledge blind relay. Kept the state simple: a dictionary mapping the freq_hash bytes to a set of connected websocket clients. When it receives a data packet, it just iterates through the set and fires off the raw bytes to everyone except the sender.

Encryption 2

Ued AESGCM for the actual symmetric encryption. Every payload gets a random 12-byte nonce (number used once not a pdf file) attached to the front of the ciphertext. I also realized i needed a way for the server to group users into rooms without knowing the AES key. So ended up taking the SHA-256 hash of the AES key to use as a public freq_hash. This way the server routes packets based on the hash, but can never decrypt the contents.

Encryption

Mapped out the e2e encryption next. Brought in the cryptography lib and decided to use Argon2id to derive a 32-byte AES key straight from the frequency string. Passed in "freq-flavortown-2026" as a hardcoded salt. Since Argon2 is intentionally slow, I had to push crypto.derive_key into loop.run_in_executor so it wouldn’t block the main textual async loop.

Talkie

I originally wanted a walkie talkie style hold to talk, but found out terminals dont natively support key release events. So I had to turn it into a hard toggle bound to Ctrl+R. Hooked into textual’s on_key event, threw an event.prevent_default() in there, and got it toggling a boolean _recording state for the mic.

Freqqy 😩

Started sketching out the architecture for freq today. I wanted a clean TUI that looked good, so I went with textual. Ripped out all the heavy borders with some custom CSS in the FreqApp class to make it pure monochrome. Got the basic layout using a RichLog for the chat history and an Input for the /tune command.