otter

Slack bot!!

Made a Slack bot that integrates with Otter to view people’s projects within Slack!

Check out #otter on the slack or run /otter to try it out!

Changelog

• feat: inaccurate search disclaimer
• feat: add github button
• feat: add dedicated channel
• fix: no projects handling
• fix: styling fixes
• fix: not in channel handling
• fix: add gzip support to backend
• feat!: slack bot project listing
• feat: slack bot

Blur hashing!!

Images are now stored alongside a blur hashed version that gets served to the client on the main search request that serves as a placeholder until the (VERY SLOW) airtable URLs load

Changelog

• fix: expanded image closing
• feat!: add blurhashes + progressive image loading

Tiny update

Added search suggestions and a text with total projects in the DB

Changelog

• feat: add temp favicon
• fix: add query data
• fix: back button
• feat: suggested searches + total projects

Feedback! (round 2)

• Added scroll pagination!!!!
• Added a comically large disclaimer
• Tweaked button positions on mobile
• Improved media handling/scraping

Changelog

• fix: make the warning even more obvious
• feat: add BIG projects disclaimer
• fix: add disclaimer
• fix: make pos absolute on mobile
• fix: theme switch position on mobile
• feat: add last refreshed at text
• feat!: better media handling
• feat!: add scroll pagination
• fix: preconnect to airtable assets url

Feedback!!!

Based on people’s feedback, I’ve improved several things like:

• Added literal results to the search algorithm (searching for projects should feel more consistent)
• Made hour formatting more consistent
• Improved the README.md
• Score display now has colors
• Low score entries are now hidden with a button to show them
• Saved current view mode in query parameters
• Added an endpoint for recently added projects
• Speed improvements
• Stopped using media URLs from the Ships API (to avoid expired URLs from being served)

Changelog

• fix: speed up literal candidates subquery
• feat: add project share button
• fix!: don’t use media URL from the ships api
• feat: recent endpoint + code cleanup + /api/v1
• feat: add recent projects endpoint
• feat: save view + apply score color
• fix: better score display
• feat: hide low score results
• fix: add demo url to literal results
• fix: clickable title
• feat: literal results + score colors
• feat: improve readme
• feat: redirect to frontend + improve README
• fix: more consistent hour formatting
• fix: fallback to homepage + add archived urls

Finishing touches

• Improved the README
• Fixed inconsistencies in the hour formatter
• Fixed back buttons + missing data

Changelog

• fix: improve readme
• fix: higher quality hero image
• feat: redirect to frontend + improve README
• feat: readme draft
• fix: more consistent hour formatting
• fix: fallback to homepage + add archived urls
• feat: improve back buttons

Cool stuff/QOL update

• Added OG tags! You can now share projects in the Slack with pretty embeds :D
• Added markdown rendering to the descriptions
• Fought against Cloudflare pages for an hour trying to figure out why everything broke (it was the HTML sanitization library )
• Improved the /project/[id] page and tweaked the image display

Changelog

• fix: add project not found embed
• fix: truncate description in embed
• fix: header tags
• fix: use ‘xss’ for html sanitizing
• fix: use the correct url 😭
• fix: enable observability
• fix: add wrangler file
• fix!: switch to cloudflare adapter
• feat: SSR og tags
• fix: description rendering
• feat: markdown rendering + better image layout
• feat: add og tags

More changes!!!

• Added username and project name guessing based on the repository URL
• Made search faster by over 10x!!
• Made the site responsive on mobile
• Made NULLs appear last in queries
• Added pagination to the search endpoint
• Added query timings
• Improved logging and fixed assorted bugs/glitches

Changelog

• fix: search data + double search + error logs
• fix: use actual length
• feat: add search pagination
• feat: use inferred data directly from the db
• feat: search endpoint query speedup + NULLS LAST
• fix: page title
• fix: improve mobile responsiveness
• feat: text input clear button
• fix: improve search endpoint
• feat: guess name from code url + table styling

It’s devlog o’clock!!

• Added user searching! You can now look up projects from a user by typing in user:username
• Improved username and project name guessing based on the code URL
• Improved the look of the big projects table (also added more fields to it)
• Added a way to share search queries using URL parameters

Changelog

• feat: guess name from code url + table styling
• fix: remove useless filter items
• feat: add more fields to table + better name guessing
• fix: numeric filter parsing
• feat: user search
• fix: search views
• feat: query in query params

Refactored the old temporary explorer in SvelteKit!

Featuring:

• Filtering by any field
• Filtering using multiple operations for each field
• Pagination with variable page size
• View sharing

Changelog

• fix: tiny style fixes
• fix: styling + filter tweaks
• feat: jump to page
• fix: remove unused explorer file
• feat: add view sharing
• feat: save search + custom row limit
• fix: improve styling and table
• feat: granular search
• feat: better data table
• feat: add project page
• fix: explorer

Finally started working on a proper frontend with SvelteKit !
Currently, only the search API endpoint has been implemented so it’s still missing quite a lot of stuff

Changelog

• feat: add dark mode
• feat: add image expanding
• feat: add spinner + image component
• feat: add more data to views
• feat: add frontend ci + improve backend ci
• refactor!: add sveltekit frontend

Separated data fetching into 2 separate jobs, so data is now updated every hour instead of every 3 hours!

• Added pagination
• Improved parsing for null/invalid records

Changelog

• fix: explorer + increase timeout
• feat: improve explorer
• feat: add query pagination
• fix: timestamp parsing
• fix: separate screenshot url updates
• feat: upsert airbridge projects
• feat: split airbridge and ships jobs

A ton of stuff again!!

• Added documentation using utoipa and Scalar
• Added observability using OpenTelemetry
• Added true hours syncing using Airbridge
• Added auto rebuilding
• Added a temporary frontend
• Improved the request retry logic

Changelog

• fix: move ysws list endpoint
• feat: add OTLP observability
• fix: endpoint nesting
• feat: temp explorer frontend
• fix: make embedding generation less verbose
• fix: also retry non http errors
• feat: add docs + better query types
• feat: add true hours from airbridge
• feat: add build.rs

I KEEP FORGETTING TO MAKE DEVLOGS

So uh hi, big devlog time
(still no decent images because the frontend isn’t done )

• Got mixed local/API embedding working with fallback handling
• Sped up the data ingestion pipeline/job with some query optimizations and by separating embeddings into their own table (turns out having 1024-dimensional vectors in every row makes insertion/updates slow)
• Added embedding caching to reduce load/AI API usage
• Added full text search
• Added an image endpoint (gets the updated image URL from the DB, this is needed because Airtable image URLs expire very fast)

Changelog

• feat: separate embeddings into separate table
• refactor: improve the query endpoint
• fix: tsv generation + improve fetch job
• fix: faster screenshot updating
• feat: add full embedding cache + api/local mixed mode
• feat: add screenshot url endpoints
• feat: add temp query endpoint
• fix: update screenshot urls separately + downgrade model
• feat: add embed generation cache
• feat: add temp test search endpoint
• feat: add req retry + programs endpoint + api client caching
• fix: job log text
• fix: speed up data ingestion + api dimensions
• feat: add embedding handling

Set up a base for the project and started working on data embedding.

Current progress:

• Data fetching
• Embedding (local/API)
• Search API
• Frontend