OpenRadar

Info

This is a tiny update, TODO.md with some stuff in which I am planning/not planning to implement in the future!
xD

Commits

• Create TODO.md (843ca46)

Info

Updated the remove old keys job so that it actually functions as it should!
Proper rate limits have been implemented, (so it queues as well and runs in coroutine for faster keys)!
Checks are alot better and now they should actually work this time with nothing weird (;

overall some good stuff for DB cleanup!

Commits

• Remove_Old_Keys rework, fixes! rate limits + better checking. (8aa026f)

Info

Realised after a verification check, that a error from a provider (500) could wipe an entire DB of keys! Along with ratelimits occuring, hence why I have disabled said job until it can be fixed. Added a-ton more providers to the checks. However some like Shopify, Twilio haven’t been added yet.
Thats all for now xD

Commits

• Disable remove_old_keys.go job since one singular 500 from a provider could technically wipe a DB if down. Added more providers to the checks! (efae35c)

Info

Added a new /api/stats API that returns the repository count, findings count and the uptime.
Also added said API to the documentation (https://open-radar.live/docs)
Created a new banner & Logo for OpenRadar!

About to ship so excited xD

(Counted time using lapse!!!)

Commits

• /api/stats added! (ae734a1)
• Upload README.md Banner (5c364eb)
• OpenRadar Logo 2.0 (b415ef5)

Info:

After my rework of worker.go, I had some commented out lines I had forgotten to remove (since I was looking back and forth at implementations of some stuff) I added some more directories I should skip during the scanning phase, and added some new tests! One for validating APIs return errors rather than succeeding with bad requests (e.g. negative time) and another for ensuring spam filters are working correctly.

Commits:

• Remove old worker.go lines! Add some more directories to skip. Added a test for both api_validation and key_test. (f68a35b)

Info

Fixed a JSON decode error where it would freeze the app by panicking!
Refactored the worker.go file (handles cloning, detecting, checks) into multiple files, and added more comments!
Updated README + Google.go check!
Thats all for now xD

Commits

• json decode fail = frozen (f0a0751)
• Add logging for github scrape fails (e.g. 403, 401) + adjust job length:: (29b2bfb)
• Fix README.md formatting. (7f896ef)
• Update google.go for better checking (8f63f71)
• Merge branch ‘main’ of https://github.com/officialmelon/openradar (658d5bb)
• Worker file refactor! (split file up + cleaner code + comments) (7f4adf2)

Info

Added websockets for when keys are added, and skip the node_modules folder when scanning!
In all honesty not that big of a devlog.

Commits

• Webhook support for discord! (fd7c004)
• skip node_modules folder (2566d26)

Info

Improved the look of the README and comprehension of self-hosting dramatically!
Added the /api/leaderboard documentation to the documentation page which was missing before now xd
Websocket origins are now active so third parties cant connect over to them (however origins are basically bypassable easily lmao) and they arent hardcoded (changeable in config)

Commits

• Make README more readable! (0825a76)
• Em-Dashes make the README look like AI! Remove them at once! (9482d56)
• Add /api/leaderboard to /docs; dynamic websocket (0b9f704)
• websocket origin allowed based on config file (not hardcoded); (30f6d3b)
• fix: missing comma between location.protocol (01b1047)

Info

Fixed some small bugs and whatnot, implemented trimming to fix the longer file paths going off the cards they were supposed to be on. Added something to find repeating chars (false positive keys)!
However TG is being mixed with AWS atm? Have to fix that asap lol
Added examples of false positive keys below

Commits

• fix panics, type + body leak on xai (cf5089f)
• check for PushEvent in service when scanning, fix possible caching bugs (702b9a4)
• remove finding job (6569042)
• repeating chars (b5b77ac)
• key trimming on frontend (837c49e)
• mutex fix (b56d356)
• Fix crash in main.js (87fb5fb)
• filepath (14ab5ce)
• filetrim substring/length change! (4a051ea)
• one more l change to 55 from 65 (72ff4aa)

Info

So turns out there was quite alot of bugs going on behind the scenes lol!
I had forgotten to add detection for OpenAI itself hence why no keys were detected.
Index.html was missing some of the providers (Mistral, etc)
Some of the regex was also broken :P
And that is pretty much it for now!

Commits

• more checks added! (96c2a53)
• fix: add openai keys to detectors (65baf50)
• change some regex, add tests, add more file ext (f4045d2)
• fix broken discord + test (b43f5ef)
• origin check (1ac5a2f)
• fix some large bugs affecting ft, dc and ant keys! fixed filter missing providers + misc fixes (aca76ef)
• filter tab (0ad6d01)
• patch openai regex bug (f3bda65)

Implemented some more of the checkers!

Info

I implemented a new package called checks which will check if an API key is valid (calls said providers API w api key) and then we save it to the DB if correct! Atm we only have Anthropic, Cerebras, Google, XAI, Groq, Openrouter out of around 20ish~ other providers! SO i need to implement some more later!

Its called using RunCheckForProvider by worker.go (worker that clones, checks, validates keys etc)

Commits

• work in progress; disable stripe temporarily (dfcc962)
• quick patch (512b02e)
• new checks function, 7 check files inc, need to implement more (1c94340)

• Flavortown API key (d815587)

Also attempted to add a Redis cache, however that failed horrendously lmao
Maybe another time.

• Change API route, New API documentation! (45ac7b5)
• documentation route (c0d94f1)
• documentation rename (ca06709)
• final docs update (0230575)
• update title (dcebb65)

Designed a mockup-esque UI and had opus complete it, then I wrote up some docs & did some misc fixes.

• Disclaimer (a3ce4ac)
• Click to copy to clipboard (2538d20)

• CF, GH, HF, NPM, PYPI, SHOPIFY, SUPABASE, TELEGRAM KEYS ADDED! (e4f5d06)

Alot of providers, added them based on a vote from someone!

• New provider types! + tests update (33f1936)

• Clean router implementation, split into files. Change api routes in README.md (92750b7)

• port added to cfg (0f6f1f9)
• remove redundant import from detector_test (7fef65c)
• redundancy (http_addr in env) (9c8a74e)
• .env fix with no env (2c62b27)
• Added dockerfile in order to make images for easier deployment (f354a33)
• Merge pull request #1 from jam06452/main (d10318d)

• Add comments to router, new /db/cache pkg, caching numbers of repo/findings every 30 seconds. cleanup some direct /jobs access (ed836c8)

Added a leaderboard!
Changed the routes to /api*

• Leaderboard update! Updated routes to /api* (e83d880)

• wuick change (8ca18d1)
• Discord keys (b606ac6)
• Memory Leak fix (Again, Again!) (427889e)

These memory leaks are insane! Debugging in Go is hell lmao

Programming from my phone rn, changed main.go to have the scan job inside a job files along with making them cleaner.

Release the job files!

Slack was added, tests were fixed.

Fixed API error with Google keys!
Added ticker that shows currently scanning repositories

• Opus 4.6 Suggested Fixes, Another 5 memory leaks removed! (3d45693)
• Scrolling provider bar on .tabs fix (ea8d802)
• ai frontend recode, added scrolling ticker and /live socket endpoint myself (: (5b95ce0)
• quick patch (e9567c1)

• webpage meta (49912aa)
• live time (d0914e8)
• Flexbox fix (f3c41d0)

• Update README.md (0f4408a)
• KB Max File fix (08029a7)
• Merge branch ‘main’ of https://github.com/officialmelon/openradar (05ad80d)
• wip (fcdb45c)
• Added jobs, remove_old_repositories that runs per 30mins. (a8dc524)
• quick patch (ad0bd30)
• scrolling fix (3c89718)

Large updates!
Setup a site at https://open-radar.live/

Created a Readme & some touchups!

• more touchups on the frontend! (d73db3b)
• Readme polishing! (971424a)

• Fix major memory leak (93d2aad)
• Base Frontend + OpenRouter (55f2ed8)

Created a cool frontend!

• new tests folder with detector test. (1ca1c08)
• App/Frontend Setup + Hosting from server (1a47ddf)

Setup frontend folder using vite + scss!

• DB now properly works, repositories now save! (7a13c6d)
• Gemini suggested fixes, Memory leak patches & quick key entry fix (65be6b9)
• Full API/Backend (be67a7d)

Cleaned up memory leak.
DB saves repositories that have been scanned.

• Github Credentials, MaxConcurrentClones works (30fd1e9)
• DB saving (b28bad3)
• Auto-Migrate fix (1f17a1f)

We can now scrape keys, search through projects, git clone into ram, search through and garbage clean it.
Also saves to our database!

• Scanning works! Fix: Regex (4f8c5b3)

Can now scan for API keys! Time to implement a API and whatnot.

• repo, repo size, scanning, queue (98566af)

Scans latest repositories!

• anotha commit (e4b4600)
• Database, config fix! (45b48db)

Setup a DB on a local server for testing!
Working on live git updates, then scraping logic.

• Setup directories & files for the project.
• .env.template added
• Configuration added!

First Golang project!