Get search results from duckduckgo in cli quickly.
GitHub Copilot for code completition and code review, fix minor bugs.
Get search results from duckduckgo in cli quickly.
GitHub Copilot for code completition and code review, fix minor bugs.
I created a utility to quickly retrieve DuckDuckGo search results from the CLI. While tools like SearchSploit help identify security vulnerabilities, I wanted to create a version of such a tool for everyday use. In particular, I was always concerned about the high data charges incurred when using a web browser to search Google while surfing the web on a mobile data network, so I set out to solve this problem. Therefore, while other similar tools like ddgr crawl html.duckduckgo.com, I designed my tool to access lite.duckduckgo.com—DuckDuckGo’s lightest version—to retrieve search results, thereby minimizing data usage. The most challenging part was parsing HTML using only net/html instead of goquery, which I implemented with the help of Copilot. As a result, I successfully built a tool that fetches DuckDuckGo search results in plain text or JSON format in about one second, depending on network speed. I’m proud that, despite using only a single dependency, it outperforms the well-known tool ddgr in terms of CPU/memory usage and speed. Since it’s written in pure Go, it can be used immediately on various platforms without any modifications.
Translated with DeepL.com (free version)
I’ve added the ability to set DuckDuckGo’s URL parameters as flags, and enabled JSON output via the -json option. Also added -version flag for get version info. You can install it on your system immediately using the go install command. I’ve written detailed documentation, and it seems to be complete now.
Log in to leave a comment
I removed two dependencies (urfave/cli and goquery) and replaced them with the standard library and golang.org/x/net/html. I also automated deployment using GitHub Actions workflows. I checked using /usr/bin/time, and it’s faster and uses fewer resources than ddgr, another similar utility. Attached video shows you /usr/bin/time’s results(ddgr-2.2 and ddg-cli)
Log in to leave a comment
We have implemented the search function, which is an essential basic feature. You can specify the number of search results to retrieve using the –limit flag (up to 10 results), and you can view a shorter list of results using the -m flag. I spent several hours analyzing all of DuckDuckGo’s specifications, but unfortunately, it wasn’t recorded.
Log in to leave a comment