quickget_cigo: Merged all changes of major refactoring. Fixed parsing of a bunch of different linux distributions, worked around Microsoft limitations for Windows.
Next, a centralized JSON schema will be used for quickget_cigo and the rest of the quickemu-rs project with much more flexibility, to allow for better support for automating virtualization of systems like macOS.
Log in to leave a comment
quickget_cigo: Modify a few more operating systems to use the new mirror parsing structure.
Round out the initial implementation of centralized mirror parsing with SourceForge. Now, we can parse the majority of HTTP mirrors (apache/nginx) and SourceForge as a lazy loaded tree of files and subdirectories.
Later work will migrate as much as possible over to this new structure to improve conciseness. Then, work will begin on the main project task.
Log in to leave a comment
Quickget_cigo, the project responsible for gathering all operating system data to automatically create configurations, has had about 20 operating systems refactored from using raw HTML parsing (with regex of href objects) to using a centralized parsing logic for the patterns used by HTTP mirrors. This improves maintainability by keeping the few necessary regular expressions for this job in one place, and readability by allowing each function to fetch Directory objects with Lazy-loaded Subdirectories, Files, modified dates, names, etc, as opposed to manually parsing HTML.
Some other refactors have been made to its utility functions to slightly improve performance (however irrelevant, since the project exclusively runs in chronological GitHub Actions) and conciseness.
Further improve readability and generally improve functionality through changes to utility packages. Use generic union type (string, or *url.URL, or custom File type for certain cases) for URL input, rather than just string. Also pass client through each subdirectory in fetching logic, to no longer require passing a client into new functions. dir.Fetch(c) -> dir.Fetch()
Log in to leave a comment
Refactor a bunch of different operating systems in quickget_cigo to use the improved mirror parsing logic. Add support for apache & nginx which use tables rather than preformat
Log in to leave a comment
start work on refactoring quickget_cigo to centralize parsing through mirror structures. First with HTTP mirrors. Rather than manual parsing HTML with regex, specifically parse types of mirror into lazy-loaded directory structure. Heavily improve code readability
customHrefRegex.FindAllStringSubmatch(page) -> c.ReadDir(url), subdir.Fetch(c), etc
Log in to leave a comment
