We’ve done some final touches on the project, and it’s finally coming together pretty nicely. While it’s in a finalized used state, there are tons of more features to add in the future.

Publish Site workflow, version 2

Transitioning from version one to two, I’ve made major changes regarding security (confidentiality, integrity), availability, usability (ease of setup), stability and features.

PHP

I’ve added another docker image dedicated to “php”, that adds PHP support seamlessly integrated into the container. It is very much optimized as usual

SSH?

Instead of using the previous HTTP backend I’ve switched to SSH to improve stability, security and ease of use. SSH only requires one config point for the backend and the client (GitHub Actions), a public key and a private key, generated seamlessly by my PKI script. The project is now as stable as it gets and really easy to configure! Time for you to publish a web application!

The old rust backend will be in a legacy unstable state, do not use it!

Exif Reader

I read EXIF data from user uploaded file and put them as table. It’s written in rust and compiled to WASM. For some reason it appears that strings enclosed by “” is not being saved in the vector, so we’ll have to fix that

Image conversion, DNS Lookup utility

One thing I’ve always wanted in the tools is a image converter, like PNG to AVIF, JPEG to webp etc.
Now, it’s finally done, and in a fairly simple and lightweight solution! I wrote it in rust, in a separate WebAssembly file to not slow down the other hash digest users. It’s about 1.3MB, which is a good size of package.
Most online converters use a backend that transcodes your files, however that’s bad for your privacy, and requires processing from the backend.
The few local online converters usually use imagemagick-wasm, however the binary size of that is about 14 MB, about 10x bigger than my binary.

I also wrote a DNS Lookup utility using Quad9 DoH!
We are still working on styling.

Patches, styling and WebAssembly using rust

Phew, we have continued on styling the website.

Previously hashing didn’t work for data in a specific size, however to increase hashing times and fix this issue I’ve rewritten it in Rust/WASM. This is working flawlessly, however I removed RIPEMD, SHA-224, SHA-384 as I deemed them unneccessary.

I’ve worked with these features today:

• More hashes
• Encoding and decoding
  I’ve also contributed to major improvements in the styling, but right now it is borked as we are moving to CSS grid to improve document layout. I also created the toolbar which took quite long time. My coworker helped with styling and continued on the unit conversion system. This means that we have these features so far:
• Archive conversion
• Unit conversion
• Currency conversion
• QR Code Generator
• Hash computing
• Encoder/decoder

We have come further with our project. At the moment we have these tools:

• Archive Conversion
• Currency conversion
• Hash Computation
• QR Code generator
• Unit converter

Me and my coprogrammer started working on the project. I implemented a local archive converter that at the moment supports

• tarball
• tarball gzip
• zip
  And I also made a publishment workflow that minimizes and publishes to github pages. My coprogrammer created the initial UI and is working on a unit converter. This is supposed to be a general purpose toolkit without advertisements and junk compared to other.

Patches

• I’ve fixed vulnerabilities mostly related to the backend and published a v1 release on the action.
• I also continued writing some documentation and the demo.
• Proper termination handling and a init system for the container.
• Docker/NGINX configuration environment variable for body size limit
• Security policies on repo

Features

• Minimal alpine image instead of Debian. You can still find the Debian docker image at debian tag.

To-Do

• Implement streaming body processing (DoS risk) [HIGH]
• Add optional features to container through configuration such as PHP [Low]
• Replace .unwrap() with proper error handling [Mid]
• Logging
• Verification of server authenticity (confidentiality) [Mid]

During this period of time, I’ve accomplished this:

• Documentation system
• Documentation
• NGINX Request Limit (2000MB, not env var yet!!)
• Various NGINX configuration trouble. This took a while

Writing documentation is boring, but it has to be done. You can find it at https://publish-site.rvid.eu/

We are almost done, and I’ve came far enough to confidently make a demo, with the API being on here.

• As you can see on this workflow, it works perfectly fine and as intended.
• It maintains confidentiality, security BUT not integrity and non-repudiation yet… This is a huge problem with simple fixes!

I deployed this on my local server with docker compose:

services:
  deploy-server:
    environment:
      API_URL: api.publish-site.rvid.eu
      CLIENT_CA: >-
        [REDACTED]
    image: ghcr.io/publish-site/backend:latest
    ports:
      - '443:443'
    volumes:
      - /etc/certificates/pub.crt:/etc/nginx/ssl/fullchain.pem:ro
      - /etc/certificates/pub.key:/etc/nginx/ssl/privkey.pem:ro
      - /mnt/SSD/publish-site:/var/www/html # Persistence

Phew… I’ve finally completed the difficult (but fun) part of the backend… Now, I mostly have docker/nginx configuration up ahead, and of course the actual workflow. I’ve successfully converted to Hyper and I use the Base64 crate, instead of the hacky and unreliable previous solution.
Instead of this:

• Create temporary directory
• Write Base64 to file using coreutils and bash
• Open decoded file
• Decompress
• Untar
  We have:
• Decode Base64, decode, untar
  in one go.

The API is in a working state, but at the moment insecure. I call the version: PRE-01-INSECURE

You feel so good when you’ve accomplished something on your own (:

Simple and seamless.

Cool! I’ve started rewriting the backend codebase to use the HTTP Library hyper! This will eventually make the application more stable and secure, and it will probably be able to handle larger and more requests. I also modified the docker building workflow, so now we have a docker container for all fellow ARM/Raspberry Pi lovers!

Ugh… Sometimes you just have bad luck when programming… I’ve been trying to fix this one single line, commit 34f7b4d832db0df3e47796c5543adbf6df62725c line 73 (https://github.com/publish-site/backend/blob/34f7b4d832db0df3e47796c5543adbf6df62725c/src/main.rs#L73) for an hour now. At this point, I plan on moving my backend to a lightweight web framework called hyper. I planned on using as few dependencies and crates as possible, however using the hyper web framework will likely make my program less susceptible to risks such as denial of service and RCE. I’d just need to rewrite the majority of the backend API, but it’ll be simpler.

Either way, I’ve still done some progress! I changed some entries in the docker container and made the example site contain links to relevant documentation.

YAHOO! I’ve finally completed the write part of the program, where it takes the site.tar.gz and decompresses the archive into a target directory! This means that the project can technically be used at this point of time, however with no security/confidentiality at all. I also used the base64 command from GNU Coreutils which should be temporary… I tried using the base64 crate but it was difficult. I also need to implement automatic removal of the files, but that’s pretty simple.

Woo! The docker container is finally published! I’ve created a workflow that builds and publishes to https://github.com/publish-site/backend/pkgs/container/backend.

Hello! For the last few hours I’ve worked on getting colors into my program, YAY! Of course it checks if it is interactive, so the potential docker logs doesn’t risk getting cluttered with ANSI. I also removed some annoying debugging stuff, so that’s good to have out of the way. I made the program print the user agent for logging purposes, I may include more stuff like content length and such. I also started creating a wiki for both the backend and action! This will make it easier for the user to spin up their server and workflow.

NOTE: I don’t know why, but no time got logged with my neovim. Switching to Code OSS temporarily (?)

Phew! This was a long one. I finally implemented the upload logic for the backend server, meaning that I can finally upload the actual files! There is a few rough edges on the actions workflow, and I still have a bunch of debugging lines on the backend, and it’s still not writing anything. Either way I’m getting closer to done. I’m gonna focus on the completion, so first the upload part, then the authentication part. Finally after that I’ll clean up. Wish me luck!

I continued working on the backend side of this project, and made a docker image server to be deployed on the server. I implemented HTTPS by reverse-proxying through NGINX. The certs in the docker-compose file are intentionally left in the commit so you guys can check it out easier. Next steps are to create the actual frontend to be published to, and to initiate mTLS through nginx. Should be easy enough.

I worked on the server-side part and has come up with a multithreaded HTTP server written in rust, see https://github.com/publish-site/backend. The next step is to implement HTTPS. I’m not sure if I wanna use this implementation due to potential security issues by user error (?) but I’ll try.

Today I worked on the base actions workflow. The workflow is executing a script so far. Haven’t worked any on the “backend”. I also created a example workflow implementation.