Publish Site workflow, version 2

Transitioning from version one to two, I’ve made major changes regarding security (confidentiality, integrity), availability, usability (ease of setup), stability and features.

PHP

I’ve added another docker image dedicated to “php”, that adds PHP support seamlessly integrated into the container. It is very much optimized as usual

SSH?

Instead of using the previous HTTP backend I’ve switched to SSH to improve stability, security and ease of use. SSH only requires one config point for the backend and the client (GitHub Actions), a public key and a private key, generated seamlessly by my PKI script. The project is now as stable as it gets and really easy to configure! Time for you to publish a web application!

The old rust backend will be in a legacy unstable state, do not use it!

Patches

• I’ve fixed vulnerabilities mostly related to the backend and published a v1 release on the action.
• I also continued writing some documentation and the demo.
• Proper termination handling and a init system for the container.
• Docker/NGINX configuration environment variable for body size limit
• Security policies on repo

Features

• Minimal alpine image instead of Debian. You can still find the Debian docker image at debian tag.

To-Do

• Implement streaming body processing (DoS risk) [HIGH]
• Add optional features to container through configuration such as PHP [Low]
• Replace .unwrap() with proper error handling [Mid]
• Logging
• Verification of server authenticity (confidentiality) [Mid]

During this period of time, I’ve accomplished this:

• Documentation system
• Documentation
• NGINX Request Limit (2000MB, not env var yet!!)
• Various NGINX configuration trouble. This took a while

Writing documentation is boring, but it has to be done. You can find it at https://publish-site.rvid.eu/

We are almost done, and I’ve came far enough to confidently make a demo, with the API being on here.

• As you can see on this workflow, it works perfectly fine and as intended.
• It maintains confidentiality, security BUT not integrity and non-repudiation yet… This is a huge problem with simple fixes!

I deployed this on my local server with docker compose:

services:
  deploy-server:
    environment:
      API_URL: api.publish-site.rvid.eu
      CLIENT_CA: >-
        [REDACTED]
    image: ghcr.io/publish-site/backend:latest
    ports:
      - '443:443'
    volumes:
      - /etc/certificates/pub.crt:/etc/nginx/ssl/fullchain.pem:ro
      - /etc/certificates/pub.key:/etc/nginx/ssl/privkey.pem:ro
      - /mnt/SSD/publish-site:/var/www/html # Persistence

Phew… I’ve finally completed the difficult (but fun) part of the backend… Now, I mostly have docker/nginx configuration up ahead, and of course the actual workflow. I’ve successfully converted to Hyper and I use the Base64 crate, instead of the hacky and unreliable previous solution.
Instead of this:

• Create temporary directory
• Write Base64 to file using coreutils and bash
• Open decoded file
• Decompress
• Untar
  We have:
• Decode Base64, decode, untar
  in one go.

The API is in a working state, but at the moment insecure. I call the version: PRE-01-INSECURE

You feel so good when you’ve accomplished something on your own (:

Simple and seamless.

Cool! I’ve started rewriting the backend codebase to use the HTTP Library hyper! This will eventually make the application more stable and secure, and it will probably be able to handle larger and more requests. I also modified the docker building workflow, so now we have a docker container for all fellow ARM/Raspberry Pi lovers!

Ugh… Sometimes you just have bad luck when programming… I’ve been trying to fix this one single line, commit 34f7b4d832db0df3e47796c5543adbf6df62725c line 73 (https://github.com/publish-site/backend/blob/34f7b4d832db0df3e47796c5543adbf6df62725c/src/main.rs#L73) for an hour now. At this point, I plan on moving my backend to a lightweight web framework called hyper. I planned on using as few dependencies and crates as possible, however using the hyper web framework will likely make my program less susceptible to risks such as denial of service and RCE. I’d just need to rewrite the majority of the backend API, but it’ll be simpler.

Either way, I’ve still done some progress! I changed some entries in the docker container and made the example site contain links to relevant documentation.

YAHOO! I’ve finally completed the write part of the program, where it takes the site.tar.gz and decompresses the archive into a target directory! This means that the project can technically be used at this point of time, however with no security/confidentiality at all. I also used the base64 command from GNU Coreutils which should be temporary… I tried using the base64 crate but it was difficult. I also need to implement automatic removal of the files, but that’s pretty simple.

Woo! The docker container is finally published! I’ve created a workflow that builds and publishes to https://github.com/publish-site/backend/pkgs/container/backend.

Hello! For the last few hours I’ve worked on getting colors into my program, YAY! Of course it checks if it is interactive, so the potential docker logs doesn’t risk getting cluttered with ANSI. I also removed some annoying debugging stuff, so that’s good to have out of the way. I made the program print the user agent for logging purposes, I may include more stuff like content length and such. I also started creating a wiki for both the backend and action! This will make it easier for the user to spin up their server and workflow.

NOTE: I don’t know why, but no time got logged with my neovim. Switching to Code OSS temporarily (?)

Phew! This was a long one. I finally implemented the upload logic for the backend server, meaning that I can finally upload the actual files! There is a few rough edges on the actions workflow, and I still have a bunch of debugging lines on the backend, and it’s still not writing anything. Either way I’m getting closer to done. I’m gonna focus on the completion, so first the upload part, then the authentication part. Finally after that I’ll clean up. Wish me luck!

I continued working on the backend side of this project, and made a docker image server to be deployed on the server. I implemented HTTPS by reverse-proxying through NGINX. The certs in the docker-compose file are intentionally left in the commit so you guys can check it out easier. Next steps are to create the actual frontend to be published to, and to initiate mTLS through nginx. Should be easy enough.

I worked on the server-side part and has come up with a multithreaded HTTP server written in rust, see https://github.com/publish-site/backend. The next step is to implement HTTPS. I’m not sure if I wanna use this implementation due to potential security issues by user error (?) but I’ll try.

Today I worked on the base actions workflow. The workflow is executing a script so far. Haven’t worked any on the “backend”. I also created a example workflow implementation.