Worked on completely redoing the AI model connected to it. The original system flow for the AI model would be to use YOLOv8 for everything, including segmentation, identification, etc. However, I rewrote the code to instead just segment the area to identify where vegetation exists in the first place, and then just use pure HSV calculations to determine vegetation health. I saw the FPS rise from ~4 to up to 7 while keeping a huge amount of the quality and expanding the size of the pixels greatly!

Sorry, it’s been a while since the last devlog! Again, I’m on a bit of a time crunch here so i dont have a huge amount of time to dedicate to writing these devlogs (I’m also entering this into a local engineering fair, NOT for school though)

Let’s go through all of the changes that I’ve made so far!

1. Fabrication of the drone:
   I’ve begun to physically attach things such as motors, the battery, and the flight controller onto the drone body. Some things have been a trouble, especially figuring out how to get the ESP32 onto there, i honestly might just tape it… I’ve also 3d printed all of the parts in CAD and attached them onto the drone! However, as you can see in the video, one of the motors was found to be defective or broken or something upon the first test, so I was unable to do my first flight test, but my calculations should say that it works!! We’ll see.
2. CAD design
   I’ve designed longer legs (theres only 2 in the image there should be 4 lol), and a bigger container to contain the seeds. In addition, I also made a cap to contain the flight controller and ESC, with holes to allow for heat transfer from the overheating battery.

Sorry for the lack of devlogs recently! This project has been getting increasingly stressful as I have a deadline to finish this project before my family goes on vacation for a while + school is getting mega stressful

Here is a brief rundown of everything that I’ve done so far:

1. Completely revamped the CAD. The body is now thinner, more screws are used rather than just glue (Which is always smart), the motors are raised up higher to give more clearance for all of the other parts, and I’m currently prototyping a cap to attach on top of the drone.
2. In terms of physical work, I got a ton done. Aside from starting the 3d printing process with the main body, I’ve also completely soldered all four motors to the ESC and flight controller, although the wires are messy and I might have to do some more work there.
3. I’ve also gotten the ESP32 and OV5640 to reliably stream to my local IP network through a remote battery, although it is highly choppy and it looks like the soldering is not well done so I’ll redo that bit.

I made some issues about the frame and how to organize all of the electronic components, but overall it’s still salvagable. I’m currently working on a cap for all of the electronics in cad and a way to screw the electronics using an internal thread m3.

Worked on properly setting up the CLIP model for the AI model to identify objects! I don’t know too much python so this took a lot of troubleshooting but after a few tutorials I got something that I am kind of happy with! The ultimate goal of the AI model is to be able to identify areas of high and low vegetation and then see where the drone needs to go, but as of right now I made it detect my face and a monitor in order to just test it out.

In addition, the video feed is running a smooth ~10 fps right now I think, which I am happy with. The main bottleneck is the AI system so I’m gonna focus on optimizing that and adding multithreading.

A lot of changes in the CAD. I calculated the weight of this thing and I found out that the volume was way too high (~1120 cm^3, which in 40% infill nylon is almost 2 pounds). So, to solve this, I cut down on the total volume of the drone by a ton in order to reduce the weight of the chassis so that I’m actually able to have a payload. Here’s list of the changes made:

1. motor arms were made skinnier and more rounded as they didn’t need to be huge blocks. They’re also more elevated, not related to weight but it allows for more clearance for the battery
2. drone frame was made thinner in terms of both height and I cut off the sides and forced all of the components into an X shape, although I’m probably gonna change this later.
3. Landing legs are more practical (no more fancy curves) but I do think that I’ll try to keep on reducing their girth.

Note that in this design the seed holder is not implemented yet and neither are the motors, but right now the volume is ~700 cm^3, which is like a 38% decrease which I am happy with! Probably gonna go up to 800 cm^3 later but its alright

Worked on soldering the motors to the esc and also the battery and capacitor to the electronic speed controller. I also started working on connecting the C8 radar controller but I accidentally dropped it and it landed on the capacitor, which completely fried it. So, I ordered new ones!

Made the main body of the drone thicker to properly accommodate the battery and esp32, and then shifted some holes and components around accordingly in the CAD.

I also made the dropping component of the drone with a simple DC motor attached to a trapdoor, as well as a second DC motor that controls the direction of the camera. I purchased a new OV5640 which should provide higher quality video feed with a better FOV, but I am very worried about video stability, so I’ll probably get some kind of foam to help, and if it’s absolutely necessary I’ll get a separate camera.

Soldered the battery onto the FC and also one of the motors, and then connected the esc to the flight controller today. In addition, I also worked on implementing a newer camera with better image quality, which will be very useful for better AI image detection, but i am having a lot of issues with quality suddenly dropping.

In addition, I also connected Betaflight to my flight controller and tested some of the controls.

started working on soldering the motors and the battery to the flight controller. It was going smoothly until my incredibly powerful battery short circuited after the wire slipped. It was genuinely terrifying because of how loud and bright it was but im totally fine.

In terms of the ai, I have the vision working but the detection is not.

Worked on adding the container and then the motor to hold the seeds and then drop them. I do not have an easy way to hold the motor in place now so the plan is just hot glue and tape, but give nthat it doesn’t have to be perfect its just prototype, it should be fine.

In addition, I’ve also worked on wirelessly connecting the ESP32 to my computer with an external battery source and then streaming the live video feed, which will be important to feed to the ai to get responses back.

Scouted out for more parts that are necessary for the drone to function. A lot of these components were incredibly expensive due to tariffs which I was a little unhappy with but whatever…

In terms of the actual CAD design, I worked on the landing gear sweep which was quite fun. I also secured it with m3 bolts and made sure to add extra stability to the wings with four bolts per wing.

VERY IMPORTANT CHANGE

I’ve decided to shift the focus of my project. The waterproofed O-Ring seal failed horribly on the fish project and some electronic components have been damaged. In addition, I’m having trouble justifying the true impact of this project, because not only does my current approach not solve the issue that I want to fix (Shape of the body is not as important as noise of the motors to do the purpose of helping minimize short-term ecological impact on coral reef ecosystems), but also because water-sealing stuff is hard and risky.

So, I’ve decided to transform this project into something different: a seed bombing drone meant to be an alternative to the highly expensive and closed source ones that companies like Airseed uses. Their drones have really good capacity and are good quality, but this also means that their drones are extremely expensive and hard to distribute, so this will be a cheaper alternative with intuitive digital ui to allow for anyone to use it, democratizing drone technology and allowing underserved communities take matters of vegetation propogation into their own hands.

To start off this design, I made the drone chassis and motor arms, and made an m3 hole to allow for a screw and hex bolt to secure each of the motor arms in place. In the main body I also have an area to attach the landing gear. I also scouted out motors, flight controllers, and batteries that would be appropriate for this project.

Created the web version by gutting the arbitrary precision and multithreading capabilities. Then, compiled into WASM using emscripten!

1. Started working on adjusting the body size in order to fit all of the electronics into the cad
2. Set up the camera function directly to my computer, incredibly low quality but it’ll do for now for a demonstration
3. Tested watertightness of the O-Ring, it works for <3 minutes which is all I was brave enough to go for, but that is good.
4. Made a little back connection area and printed that out, increased size of pectoral fins

Tested the press fit after 3d print

Due to my poor soldering the motors did not work anymore, so I need to get those working. I ordered new all-metal motors with stronger torque because that is more important for his project + less easy to get water inside the parts.

I’m also purchasing the final things that I’ll need for the project, they should be arriving soon!

Redesigned body and pectoral fins in cad with an area to fit a silicone o-ring to ensure water-tightness.

Vision is going to be done with an underwater endoscope through that little hole in the front.

I’m going to print this out and then find a way to actually work the motor area in the back

Made the website for the viewer! very minimalist design but im ok with that, I’ve decided to try and make the preview switch you to a different tab where that runs the web assembly, which I think is more clean.

It’s a pretty small site but I had to remake it like 3 times because I didn’t like it. I feel like just having this is enough.

Now that I’m using Tauri I have to rewrite all of the autoclicking code in Rust, which I did after learning the bare minimum and barely scraping by, but the code works and I can autoclick now!

In addition, I made a brand new UI design style which I am kind of happy with, and I’m working on a customization screen too.

I’m planning out how to implement macros, although it seems to be going to be a lot more difficult

LEARNED HOW TO SOLDER STUFF

Took ~6 hours and ended up ruining 3 of my drv8833 drivers, but I finally got one working and the motor spinning! I also found out that the polarity on my battery was switched so I had to flip that around.

In terms of CAD, I redesigned the fish fin to be more flexible by making the bridges snappable, and made a simple program to make the motor spin back and forth. I’m going to test if it has enough torque soon. Then i’ll test waterproofing and the camera.

JULIA SET IS WORKING 🎉🎉🎉🎉

Honestly much harder than working out the mandelbrot set, but it felt a lot better when I finished it, because the actual math is very similar except something else gets parameterized and you just set a point on the mandelbrot set as your constant.

I also made small UI dividers and organized the sidebar so it is actually kind of usable instead of messy lol, which I am pretty happy with. I’m running into the issue of the thing being too large though, so I might add a way to scroll down.

However, something that I do need to do is make a better point selector for the Julia set, I think I’d want to make a little mandelbrot texture and then let the user drag a point to choose instead of using regular entering, would be more intuitive.

Finalized design of the popups, and also fixed sensitivity issues. These popups are supposed to give you more information about how to use the viewer, web limitations, crash reports etc, because a lot of people generally don’t like reading readmes anyways.

I also started working on a julia set implementation! Right now everything breaks with values other than 0 0 2, but it is progress regardless! picture attached, I need to make a better viewer to how to chose this, probably what im gonna do next.

Remade the loading screen and fixed bugs about it!! in the previous version of this I didn’t really know what i was doing and vibe coded everything, but now it works well now!

Brought back mouse movement too, but still can’t get zooming right.

First devlog in over a week! I had finals so I had to put off a lot of this coding stuff, but im excited because I’m almost done with this project and I can ship with ~45 hours maybe?

So sad that i’m slowing down…

The biggest thing that I did was a custom function in UI that can create a popup to show information (For ex, nudging to switch to arbitrary precision, general info when first loaded)

Heres more some little stuff I did:

• Loading text
• Fixed minor bugs with UI like thread inputs
• fixed mirror image bug when zooming out
• Manual refresh option with “R”

I just wrote like 4 paragraphs and i dropped a video on the page which deleted the entire thing and im really unhappy so Im just going to write a way shorter version

tldr:

1. Switched from electron to raylib because chromium was unnecessary
2. brought all of the code over to raylib, very light beacuse its all pure c++
3. realized that things like ui creation and fonts are going to be a pain in raylib
4. Found tauri, started setting up the project
5. made a design in Figma for a UI concept design (not finished)
6. start learning rust for Tauri

Made it so that the autoclicker can change the time it spends clicking, number of clicks, and click interval

I’ve also realized that I really dont need electron to do this, im going to turn this into pure c++ code later because electron brings all of chromium and IPC is annoying I’m not too sure why I decided to do this, so that’s the big thing that I’m gonna be working on

its autoclicking for sure but i cant exit it because i forgot to add that functionality hlep

Made it open

Moved all of the rendering stuff (INCLUDING arbitrary precision) to renderBrot to further cut down on space in the main file which I am quite happy with this change took off ~200 more lines. Worked on the website more and then also organized my code.

Created a mold design and 3d printed it, its going to be made out of silicone resin and use phase lag to create the wiggling effect rather than actuators. Ribcage inside meant to stabilize it

Started relearning web design to make a website for the viewer, setting up emscripten to get raylib working. Threading and gmp doesnt work because of wasm limitations…

Short readme to relax

Low detail mode and uldm mode for 4x and 16x faster rendering since the pixels are bigger, useful when arbitrary precision library is used. Mouse zooming now responds to how strongly you scroll not just if youre scrolling up or down. Inputs now use strings instead of c strings and an option for using keyboard / mouse interchangably

Cut down the UI into this bit and main file went from ~900 lines to now 600

Zooming with mouse is fixed because it was buggy, but most importantly the checkboxes and positioning drawing is now in a different file! saved a lot of space in my main file but the text input ones arent in yet

Timing and detail level functionalities. Timer is to give the user time to zoom in or adjust the frame to what they want to see so they dont have to move nonstop, detail level reduces detail which is good for slower computers or arbitrary complexity

Added functionality that allows mouse movement and zooming to also register as offset input, that didn’t take me too long but I tried two times to try and split my main file into two separate files and no matter what I tried to do it wouldn’t work because I’m not smart… my main file is getting close to some thousand lines

Stretchable window, better ui. made it so that selecting one option deselects the others stuff like that. Collapsing window

Arbitrary precision libraries implemented. Now technically there is infinite zoom for the viewer. If you have it on, you could technically zoom in forever albeit it will get very slow very quickly.

The effective limit for this program used to be a zoom of e+16 because thats where long double precision ends, but GMP extends this technically indefinitely I think…

There are a lot of issues with optimization and performance though, after zoom exceeds e+30 the program starts taking up like 700%+ of my macs cpu usage and ends up taking ~30-40 seconds to render one screen. I’ll see what I can do to make it better.

Cleaner UI and customizable amount of threads being used, dynamic iteration changing for the best experience

I just multithreaded so hard I made the mistake of going in raw and just trying to do without any help or tutorials so I struggled a little but now it works and its quite a bit faster so I am happy with it

It’s hardcoded at four threads right now because I wanted simplicity but I’m going to make it customizable

Set up the 3D printer that had appeared from under the tree for some reason…

I’ve decided to use a non connected rib system with a thickness gradient inside a silicone mold rather than using hinges because that was stupid and the silicone rubber would probably get into the hinges so hopefully something like this works

• Editable iteration and zoom speed
• Made the periodicty checking actually work
  All of the rendering is also now no longer hard coded so the dimensions of the viewer can be whatever
  Dynamic detail option that turns up the iterations as you zoom in

GOT COLORS!!!!!! 😸😸

Coloring rectangles based on how many iterations it takes them to escape the mandelbrot set
Also added a zooming rectangle when you press m so that you can actually see where you’re about to go in terms of zooming
Things to work on are probably UI, an option to turn up the detail, access arbitrary precision libraries, sharing position, and maybe a little built in game that would encourage you to find the different locations in the mandelbrot set…

Better zooming UI (the blue square) to show where exactly it is, performance improvements, fixed the periodicity checking because it didnt work. Better information in the top corner.

Considering designing a better ui, adding a slider for detail amount, and then finally making the heat map thing! been putting off that for a while. maybe then an option to switch to arbitrary precision libraries because as you can see in the video theres a limit

• Now able to zoom in on the mandelbrot set
• Instead of recalculating all the time, the app only recalculates the position after each movement you do (For ex, moving, zooming in, etc.)
• Changed all of the floats to doubles and then to long doubles to maximize size, I was hitting precision errors all the time
• Began to implement a heat map thing but I don’t want to create that right now
• Local coordinates rather than global coordinates for drawing out the points

made the mandelbrot set show up!! issue is is that its incredibly slow and movement slows down a lot, the zooming feature that I tried to get working also literally doesn’t work so I’ll have to figure that out, but the thing shows up now! That’s huge

Sorry, nothing impressive yet!! :(

Set up the raylib environment and made basic movement, this is my second time using raylib so I’m kind of clueless but thats ok (Crossed lines are the origin)

Redrew tail fin, organized structure through a flow chart for how I want the fish to operate in figma, and started working on basic code for the buoy movement in VS (even though i dont even have it yet)

still waiting for those parts and the 3D printer to come in, when that time comes I’ll be happy and that means that I can actually start building the thing and testing it out

There’s about ~30 minutes of time in here that was supposed to be for a different project (My raylib mandelbrot set viewer) but I forgot to open a new workspace in my ide or something so now all of the time is allocated here… whatever… :(

Made a rough draft for a back fin for the fish which is going to be a silicone mold rather than a 3D printed hard part, also made a quick thing for the pectoral fins, still waiting for the parts to come so I can test out their movement

Making the fins are really hard because im buns at making things look smoothy, I’m not going to try too hard on the fins I just need to make sure it doesnt sink in the water