Devlog 6

Kind of a mixed day today. First 45-60 minutes of this devlog were spent fixing issues regarding the uploading migration feature, which had some mismatches and utilized the unused miles feature. You can see in the video now that the total time added up is now correct, as in the previous devlog it just concatenated all of the strings together as one. Also fixed some parsing issues with the road type and standardized the data so its easier for me to work with later.

For the rest of the period, I decided to get started with React Native! After a short tutorial and testing out for a bit I’ve started to work on the basic design and test it on an IOS simulator. The learning curve is a little strange since everything is familiar but also so different in execution, but I kind of replicated the login page that I wanted? I began to implement the firebase dataset inside this and I’m planning on fully implementing that later!

Many things learned about React Native today, many of which I don’t like……

Devlog 5

Begun working on the design process for the app! I’ve identified a style that I like and I’m probably going to keep this until I find something more appealing or professional. I took some heavy inspiration from RoadReady Ohio and other apps, but I will be sure to make it have its own identity eventually. It took quite some time for me to get something that I liked, most of my previous attempts felt too unprofessional but I’ll probably still make it more serious later.

After that, I’ve begun to implement this design in the actual HTML (Just the Login bit), as well as adding little page animations that I hopefully won’t regret when I try and migrate this project to React Native.

Speaking of React Native, I’m also gonna be starting to research how I’m gonna migrate over and how to make basic projects in React Native. I think that after fixing up some of the issues with the driving log migration features, I’ll start building in React Native very soon!

Devlog 4

Pretty big devlog, the main thing that I’ve been working on is the migration function from RoadReady to this app, in which you can upload a file and it’ll parse it into driving logs for you. It utilizes a regex and then also string splitting to identify logs inside a PDF, before then populating the entire drive log. Understanding how regexs work and how to get it to work consistently across formats was quite difficult, but I do think I learned a lot from it.

It took quite a lot of iterations to get to this point, for example just trying to iterate through each character to find patterns or potentially using an AI api to look through the parsed text to try and find it, but I think that I’m happy with what has come out. Obviously there’s a lot of discrepencies and the app still looks like buns, but hopefully that’ll change soon!

Right now I’m working on creating a design for the app that actually looks good, also is gonna help me start structuring everything.

Devlog 3

A smaller day today, but quite a bit of work still! I started working on making some of the UI elements look better, although I am kind of struggling a little bit because I’m having trouble deciding what kind of theme I want to go for for this app, so for now enjoy this generic looking UI style as I fully design it out.

This first week for Lockin I don’t really think that I’m going to start working on the design just yet, I’ll do that in Week 2 and work on rewriting it into React Native Expo by Weeks 3-4 along with additional features such as GPS tracking that would need huge rewriting if I did do them as PWAs.

The big thing that I did today was further flesh out the driving logging to fulfill all of the Michigan requirements (Weather, time of day, start time, road type etc.), as well as making those show up as driving logs in the main menu. I added more functionality to delete them as well.

I’ve also begun work on a file parser to allow for easier migration from RoadReady to this app, because many people probably wont start from 0 using this app, improving the UX although that bit isn’t completely finished yet!

Devlog #2

Started work on a bottom bar for design, the plan for it is to have all the pages on that and the profile picture part be able to switch between children for a parent to control in a family, but I don’t think that I’m going to work on the actual design of the website until later.

Today, the big thing that I worked on was making it so that the time tracking feature actually worked, although there are currently no security checks to make sure that the time recorded is legitimate, which can be added with GPS functionality later. This data is stored in Firebase immediately so it is consistent and updates every time a log is stored for consistency. Also fixed issues with the login system because for example the code would allow for multiple people to use the same email to sign in.

Also reorganized some of the data structures to work with Firestore (No nested arrays for ex, that was causing errors) and nighttime and daytime hours are stored separately since that is a requirement.

After adding a feature to export the data as the PDF format that Michigan state accepts for drivers, then basically all of the features of RoadReady will be implemented, the only things left are to fix backend errors and finish the design. However, I have many more plans to implement things such as being able to sync data with multiple phones to allow for more convenience, as well as being as able to import data from RoadReady into this new app to allow for easier migration.

Devlog 1

I’ve begun development of my RoadReady Michigan app. My ultimate plan for this is to turn this into a React Native application, but I don’t know React nor do I know React Native so for the first week of Lock In I’m going to be making this as a PWA.

I started out by making a bare-bones starting application with ZERO styling and pure functionality, beginning with the authentication system and database with Firebase. I have zero experience with backend so I messed around with a lot of stuff like Auth0 and Clerk but I think that this works the best because it doesn’t redirect you anywhere + it is easy.

After messing around with backend stuff I laid out some basic features for account management and creating profiles for their kids, since the parents are supposed to register for their kids.

I started work with the logging bit as well!

Made some more adjustments to the PID and found out that one of the reasons as to why there were stability issues was because roll and pitch were swapped. After fixing that, I added an angle mode instead of acro, which auto-balances the drone in Betaflight.

Now, the drone flies somewhat stably, although there is still drift! I want to know if theres some way to manually change that to have better flight.

Worked on final fixes and PID tuning, but there are still a lot of issues with stability! The chassis itself is structurally sound and all of the elctrical stuff works fine, but as evident by the video there are still some issues with the flight controller setup that need to be addressed.

I’ve also begun work on the demo website, which will make it look more presentable.

Added functionality for the vegetation index to be able to be adjusted by the user through changing the reference images that they use. Also used a vectorization approach to speed up the blending process significantly and also used a gaussian blur to speed things up while keeping much of the accuracy. Ended up seeing improvements from 7 to 27 fps and now the YOLOv8 segmentation itself is the longets step of the process (last row is total)

Worked on completely redoing the AI model connected to it. The original system flow for the AI model would be to use YOLOv8 for everything, including segmentation, identification, etc. However, I rewrote the code to instead just segment the area to identify where vegetation exists in the first place, and then just use pure HSV calculations to determine vegetation health. I saw the FPS rise from ~4 to up to 7 while keeping a huge amount of the quality and expanding the size of the pixels greatly!

Sorry, it’s been a while since the last devlog! Again, I’m on a bit of a time crunch here so i dont have a huge amount of time to dedicate to writing these devlogs (I’m also entering this into a local engineering fair, NOT for school though)

Let’s go through all of the changes that I’ve made so far!

1. Fabrication of the drone:
   I’ve begun to physically attach things such as motors, the battery, and the flight controller onto the drone body. Some things have been a trouble, especially figuring out how to get the ESP32 onto there, i honestly might just tape it… I’ve also 3d printed all of the parts in CAD and attached them onto the drone! However, as you can see in the video, one of the motors was found to be defective or broken or something upon the first test, so I was unable to do my first flight test, but my calculations should say that it works!! We’ll see.
2. CAD design
   I’ve designed longer legs (theres only 2 in the image there should be 4 lol), and a bigger container to contain the seeds. In addition, I also made a cap to contain the flight controller and ESC, with holes to allow for heat transfer from the overheating battery.

Sorry for the lack of devlogs recently! This project has been getting increasingly stressful as I have a deadline to finish this project before my family goes on vacation for a while + school is getting mega stressful

Here is a brief rundown of everything that I’ve done so far:

1. Completely revamped the CAD. The body is now thinner, more screws are used rather than just glue (Which is always smart), the motors are raised up higher to give more clearance for all of the other parts, and I’m currently prototyping a cap to attach on top of the drone.
2. In terms of physical work, I got a ton done. Aside from starting the 3d printing process with the main body, I’ve also completely soldered all four motors to the ESC and flight controller, although the wires are messy and I might have to do some more work there.
3. I’ve also gotten the ESP32 and OV5640 to reliably stream to my local IP network through a remote battery, although it is highly choppy and it looks like the soldering is not well done so I’ll redo that bit.

I made some issues about the frame and how to organize all of the electronic components, but overall it’s still salvagable. I’m currently working on a cap for all of the electronics in cad and a way to screw the electronics using an internal thread m3.

Worked on properly setting up the CLIP model for the AI model to identify objects! I don’t know too much python so this took a lot of troubleshooting but after a few tutorials I got something that I am kind of happy with! The ultimate goal of the AI model is to be able to identify areas of high and low vegetation and then see where the drone needs to go, but as of right now I made it detect my face and a monitor in order to just test it out.

In addition, the video feed is running a smooth ~10 fps right now I think, which I am happy with. The main bottleneck is the AI system so I’m gonna focus on optimizing that and adding multithreading.

A lot of changes in the CAD. I calculated the weight of this thing and I found out that the volume was way too high (~1120 cm^3, which in 40% infill nylon is almost 2 pounds). So, to solve this, I cut down on the total volume of the drone by a ton in order to reduce the weight of the chassis so that I’m actually able to have a payload. Here’s list of the changes made:

1. motor arms were made skinnier and more rounded as they didn’t need to be huge blocks. They’re also more elevated, not related to weight but it allows for more clearance for the battery
2. drone frame was made thinner in terms of both height and I cut off the sides and forced all of the components into an X shape, although I’m probably gonna change this later.
3. Landing legs are more practical (no more fancy curves) but I do think that I’ll try to keep on reducing their girth.

Note that in this design the seed holder is not implemented yet and neither are the motors, but right now the volume is ~700 cm^3, which is like a 38% decrease which I am happy with! Probably gonna go up to 800 cm^3 later but its alright

Worked on soldering the motors to the esc and also the battery and capacitor to the electronic speed controller. I also started working on connecting the C8 radar controller but I accidentally dropped it and it landed on the capacitor, which completely fried it. So, I ordered new ones!

Made the main body of the drone thicker to properly accommodate the battery and esp32, and then shifted some holes and components around accordingly in the CAD.

I also made the dropping component of the drone with a simple DC motor attached to a trapdoor, as well as a second DC motor that controls the direction of the camera. I purchased a new OV5640 which should provide higher quality video feed with a better FOV, but I am very worried about video stability, so I’ll probably get some kind of foam to help, and if it’s absolutely necessary I’ll get a separate camera.

Soldered the battery onto the FC and also one of the motors, and then connected the esc to the flight controller today. In addition, I also worked on implementing a newer camera with better image quality, which will be very useful for better AI image detection, but i am having a lot of issues with quality suddenly dropping.

In addition, I also connected Betaflight to my flight controller and tested some of the controls.

started working on soldering the motors and the battery to the flight controller. It was going smoothly until my incredibly powerful battery short circuited after the wire slipped. It was genuinely terrifying because of how loud and bright it was but im totally fine.

In terms of the ai, I have the vision working but the detection is not.

Worked on adding the container and then the motor to hold the seeds and then drop them. I do not have an easy way to hold the motor in place now so the plan is just hot glue and tape, but give nthat it doesn’t have to be perfect its just prototype, it should be fine.

In addition, I’ve also worked on wirelessly connecting the ESP32 to my computer with an external battery source and then streaming the live video feed, which will be important to feed to the ai to get responses back.

Scouted out for more parts that are necessary for the drone to function. A lot of these components were incredibly expensive due to tariffs which I was a little unhappy with but whatever…

In terms of the actual CAD design, I worked on the landing gear sweep which was quite fun. I also secured it with m3 bolts and made sure to add extra stability to the wings with four bolts per wing.

VERY IMPORTANT CHANGE

I’ve decided to shift the focus of my project. The waterproofed O-Ring seal failed horribly on the fish project and some electronic components have been damaged. In addition, I’m having trouble justifying the true impact of this project, because not only does my current approach not solve the issue that I want to fix (Shape of the body is not as important as noise of the motors to do the purpose of helping minimize short-term ecological impact on coral reef ecosystems), but also because water-sealing stuff is hard and risky.

So, I’ve decided to transform this project into something different: a seed bombing drone meant to be an alternative to the highly expensive and closed source ones that companies like Airseed uses. Their drones have really good capacity and are good quality, but this also means that their drones are extremely expensive and hard to distribute, so this will be a cheaper alternative with intuitive digital ui to allow for anyone to use it, democratizing drone technology and allowing underserved communities take matters of vegetation propogation into their own hands.

To start off this design, I made the drone chassis and motor arms, and made an m3 hole to allow for a screw and hex bolt to secure each of the motor arms in place. In the main body I also have an area to attach the landing gear. I also scouted out motors, flight controllers, and batteries that would be appropriate for this project.

Created the web version by gutting the arbitrary precision and multithreading capabilities. Then, compiled into WASM using emscripten!

1. Started working on adjusting the body size in order to fit all of the electronics into the cad
2. Set up the camera function directly to my computer, incredibly low quality but it’ll do for now for a demonstration
3. Tested watertightness of the O-Ring, it works for <3 minutes which is all I was brave enough to go for, but that is good.
4. Made a little back connection area and printed that out, increased size of pectoral fins

Tested the press fit after 3d print

Due to my poor soldering the motors did not work anymore, so I need to get those working. I ordered new all-metal motors with stronger torque because that is more important for his project + less easy to get water inside the parts.

I’m also purchasing the final things that I’ll need for the project, they should be arriving soon!

Redesigned body and pectoral fins in cad with an area to fit a silicone o-ring to ensure water-tightness.

Vision is going to be done with an underwater endoscope through that little hole in the front.

I’m going to print this out and then find a way to actually work the motor area in the back

Made the website for the viewer! very minimalist design but im ok with that, I’ve decided to try and make the preview switch you to a different tab where that runs the web assembly, which I think is more clean.

It’s a pretty small site but I had to remake it like 3 times because I didn’t like it. I feel like just having this is enough.

LEARNED HOW TO SOLDER STUFF

Took ~6 hours and ended up ruining 3 of my drv8833 drivers, but I finally got one working and the motor spinning! I also found out that the polarity on my battery was switched so I had to flip that around.

In terms of CAD, I redesigned the fish fin to be more flexible by making the bridges snappable, and made a simple program to make the motor spin back and forth. I’m going to test if it has enough torque soon. Then i’ll test waterproofing and the camera.

JULIA SET IS WORKING 🎉🎉🎉🎉

Honestly much harder than working out the mandelbrot set, but it felt a lot better when I finished it, because the actual math is very similar except something else gets parameterized and you just set a point on the mandelbrot set as your constant.

I also made small UI dividers and organized the sidebar so it is actually kind of usable instead of messy lol, which I am pretty happy with. I’m running into the issue of the thing being too large though, so I might add a way to scroll down.

However, something that I do need to do is make a better point selector for the Julia set, I think I’d want to make a little mandelbrot texture and then let the user drag a point to choose instead of using regular entering, would be more intuitive.

Finalized design of the popups, and also fixed sensitivity issues. These popups are supposed to give you more information about how to use the viewer, web limitations, crash reports etc, because a lot of people generally don’t like reading readmes anyways.

I also started working on a julia set implementation! Right now everything breaks with values other than 0 0 2, but it is progress regardless! picture attached, I need to make a better viewer to how to chose this, probably what im gonna do next.

Remade the loading screen and fixed bugs about it!! in the previous version of this I didn’t really know what i was doing and vibe coded everything, but now it works well now!

Brought back mouse movement too, but still can’t get zooming right.

First devlog in over a week! I had finals so I had to put off a lot of this coding stuff, but im excited because I’m almost done with this project and I can ship with ~45 hours maybe?

So sad that i’m slowing down…

The biggest thing that I did was a custom function in UI that can create a popup to show information (For ex, nudging to switch to arbitrary precision, general info when first loaded)

Heres more some little stuff I did:

• Loading text
• Fixed minor bugs with UI like thread inputs
• fixed mirror image bug when zooming out
• Manual refresh option with “R”

Moved all of the rendering stuff (INCLUDING arbitrary precision) to renderBrot to further cut down on space in the main file which I am quite happy with this change took off ~200 more lines. Worked on the website more and then also organized my code.

Created a mold design and 3d printed it, its going to be made out of silicone resin and use phase lag to create the wiggling effect rather than actuators. Ribcage inside meant to stabilize it

Started relearning web design to make a website for the viewer, setting up emscripten to get raylib working. Threading and gmp doesnt work because of wasm limitations…

Short readme to relax

Low detail mode and uldm mode for 4x and 16x faster rendering since the pixels are bigger, useful when arbitrary precision library is used. Mouse zooming now responds to how strongly you scroll not just if youre scrolling up or down. Inputs now use strings instead of c strings and an option for using keyboard / mouse interchangably

Cut down the UI into this bit and main file went from ~900 lines to now 600

Zooming with mouse is fixed because it was buggy, but most importantly the checkboxes and positioning drawing is now in a different file! saved a lot of space in my main file but the text input ones arent in yet

Timing and detail level functionalities. Timer is to give the user time to zoom in or adjust the frame to what they want to see so they dont have to move nonstop, detail level reduces detail which is good for slower computers or arbitrary complexity

Added functionality that allows mouse movement and zooming to also register as offset input, that didn’t take me too long but I tried two times to try and split my main file into two separate files and no matter what I tried to do it wouldn’t work because I’m not smart… my main file is getting close to some thousand lines

Stretchable window, better ui. made it so that selecting one option deselects the others stuff like that. Collapsing window

Arbitrary precision libraries implemented. Now technically there is infinite zoom for the viewer. If you have it on, you could technically zoom in forever albeit it will get very slow very quickly.

The effective limit for this program used to be a zoom of e+16 because thats where long double precision ends, but GMP extends this technically indefinitely I think…

There are a lot of issues with optimization and performance though, after zoom exceeds e+30 the program starts taking up like 700%+ of my macs cpu usage and ends up taking ~30-40 seconds to render one screen. I’ll see what I can do to make it better.

Cleaner UI and customizable amount of threads being used, dynamic iteration changing for the best experience

I just multithreaded so hard I made the mistake of going in raw and just trying to do without any help or tutorials so I struggled a little but now it works and its quite a bit faster so I am happy with it

It’s hardcoded at four threads right now because I wanted simplicity but I’m going to make it customizable

Set up the 3D printer that had appeared from under the tree for some reason…

I’ve decided to use a non connected rib system with a thickness gradient inside a silicone mold rather than using hinges because that was stupid and the silicone rubber would probably get into the hinges so hopefully something like this works

• Editable iteration and zoom speed
• Made the periodicty checking actually work
  All of the rendering is also now no longer hard coded so the dimensions of the viewer can be whatever
  Dynamic detail option that turns up the iterations as you zoom in

GOT COLORS!!!!!! 😸😸

Coloring rectangles based on how many iterations it takes them to escape the mandelbrot set
Also added a zooming rectangle when you press m so that you can actually see where you’re about to go in terms of zooming
Things to work on are probably UI, an option to turn up the detail, access arbitrary precision libraries, sharing position, and maybe a little built in game that would encourage you to find the different locations in the mandelbrot set…

Better zooming UI (the blue square) to show where exactly it is, performance improvements, fixed the periodicity checking because it didnt work. Better information in the top corner.

Considering designing a better ui, adding a slider for detail amount, and then finally making the heat map thing! been putting off that for a while. maybe then an option to switch to arbitrary precision libraries because as you can see in the video theres a limit

• Now able to zoom in on the mandelbrot set
• Instead of recalculating all the time, the app only recalculates the position after each movement you do (For ex, moving, zooming in, etc.)
• Changed all of the floats to doubles and then to long doubles to maximize size, I was hitting precision errors all the time
• Began to implement a heat map thing but I don’t want to create that right now
• Local coordinates rather than global coordinates for drawing out the points

made the mandelbrot set show up!! issue is is that its incredibly slow and movement slows down a lot, the zooming feature that I tried to get working also literally doesn’t work so I’ll have to figure that out, but the thing shows up now! That’s huge

Sorry, nothing impressive yet!! :(

Set up the raylib environment and made basic movement, this is my second time using raylib so I’m kind of clueless but thats ok (Crossed lines are the origin)

Made a rough draft for a back fin for the fish which is going to be a silicone mold rather than a 3D printed hard part, also made a quick thing for the pectoral fins, still waiting for the parts to come so I can test out their movement

Making the fins are really hard because im buns at making things look smoothy, I’m not going to try too hard on the fins I just need to make sure it doesnt sink in the water