Baller

opencv ball detection appears to suck (or perhaps i suck at using it) so I trained a YOLO26 model to detect balls instead. with this change, as well as a funny coordinate system error, the 3d tracked ray is now correct! the 2d object detection is quite shaky and doesn’t work when the ball is thrown, which means i now need to spend a lot of time labeling more training data :(

i only just realized that much of the inaccuracy was probably caused by camera distortion, despite the fact that iphone lens correction is supposed to be “good,” so i have decided to ditch fspy for traditional camera calibration, and use a camera-relative coordinate system instead of that provided by fspy

(kinda sketchy) 3d reconstruction!!

this implementation projects each ball’s 2d screen-space coordinate into a world-space line of sight, then uses the ball’s pixel diameter to estimate a distance along that line of sight. i wasn’t sure what formula to use to determine the exact distance, so it just brute forces a correct enough distance using a linear regression (…heh). in researching this, i also found a paper that seems to address a problem very similar to what im trying to solve (see http://doi.org/10.1109/CVPRW56347.2022.00391), which i’m still in the process of reading

2d ball detection!

forward projection given fSpy parameters now works! turns out the issue was differing coordinate systems between fSpy, openCV, and Desmos >:(

the current goal is to be able to accurately detect ball positions from a single image, with the following steps:
1) ask the user to draw axis-aligned lines to determine camera parameters
2) detect yellow balls in image space and project them into 3d world space lines
3) use the known ball measurements to find a point along the line

so far, i have figured out how to determine camera parameters by offloading the work to fSpy. i then tried to figure out how the returned parameters work by creating a world-to-camera projection in Desmos, but it turns out that most high schooler’s haven’t learned linear algebra :(.