complex-plotter

This is, by far, the devlog with the most hours registered this project has ever had. So, what new, cool thing did I implement? Nothing! This time was almost entirely dedicated writing documentation (my kryptonite), and preparing the project to be released.

One of the main challenges I knew I’d have to face when starting this project was explaining it to other people. The math behind it is quite complex (get it?), so I knew that, eventually, I’d have to write some sort of educational guide to tell people what it does, and how to use it. More than nine hours later, we’re done!

I figured most people learn better visualy, so I poured some of my time to learn how to use Manim, the animation software 3blue1brown uses for his videos. I made about a dozen animations with it, and wrote a guide to introduce a laymen into complex analysis and function plotting.
Additionally, I also documented how the tool works internally (basically doing the same thing that I already did with the devlogs, but to more depth), and its features.
This, plus an install guide, getting a bunch of screenshots and videos from the plotter to put in README, makes us have completed our project’s documentation!

A small portion of this time was also dedicated to making the project build to a single executable, for it to have a favicon, and other small quality-of-life changes. Next we’ll bugfix, and this project will be ready to ship!

Attached, some of the animations!

Related Commits
Commit a160c09: Finished basic settings for deployment
Commit fae6ce0: Added grid warping
Commit 42ee549: Finished the-basics documentation
Commit 5bb3f81: Added advanced.md and assets
Commit 6a5f185: Added images to features.md, started README.md
Commit 30b3560: Finished readme.md

We’ve got a usable UI, and the good stuff that comes necessary for any good math program, now!

First, let’s get the developer stuff out of the way. Before, if we wanted to add a new function, we’d have to write code in 6 distinct spots, as discussed in Issue #31, a consequence of my #ifndef clauses black magic. This, now, is all done in load time,making my life far easier.

Now, for the math stuff. I’ll make a rare use of bullet points, I:

• Added every elementary function to our constant-folding simplification step.
• Added some (one) utility function: the modulo (%) operator!

Our most important addition is, by far, hovering to show a function’s value. All our functions are calculated on shaders, so we’d either have to hard code a C++ counterpart for ever GLSL function, or find a way to send data from the shader to the main program.
The problem is, while it’s easy to send data to shaders (through uniform variables), it’s quite difficult to do the opposite. And even if there was an easy way to do this, remember: our fragment shader runs for every pixel, so they all would be sending data to the CPU at the same time, which would be far too expensive.
Instead, what we do is create a 1x1 pixel shader, and send it a copy of our function, plus the mouse coordinates. It then shares a single vec4 variable that stores both z (the input) and f(z) (the output). Voilá, we can get things from the shader, now!

Drawing the grid is just some additional math in the fragment shader, and the UI changes are, well, UI.

Related Commits
Commit 1d123a6: Automated interpreter shader creation
Commit ba3f299: UI and constant folder overhaul
Commit 5893ae0: Inspector done!
Commit 1ff791c: Added grid

Related Issues
Issue #31

Unfortunately, it’s time to do our laundry.

With 3D rendering complete, all of the essential features of the plotter are done. This means that, had we wanted to, we could ship this as a fully-fledged math engine and leave the job of actually turning it into an application to someone else. But we can’t be doing that! Therefore, it’s time for us to start making way to shipping the project.

First of all, I wanted this plotter to be complete, meaning that it implements every elementary function (those being, in simple terms, a set of well-behaved functions that mathematicians use). This is simply a manner of writing the already existing real and imaginary components of these functions (and their derivatives), which are well-known and defined.
Is this essentially just writing boilerplate? Yes! Thankfully, this is not my first rodeo implementing these functions, so I could port a good amount of code from an old project. This is all done now: all elementary functions and their derivatives have been implemented!

Now, for the engine-specific things: so far we have been writing shaders into files and reading them at runtime. This works, but adds unnecesary file I/O operations, and makes it impossible to port our entire project as one .exe file. So we had to remove that!
One way to do this would be by writing our GLSL code in strings, but this removes linting. Instead, I implemented a dynamic source string builder at build time.

Attached, the new functions!

Related Commits

Commit 7923ee3: File conversion to strings at build-time
Commit 5ba94c0: Complete removal of file reading logic
Commit 0a56357: Finished implementing elementary functions
Commit 48d08fc: Derivatives of elementary functions

Related Issues
Issue #30: Switch from shader files to strings

We can do things in 3D now!

This might make you ask what are we plotting in this additional dimension, and the answer is simple: nothing that we weren’t showing already

In our plotter, we use brightness to represent magnitude (how large a number is). Black corresponds to zero, and white corresponds to a very high number, making roots of functions look like “sinks”. It turns out that it’s far easier to visualize these “sinks”, “valleys” and “peaks” with the help of a third dimension (akin to looking at a satellite picture of a mountain range vs being there).

There’s many ways to render functions in 3D, but the one I chose is by laying out a large 256x256 grid of “mini-planes” that are molded to look like the function (there are methods with ‘infinite’ precision but they are horribly slow and, if we want more precision, we can just add more segments, anyway!). This required me to create a Mesh struct and functions to handle their creation and use.
Also, since all this logic operates on vertices, we have to move the 3D code into a vertex shader. This required massive refactoring of pretty much all of our preprocessing functions, since they were all rigged to work with the fragment shader. We also had to do more dependency injection trickery to make sure the 2D and 3D shaders remain synchronized (of course).
Finally, we had to create a custom camera to actually move around our 3D plot, and update our state variables to see if the plot is 3D.

Attached, the third dimension! :O

Related Issues:
Issue #29: 3D Plotting

Related Commits:
Commit 4a0ab12: Fragment and vertex shader for 3D
Commit 0a17f71: Refactor of preprocessor logic
Commit 614238a: Created mesh grid
Commit 4bfd2ed: 3D Rendering done
Commit 0a17f71: Working camera!

We can take derivatives of functions now!

For those not familiar with the term, a derivative is a higher-order function (meaning it takes a function as a parameter, and outputs another function). This is not a feature natively supported by our shader’s language (GLSL), so we have to do all the higher-order logic ourselves in the parser. Which makes sense: if d(z^2) = 2z, and the user types d(z^2), we can just send 2z to the shader.
Our stack is great for evaluating functions numerically, but not optimal for symbolic manipulation. An Abstract Syntax Tree, or AST for short, works far better for this, because it simulates the “recursive” nature of expressions. So, we had to convert our RPN queue to an AST, and then back to RPN. Some time put into that!

Now, there is a generic way to calculate any derivative, but it would require us to work with infinitesimally small quantities, which just do not exist in computers. So, if we tried to use it, we’d get a rough approximation at best, and a very innacurate result at worst. This, however, can be fixed with a derivative table, falling back to this numeric method whenever needed

All this higher-order tree manipulation is of intermediate difficult when using raw pointers (int*) but quickly becomes hell when using C++’s smart pointers (std::unique_ptr). Safe to say, this difficulty is what caused most of these dev hours.

Attached, the new stuff!

Related Issues:
Issue #8: “Computing higher order functions”, and it’s sub-issues:
* Issue #9: Stack into syntax tree
* Issue #10: Syntax tree into RPN stack
* Issue #11: Differentiation
* Issue #12: Operator rules
* Issue #13: Numerical differentiation
* Issue #14: Analytic differentiation

Related Commits:
Commit b56b9fa: ast -> stack, stack -> ast
Commit f410dba: Finished implementing derivatives

We’ve got got compiled shaders up and running!

All the benefits and reasoning behind doing this were throughouly discussed in Issue #23. What matters is now we have our interpreted shader as a “preview”, and whenever we want improved performance, it’s just a matter of hitting “enter” and we’re done!

First, we had to convert our stack of opcodes into a GLSL string. After the type overhaul, this was simple enough of a task.
Since I’m dead-set on having a semi-decent developer experience while coding with my fragment shader (shader/plotter.frag), I had to do some string manipulation trickery to guarantee that all important function and coloring definitions are shared between shader modes.
This in turn required me to modify our existing Shader class dependency to accept strings rather than file paths to compile, and to create a new CompilerShader class to handle all the aforementioned string manipulation. There are some optimizations still there to be made to increase compile time (namely ommiting function definitions we’re not using), but this is not necessary for the time being, as compile times are low.
Also, since compile times were shown to be tiny, the async proposal related to Issue #28 might not be necessary.
This also involved changing our global function state and callback functions to take a reference to different shaders, so we can change them on the fly.

That’s all for this devlog. Attached, a video of me rendering a huge function with the interpreter vs compiler: the performance difference is palpable!

Related commits:
Commit 5f38f07: Converting stack into glsl infix string
Commit 8f40852: CompilerShader class done
Commit 48d3fed: Added shader switching for compilation

Related issues
Issue #28 and its subissues.

Longest time without a devlog yet! So we’ve implemented something amazing and new, right? We’ll get a new nice feature, right??? WRONG! This coding session was mostly concerning architectural details for the project, so we don’t have any new “features” per se, that is, besides developer experience and documentation.

First, the non-code part: performance and precision. We’ve been using a stack that’s dynamically interpreted at runtime for our shaders so far, which obviously hinders performance when compared to a compiled string. This is exceptionally true for very large expressions, like computing z^327manually, which makes the interpreted version unusable.
I tried many different precision techniques in the meantime (these demos are not documented). Some of them, like double-double or emulated double (two floats for one double) didn’t work at all at medium performance cost. Others, like using the native double type gave awesome precision, at the cost of the program running about 40x slower. This made me conclude that, if we want high precision, we lose performance. And we can’t have that added to the performance loss of the interpreter.

Now, since we wanted to implement a compiler, I’d have to convert TokenOperators into GLSL strings that represents each function. I started writing a to_glsl_string function, but suddenly a wave of clarity crashed down on me and I asked myself what the hell I was doing. At that point of that project, every time I wanted to add a new function, I’d have to modify code at 6 different spots! 6!!!!
Hence, we had to stop our math fest and do some refactoring. I hope you can forgive me.

Related Commits
Commit 1ed13ea: Overhaul of type system

Related Issues:
Issue #23: Add compiled shaders (Goes into more detail - highly recommended read!)

We can write functions in the program now! Before, we had a hardcoded string that was directly put into parser::parse(), which required a recompilation every time we wanted to see a new function. Of course, we can’t be having that!

You might think that this is a simple feature to implement (just read a textbox in your render loop, right?), but it is most certainly not! Why? Because, so far, we were assuming the strings sent to our parser are well-formatted. So that means we were expecting strings that:

1. Can’t contain any incorrect math (2 * 3 +)
2. Can’t have messed up numbers (2.3324.2 * 3)
3. Can’t have unknown symbols (z * skibidi * dopdopdop)
4. Can’t have any mismatched parenthesis ()z*(z)
   So I had to rewrite all my parser functions to throw errors whenever they run into any of these weird cases. I’m still catching something minor here and there, but the error handling seems to be robust enough for the time being.

Implementing the UI itself was more complicated than i thought and introduced an entirely new dependency: ImGui. I could use the user’s operating system’s native components for the GUI, but that would make it impossible to port the project to Web Assembly as is. So yeah, there’s that. It’s not the prettiest so far, but it works! Attached, there is me trying out this new feature.

Related commits:
Commit d30ec7c: Added text validation, added ImGui support
Commit d37dd49: Text input working, but small
Commit f7da8cd
Related issues:
Issue #18: Inputting functions at runtime

• Issue #19: Reading at runtime
• Issue #20: Validating user input

A careful eye might have caught on that in our old code, we were setting a uniform for u_range, u_resolutionand u_range by hand once before the rendering loop. Why? Well, because I hadn’t bothered to allow user input to change them dynamically before.

I have implemented callback functions for panning, zooming and resizing the window. Now, you can explore the plot of a function! I plan on, in the future, changing the cursor to a custom “hand open” and “hand closed” cursor. Another nice note: this is running is super high FPS (around ~1000fps), but that is likely because we’re rendering a simple function with a small stack size. For an expression with ~200 stack_operators, this drops to around 200fps, which is still good enough!

Here is a video of me panning and zooming around the function of the previous devlog!

We can finally send things to the shader! This means that our input (like z*x + z*(y*x)/(2x) will be interpreted, transformed into a TokenOperator stack, broken into its unsigned char and glm::vec2 counterparts and sent to the shader.

Along the way, I ran through a bug where simple expressions like “z+x” wouldn’t render. This would happen because of a logic error with the ordering of operations, making so SHADER_SUM had a value lower than SHADER_VALUE_BOUNDARY, which made the shader interpreter treat it like a constant. This bug was a slight headache to fix, but it’s fixed!

Here’s the input function rendered!

Related commits:
Commit 076fd8d: Sending things to the shader done!
Commit ea2d8bc: Added #ifndef guard clauses to solve definition errors
Commit 2594ed3: Finished synchronization - fragment shader now shows 73 errors. Uh oh.
Related issues:
Issue #4: Sending things to the shader

• Issue #5: Generating constant and operator stack from rpn stack
• Issue #6: Converting TokenOperator stack into unsigned char IDs
• Issue #7: Keeping main code and shader IDs synchronized

This is a project that has been in development for a little bit, so if you want to see what has been going on, you can see it here: https://github.com/Sekqies/complex-plotter/issues

The bottom line is that we had a bunch of C++ and GLSL constants, and we needed a way to keep them synchronized. We could rely on a watchful eye to do so, but this is bug prone and dangerous, as discussed in issue 4: https://github.com/Sekqies/complex-plotter/issues/4 . So we had to fix it!

Now, we have a build process that sets the constants in both GLSL and C++. This means we can start sending the stack as Operators instead of magic numbers!

Below attached is the colored domain produced by that code (that corresponds to) f(z) = (1+i)z