Obsidian Tag Folder Explorer

Optimization!!!

So this ship as a whole was focused on optimizing my project. Being a very big project and having a lot of moving parts, it was super difficult to actually ensure that everything worked perfectly and was optimized in the first go.

So what I focused on was making the product first, and I thought that I would optimize it later. So this whole ship was focused on optimizations.

I went over the codebase section by section and looked at the things that I could optimize. Out of them, I performed three which felt super important to me:

• First, I realized that unlike a normal node, which stored its children as a record making tagnode child retrieval super fast, i had overlooked this implementation for the root node. This happened because technically the rootnode was not actually a full TagNode Object but was just a proxy place holder.

So by optimizing that, i made the whole tree compute faster bringing down the computation from O(n) to O(1).

The physical benefit was a loading improvement from 1018.90ms to 672.70ms which comes around to a boost of 1.5x

• Secondly I realized that something similar was happening for file retrievals under a TagNode. I was performing an O(n) search.

This optimization would have required me to construct per TagNode records in order to make retrieve files faster, but i leveraged the fact that i already had a global file map and so just extracted the FileLeaf object from there and verified if it was the same in the set of files. Verification is an O(1) action, making the whole process O(1).

The physical benefit here was another loading improvement from 7.10ms to 4.90ms for loading a specific file details in the right sidebar, which comes around to a boost of 1.4x

• Finally I realized that in the command for pulling tags from the body into the frontmatter, I was opening and closing the file for each individual tag that I wanted to process leading to unnecessary reads ie the (n+1) issue.

Optimizing brought the time for the same text case from 13489.40ms to 20.60ms to 1.00ms. Here I actually performed two separate optimizations. The first one was removing the n+1 issue and the second was that while merging the two lists, the old method was looping over the second list in order to ensure no duplicates. This led to a complexity of O(n^2). By just appending both lists and converting to a set and then back to a list, I made the process super fast.

hey, so time for another devlog!!

last time i had upgraded the mainview and added multiple options and ways to view the files there. so this time i did something similar in the left sidebar.

untill now the left sidebar only had a nested tree of the tags and files under them and one separate folder for the untagged files.

now it has multiple views including:
Classic, Only Untagged Files, All Empty Folders, All Files, All Folders

Second most important feature is the ability to right click on files and folders in the mainview.

So each file and fodler in the mainview can be right-clicked and opens a context menu. the menus for folders and files are pretty similar except for a few commands.

a new command for files is something that i think is kinda experimental cuz it might not always work and probably goes against the right way to use the obsidian api, but it works for now so im going to keep it. right clicking on a file can provide the option to reveal the file in the obsidian original sidebar. so yeah, thats pretty cool

enjoy!!!

hello! this is my first update after my ship!
dw guys, ive been COOKING!

so one of the first things id like to show is the raindrop.io integration.
when i have wesites or links that i want to store with respect to a specific project and i dont need them right now, but ill need them later, then i save it to raindrop. but what use is saving it if i cant retive it.

so what i implemented is the ability to link raindrop collections to individual tag folders. that way i can have a specific collection appear when i open a specific folder

Another major feature that i have implemented is multiple views for the main section. In the windows file explorer, there are multiple views in which we can go through the files in the current folder. ive implemented something similar here. until now we only had a blocks view. now we have a list like view and a details view.

also i updated the drag and drop index method making it way way way more robust!

Hey, big updates again. i think v0 is ready to ship!!!

lets goooo!!!

just a list of all the new features:

1. added tooltips on files and folders
2. added pinning feature on files and folders inside folders
3. files and folders can be dragged and dropped in the gui in general
4. files can be dragged from the original file explorer into my gui
5. both main view and tag tree view can be sorted in ascending and descending alphabetical order
6. added a rename button to rename files and folders inside gui
7. ctrl click opens folders in a new tab
8. ctrl drag uplicates files and folders in new location
9. bottom right corner shows state of the backend (preferably dont make canges while it is red)
10. added a loading screen
11. new command to pull all tags from body into metadata
12. added a delete button for both files and folders
    13, added event listeners to keep virtual state in sync with physical state of files

hello.
so i know that a devlog is due.
“like dude where have you been?”
calm down mon petit papillon, ill explain everything. but this is gonna be a long one, so buckle up and get that popcorn cuz i aint gonna repeat this.

so ive been gone cuz i was on holiday for like 2 weeks or something, but then i come back and boom. i have no idea what i was working on. like the whole codebase is alien to me. but i persist. and so it took me a couple of days to even get back uptodate and understand how things were working in the first place.

so now that i actually understood the code, here comes the crazy part. i start to see inconsistencies in my own code. being completely fresh to my own codebase, i started asking questions about functions and stuff that i had just assumed till now. and then i found MAJOR issues.

for one a few functions were completely duplicated in two very separate files. so then i had to sit and integrate botha dn resolve any dependancies.

secondly a lot of my functions were reading from a file’s stale cache and based on that calculating and writing to the actual file. this took a very very very long time to realize and even figure out.
i was actually trying to implement legends for my files, and for some reason it sometimes worked and sometimes failed. so yeah. i was frustrated. i checked my code and saw that i was correct. then did a whole lot of digging to find that actually it was the cache read and physical write issue that i mentioned above.

now normaly this would have been a simple problem to solve. just swap the cache read with a physical read and boom solved, but life is rarely ever so simple. turns out that the read file fuction is great cuz it works great. but its async. and my whole codebase until now has been sync, so i need to now just change my WHOLE CODEBASE.

any way so legend assignment works for both folders and files now. so WOW :D

more in the comments if you want to read cuz the devlog body got too long

hey again.
So i just completed the orchestrator file. That is a huge deal since basically the orchestrator is the one dude in charge or controling how i interact with all files.

so if he is clean and ready we basically are almost complete.

almost….
how many times have we heard that huh XD

Other functions and functionalities ive worked on
Copy - Cut - Paste
Bookmark
Pinned
Legend

created individual functions for each
paste handles most of copy cut paste
both files and folders can be pinned bookmarked and have a legend
made mofifications to rollback structure for this
added concept of owner to each action log

So now we have a rollback tracking system. Think this is the final backend devlog that im going to make.

ive created all the functions that a frontend might require. ive setup everything that i could need from the backend. i think im good to just switch to the frontend now.

ok so i am a god and my potential is unmatched.

FEAR ME MERE MORTALS!!!!

point is i forgot to kinda mention this in a devlog and honestly since i had done this around a week ago, i myself kinda forgot that i had done this in the frontend.

thing is that since im using svelte, each window requests its own snapshot of the backend tree. because of this if i update the tree from one window, it will cascade to the backend original tree, but i would have inconsistencies between windows.

i initially thought that since the plugin class kinda keeps a track of all the workspace view instances (windows) id just do a broadcast. but that would get unnecessarrily heavy. and you know me. i love myself some optimiiiiiization.

so now whenever the backend tree is updated. all i have to do is call one function which checks if a window is visible and if it has a old snapshot, sends it a new one. similarly when a window is made active, the same check is performed.

im cool

maaaaajor changes.

• ported all metadata files from md to json
• updated fileLeaf object creation to optimize meomory space. now only 1 fileleaf objet per physical file. this is done via a central fileleaf map that stores all fileleaf objects with paths as the key.
• the file leaf objects no longer contain a tagpath property since they are not limited to one parent.
• also now file retrivals are performed using teh filepath not file name which would have led to clashes.
• also tree construction is now way more optimized. tbh last time it was done badly. now its done correctly.
• considering implementing a rollback architechture and so created a normalized json format for any write action on files and folders. will later use this same thing chained to each other for tracking and undo redo
• the treeroot object now contains a roottagnode object. finally. but this only has public functions no private variables. maybe might need to unify this somehow later. but it will be a big architectural change. not now.

nothing much to show new on the ui side tbh..

new update.

shifted the plugin to parse tags only from the yaml metadata
no manual slicing of the files. no corruption.

also shifted the folder metadata files from md to json.
makes parsing much more easier.

wrote functions for clean reads and manipulations of the json files.

hello again.
so we shifted back to the backend model and most of the time was spent on designing the architecture of how the frontend would control the backend.

so yeah. pretty much it. designing file manipulation functions and testing them.

one major update is that maybe we might have to/ be able to only parse tags from inside file yaml and not file body. thing is that reading these tags is easy and clean, but manipulating them requires me to manipulate the file via a cache. if consecutive manipulations are made and the cached file is not updated in between, the function ends up slicing the file wrong, corrupting the data.

metadata has specific functions that cleanly handle reads and writes. so im a bit confused about what i should be doing.

see ive already implemented a command to pull all tags from the body into the frontmatter without manipulating the file. this is great. so i kinda loose nothing. users who write tags inside the body itself, just have to call this command. or maybe we call this command before constructing the tree. idk. but its a solvable problem

the issue wil probably come when i want to implement chunks. idk how ill controll chunks if i cant assign tags to a chunk inside the body.

one solution that i think i can use is that we dont just add tags. we add chunk specifc tags that look like “chunk_1770208637_tagname”. the timestamp is the unix time corresponding to the chunk time. thus the body is only read and the yaml is modified at any point.

issue is what happens when u accidentallly modify the time of a chunk? or i have to really modify the time of a chunk? (all inside the editor)

maybe i can provide another function where i can place the cursor inside a chunk and then call the method. provide it the tags i want to add and then it reconstructs the tag in the yaml…..?

chunks is gonna be a complex one.

added a few buttons for minimal interaction.

ok so right now this is a glorified tag based file viewer. it just provides a new view for the tag obsidian new. no interaction, no file moving, no file copy etc etc.

basically its just a read only right now.

but its gonna get better. now we have a whole new aspect that we have to integrate into this project.
big leap and another major architectural change. we now interact with the file tag system.

lets go.

wow ok. another big milestone. now we can render file and folder metadata very efficiently.

so the right sidebar is mainly focused on giving a complete overview of what the current selected file/ folder is about. for that we are using the folder/ file metadata.

in terms of the file metadata, it already exists at the top of the file or in the file properties.

however, for folder metadata we had to go a bit further. Since folders here are virtual, we maintain a folder metadata file for each folder that ever existed (until you delete it).

that way each folder now has some metadata that can easily describe the folder without opening it.

as of now, read only functions are working great!!

wow ok so a lot to unpack here.
as of now we have a great read-only view for the project!!! 🥳🥳🥳🥳🥳

the left sidebar and the main view both work cleanly! they can open new folders in teh same view and open files in the background!!!

wheeeeeeee!!!! i know this looks childish, but im just so dam happy!!!!

hey. so not much to show today. mostly because ive been working on the project outside vscode, mainly on paper

right now i have a good scaffolding, but tbh, this upcoming piece is the biggest architectural implementation. ive been trying to figure out how to isolate the backend and the frontend from each other in svelte. dont really understand it, so i spent yesterday trying to go over the svelte docs and today setting up a demo project that i can try to simulate the backend and frontend communication, which i will replicate in my project.

ive got a fair idea of what i need to do. so yeah, big code marathon coming up. :)

wahoooo!!! created a frontend ui (with dummy data for now) with collapsible sidebars in obsidian. looks dam great. also tested my tree construction and im getting really good results.
things are lookig good!

ive attached screenshots with the og obsidian sidebar opened and closed and my sidebars collapsed aswell

wrote the treeroot object that will hold the entire virtual file structure in it. the idea is that all manipulations to the tree will happen via this. while real file structures already exist and need not maintain a “map”, since all tg based folders are virtual, we need to keep a virtual track.

this virtual tree needs to be in constant sync with the actual files. i dont know how im going to do that. for eg if a person is typing and adds #subject, my event listener will instantly add this file to the subject folder. but if the user continues to type #subject/math, this will trigger the event listener again. that is bad. a typo is even worse.

so i gotta figure out how im going to do this and then maybe later validate the places where this file is present in real and in the tree and see if they match overwritng the real with the tree. lets see

wrote the interface for the tree root object. might need to rework it. i already cut down a lot of it. lets see.

this is a project ive been working on for quite some time, but never gotten to finishing it.

just updated the readme today.