Ghost In The Shell

Ok im shipping. I wrote the issues that will be in the vulnerable version, I have implemented the first stage and a couple of the vulnerabilities in the second stage ohh its been hard

Updated the readme with the new structure after refactorization

Ive been working in the readmes and all that boring stuff for the damn vulnerable shell branch. Im really happy of how is it going tho, Im really passionate about what comes next

I completely rewrited the main fuction and the declaration of some functions for maintainability purposes. I applied all the things i have learned while develping the project, now the code base is way cleaner and easier to modify/debug. Im finally soon to shipping and it feels weird, im both sad and happy, I hope people enjoy the project as much as I enyojed making it

I succesfully merged the branch with patches for some security, claude rated currenty quality as 7.5, deeper improvement include complete refactorization that ILL DO but before I want to get rid of some TODOs that had been bothering me

This is getting out of hand.. After I succesfully corrected some logic bugs that leaded to vulnerabilities I went to the ones that i was more afraid of, the mem allocation issues. Its going BADLY, with normal usage shell works as expected but as soon as you stress it a little theres crashes everywhere and some memory leaks, I was able to reduce from 30 objects to just a couple but dammmn is hard

I just discover that exist a flag(-fsanitize=address,undefined ) for gcc that will show you all the memory leaks found on RunTime, knew this would made my life much easier while analyzing.

Ive been wokring in the things I marked before and trying some other ideas I had, most of them wouldnt work but at least some advance has been achieved, Ive found and corrected some minor crashes ~4 and most important have found 6 vulnerabilities that allow RCE and some other nasty stuff, im gatekeeping those until I can completely erradicate from main and upload them to the vulnerable version

So recently Ive been really into cybersecurity, its prolly going to be my career path, I just find interesting and supper funny read code and trying to guess ways to break it.

In order to improve security I have a lot of stuff to do so this hour I just marked what I think that can be dangerous and investigated a little about funny ways to break things, ill be investigating more of what I think might be dangerous and then I can correct everything

Before I can properly improve security of the program I needed to refactor ALL the code cause it was supper messy with a lot of files that made searching for a definition a ton of pain

I updated the readme, took me a while since I did a lot from last time, the shell works and could do basically anything that others can.

Also, while writing the readme I found some redundancies and functions that were not implemented as I originally thought so I cleared those up, since tomorrow Imma start with the security improvments and then when the shell is robust I will work in the highly vulnerable branch for pentesting/cibersecurity learning

HAHAHAHAHAHAHAH IT HAPPEND AGAIN,

Quick devlog just cause once again 10 hours were taken from the project and give it to me for devlogging (again)

idk why this keeps hapenning but yeah

workin in update the readme, its gon take a while

I got the terminal working, I think that features are good enough to call it a first version/prototype, you can do basically everything that you can do with any terminal such as kitty or ghostty. Some functions are pretty limited but for most general use the program will work just fine.

In the repo are a makefile and some python scripts to easilly try the features of the terminal

Security is one area where the terminal paryicularly lacks so im not shipping any time soon, Ill correct and improve and then I will ship

something super weird happened. So before this a couple days ago I had 53 hours but now that im making this devlog I have 43 ????, Im not worried tho, im devloging 15 hrs and I know I haven worked that much so, the 10 missing hours should be registering here.

I had a rough week, im in partials and well.. Is not going that well, the last two weeks Ive been super busy so couldnt work much. Fortunately, today my classes were cancelled so yesterday I worked so much and today I hope devlog about 6 hours so I can recover what I lost.

BUT I HAVENT TALKED ABOUT THE PROJECT YET!! . Im pretty proud, im changing some last things and working in the backgroundJobs but then I could say that is a pretty functional shell.. Im still not shipping yet, id like to improve security, refactorize a little and make some bug bounty. From last devlog I completed the directory self completion, it was rpetty easy, I just reused some logic from the commands completion and there we go, the difficult part was with history, prolly because I was tired but damn it took me so long to figure out how to get it to work with modifiers (such as !n !! etc) but I made it

the implementation of pipleine and globbin has been succesfull :D, now whats left to do is to correct and improve some features that already exist so I can say terminal is done. OH AND TODAY I LEARNT SOMETHING COOL, i knew that the static keyword existed but i didnt really knew what it was supposed to do. As far as I understood is that it was used to declarate and use functions only in the scope of a file (important if you used the file itself as a header) so you dont have to publicly declare it in the .h. It turns out that is different to what I thought and was pretty useful to write cleaner code.

Pipelining was horrible, globbing was ezzzz id like to say I have a lot of things to say but nahhh I just turned off my thoughts and started writing some code that actually worked

Today I learned that you can use if statemtents without the braces , instead of if(statement){}, is just if (statement) action;;. I can not see where this is usefull or at least superior than with the braces (I even found it harder to work like this) but hey, is good to know at least it exist. Pipelining is done overall, still are some test left and lack some stuff but it does a pretty decent job.

I think I havent talk enough about how Im making this without ANY external libraries, this makes parsing something pretty common to fail; This makes some bugs of pipelining completely related to this, Imma work it out but do not promise to achieve nothing, correct parsing uses AST and im pretty scared of those.

Final overview of the program: im refactoring some code to have something readable, updating readme, creating the test scripts and finally updating ALL buffers to be dynamic preventing buffer overflows and similar stuff, once this is done Ill develop the secondary (vulnerable branch) taking back the fixed buffers and some other fun stuff that I thought while making the program and then ship.

Hopefully all of that gets me at least 30 more hours so I can get that cute printer

Quick devlog !! is been a while since the last time i had the time to work in this, I was in a event in another city so I lost the whole week T.T, my objective with this project was to hit the 100 hours so I can get the 3d printer but to be fair Im already almost at the end, theres a lot of reading and testing that doesnt reflect in registered hours which make this work tedious. imma work to get the pipelines done, some basic scripts and Im done, I really need to take a break after this project.. Dont get me wrong, i loved doing it and really learned a lot in the whole procces, but the codebase is so big and complicated that it gives me headaches and makes really difficult to keep working.

With that said, i litterally just refactored some code (especifically builtin commands) and made it so the output can be redirected using “>” operator , the scripts for testing are getting way difficult that it should, it was a design thing that has to be with the autocompletion

Im working with python to create some scripts for quick testing, wasnt as easy as i thought but the design of the shell itself made it difficult. Since last time I had to modify a lot of stuff, specially about builtin commands. im working in pipeling but damn its hard, it will take a while this is a rather short devlog cause i havent made much since last two, most of the things i did was refactorizing and adding a little of the pipeline logic

something that took me a while to figure out is that redirection was only working for binaries.. yeah its embarassing, I did suspect it tho I was just too lazy to test it but now that Im closer to the end well, is neccesary SPECIALLY FOR PIPELINNG omg i was just reading the concept at its so confusing, it wont be a nice ride

In this devlog besides yap and cry a lil about that Imma add that im rewriting the whole builtin logic so it can be modified with operators (> , | , &&)

Also the && is almost there, some problems such as compatibility with other functions seems to be a problem T.T Ive been having some crazy bugs due to that when its done Ill let yall know

preparing a set of scripts so people can auto test easily and quick, I also achieved the redirect to work with various operators (>/»), the && function will have to wait, it seems pretty complicated. Also echo builtin is bugged and idk why, it was working fine 20 minutes ago, i saw the bug while preparing the scripts expected output, is kind of stressing.. But project shall be at 60~70 % of completion

Json is trash btw

Im now trying to implement the use of several operators in the same line (command > outputFile 2> error outputFile) but for this I had to implement the token creation logic and interpretation. Overrall is working for some cases but still has some pretty bad bugs

Went a little of track to add a function that renders your cwd(just the last 2 dirs to not be excessive) like most shells do, was pretty fast and simple I just reused the logic that makes the binary execution, once you understand how strtok works the most difficult part is ensure to null terminate all

MY TOP 3 FAVORITE BUGS :
When the program crashed if you didnt enter any command
when the program crashed if you went up in the history if there was not an history yet (this ones pretty reasonable)
When i tried to implement the left and right cursor movement and failed miserably (all turned black)

With that said, the history stuff was by far the easiest thing to implement, but navigate through the history has to be the most painful thing Ive made, it was super confusing and I had to beg some friend for help but at the end I got it, with this I can say Im almost 50% complete which is incredible but is also pretty sad.. Ill keep working and hope I can get the printer

quick devlog, I started a draft of the history function where each time the user run a command, the whole argv is duped to a “history buffer” that works but is not ideal and highly problematic, when history is called then history buffer is printed.

The advance in functionality could look like is not a lot (and it is) but its because most of the time I spent documented the project, I havent even made a readme since I started and damn it took time, even tho I remembered most of the things that doesnt stop me from having to look a couple times all the code base again. The task made me realize some posible bugs and vulnerabilities the project may have which was something good so it wasnt like a waste of time

I thought that designin all the autocomplete stuff would be harder but wasnt that difficult to be honest.. is not like it didnt made me crash out several times (work with dinamic memory is horrible) but compared to other stuff Ive made for this same project, the logic wasnt really that complicated. Most of the time I spend was with errors on memory usage and crashes related to that.

Also I do learnt a lot of stuff about linux that I couldnt even imagine, this specific module helped me to learn about linux standards, syscalls, System-wide macros etc.

Trully a great project, atp Im really starting to feel the terminal and actual terminal, when I test the new changes and all is working right I even push to github those changes inside the program and it works!. Theres a lot of stuff to do in order to say this project is at least at 80% but I mean, I see the light at the end and is not that far

whats good gang, some quick info about todays work.
First of all, today is a sad day chat (Build your own shell challenge is no longer free in codecrafters) i wouldnt mind but the test scripts were pretty usefull to know everything is working as it should. I may ask chatGPT or so to made some python scripts to test functionality (o maybe not, im hater #1 of AI).

With that said today I completed the challenge of quotation (use quotes to accept specific inputs), it was hard yep, but with all this terminal stuff I have learnt some clever trings that allowed me to not suffer that much this time, is a simple lexer and parsing stuff that with some well thinked designed turn out to be easier than I expect. Theres also now this redirect thingy “>” which I knew it existed but didnt really understand it until now. Turns out that the best way to understand something really is to create it myself. In this stage what I found more interesting was system calls (write, open, close) they were pretty interesting but a little complex at first glance took me a while to fully land how it worked and file descriptors are something that I really dont want to experiment again

ok so I was right with what I said past devlog, hackatime is detecting time.. funny(? idk what are the parameters but it looks like is tracking time from 3 different subdirs I use, I mean it takes time of /combatDiary and /combatDiary/codecrafters-shell AND /combatDiary/codecrafters-shell/src idk why but yeah, quick devlog cuz of that, i havent made that much besides refactoring once again how argv extraction works cuz with this new stage previous literally wont work

I think hackatime registered incorrectly worked hours cuz aint no way what I just did was 2 hours.. So apparently you can use quotations for specific properties in a shell (I didnt knew it), obviously that thing had to be mannually implemented and oh my god was it hard, prolly the most awfull and complicated thing Ive made for this project, atp i see NULLS whenever I try to sleep.

Im proud of who the project is going whatsoever, is being really fun and im learning a lot of stuff, i will complete the challenge and then Id like to modify the code to make a hella vulernable shell and try to hack it.

Some other not so important changes I did was : I created a makefile (which was confusing at first) to compile the project because writing the command manually each time was painful and search in history slow, Refactored some code and the logic behind some functions, specifically about the ones related to extract argc and argv of commands. Oh and its not a change but i finally hit the #1000 mark in this challenge

did yall knew that theres a whole function used to create a new proccess without the need of sys calls?.. well, i certainly didnt know and figured it out way too late in the development. It wasnt that hard tho, but it gave me a headache when implementing the usage of executing bins with arguments, at the end I decided that copy how C compiler has the argc argv thing will work. It was tricky to implement but pretty simple, by now this terminal has some pretty basic built in functions and the ability to use system binaries, is going great

HI guys, idk where to start.. IM CRASHING OUT FOR GODS SAKE, turns out theres more than a lot of considerations you need to have in order to have a working terminal.. also theres a ton of c std libraries i feel fear of the people that remember more that stdlib, string and stdio… that was my reaction when I had to use unistd. today I tried to implement the function that search for local binaries it works at most but its also bugged as hell and highly vulnerable
I know where the problems come from but god is it confusing, ill be working on that

did yall knew that scanf just scans until it founds a blank space? WELL I DIDNT and passed like half an hour debugging echo cuz it wasnt working. Well for today I just implemented echo command.. Im following the codecrafters challenge as a guide but once Im done with their program id like to implement sum functions of my own u know?. All that test stuff is horrible btw, the other half an hour of implement echo command was because test keep being rejected thanks to a blank space in “: command not found” i got sum fun ngl

I created a loop that keeps prompting for a command until the exit command is written (it doesnt accept any command yet)