devlog 2 - subtraction and integers
in this devlog, i added subtraction and the ability to add integers of variable length. it may not sound like much, but i’m trying to provide a good base for the rest of the interpreter. i have my obsidian notes on lexical analysis pasted below, but i’m not sure how well it will format. for the next devlog, i plan to implement operator precedence and multiplication + division, and also learn about writing tests in go (since writing tests is probably a smart thing to do)
notes
Lexical Analysis is a stage in interpretation that turns a string of text into analytical tokens, typically for a later stage like the parser.
What is a token?
A token is a type and a value. For example, an integer 8 would be converted into a token of type: Integer and value: 8.
How-to Lexical Analysis
The lexical analyzer, or lexer, breaks down a string of characters into tokens. It goes character-by-character, and has no context for the surrounding characters. It then passes the list of tokens to the interpreter to evaluate.
Lexemes
A lexeme is a collection of characters forming a token. For example, this can include simple 1 character tokens (operators, parenthesis, etc.), but also include integers, floats, etc.
A relationship between token types and lexemes can be seen in the table below:
Token Sample Lexems INTERGER 142, 19, 23, 1 PLUS + MINUS -Log in to leave a comment