Abstract Syntax Seed

I keep pressing keys until I have a fully functional lexer.

Using the black magic we developed for the regex engine we can easily create a state machine for any set of lexer rules.

On the journey to reinvent the wheel yet again while learning formal languages, it is time to build a lexer to generate old-school non-AI tokens

Working as a software engineer is much more than writing code.

Defining the classes of an AST can be cumbersone due to the high amount of nodes and redundant code.

The error recovery PoC works, but it's both underperforming and has a big flaw: It doesn't work with repetitions

Autocompletion has a fatal flaw.

At first glance, rules might seem like a odd and complex mechanism, specially since they allow recursion.

The autocompleter works by traversing the ATN, which is ANTLR4's state machine.

ANTLR4 grammars are commonly used to build parsers and syntax analyzers.

Grammars define a language, and a language is a set of strings...

With all the features of the engine finished the only question left is, what would have changed if instead of a NFA engine we had made a DFA engine?

Backreferences match the same text captured by a previous capturing group.

Atomic groups can be used to avoid unnecessary backtracking and optimize regex.

We finally modernize the interface of the regex to allow searching for multiple matches

The start and end of string anchors (^$) allow to check that a string starts or ends in some way.

With character classes you can test if a character is inside a certain set of symbols.

A formal regex engine is cool but kind of useless.

We complete the engine for formal regex by translating a regex engine to a NFA

Let's talk about exacly what's a finite automata, its two types (DFA and NFA), and how to implement a NFA

Just enough information about formal language theory to answer "What is really a regex?"