That is exactly the problem I set out to solve when I started building Ion+ (Ion Plus). I wanted to create a low-level programming language that tracks indentation, uses clean syntax, and provides a powerful frontend foundation. Today, I am excited to share the early-stage development of the Ion+ compiler, written entirely in Python, and walk you through how the Lexer and Abstract Syntax Tree (AST) parser work under the hood.

Check out the source code on GitHub: https://github.com/YashAryaPersonal/ion-plus

What is Ion+?

Ion+ (with source files ending in .ionx) is a custom programming language project focused on building a robust compiler frontend. The ultimate vision is a C-like systems programming language, but without the clutter of traditional brackets and semicolons. Instead, it relies on clean indentation and block structures.

Right now, the project is heavily focused on the frontend compiler stages. The compiler can successfully:

• Read custom .ionx source files.
• Tokenize complex keywords, operators, literals, and identifiers.
• Detect and handle indentation and block structures perfectly.
• Parse variable declarations and basic expressions.
• Construct detailed AST-like (Abstract Syntax Tree) nodes for future type checking and code generation.

The Syntax: Clean, Readable, and Low-Level

To give you an idea of the direction Ion+ is heading, here is an example of what the syntax looks like. Notice how it blends the low-level memory concepts of C (like pointers and structs) with the highly readable block indentation of Python:

struct Buffer:
    int size
    ptr byte data

?int Buffer.validate(int max_size):
    if size <= 0 or size >= max_size:
        return none
    return size

It is designed to feel intuitive while keeping you close to the hardware!

How the Compiler Frontend Works

Building a compiler from scratch is a massive architectural challenge. For Ion+, I wrote the entire toolchain in Python.

Here is the exact straightforward flow the Ion+ compiler uses to process code:

1. File Reading: The compiler ingests the .ionx file.
2. Scanning (Lexical Analysis): The scanner.py breaks the raw text down into recognizable tokens (keywords, literals, etc.).
3. Indentation Handling: A custom post-lexer processes whitespace, tracking indentation levels to determine block scope without needing { } brackets.
4. Parsing: The core_parser.py and main_parser.py read the token stream and parse them based on the language's grammar.
5. AST Construction: Finally, the tokens are mapped into structured objects inside nodes.py to prepare for the backend semantic work.

An Inside Look at the AST Output

When you run the compiler using PYTHONPATH=. python test/main.py, it generates a beautifully structured Abstract Syntax Tree. For a simple variable declaration with modifiers (like const volatile int x = 10), Ion+ generates this precise output:

Program(
  line=1, 
  col=0, 
  body=[
    VariableDeclaration(
      line=1, 
      col=16, 
      type=Int(line=1, col=20, is_custom=False), 
      identifier=Identifier(line=1, col=20, value='x'), 
      value=IntegerLiteral(line=1, col=26, value='10'), 
      modifiers=[Const(line=1, col=7), Volatile(line=1, col=16)]
      )
    ], 
  name='test2.ionx'
)

As you can see, every single token is tracked with precise line and column numbers, which is crucial for building the custom error-reporting engine inside the compiler/Errors module!

Project Architecture

If you want to dive into the codebase, I have kept the architecture modular and easy to navigate.

• /compiler/Lexer — Where the magic of token generation and indentation handling happens.
• /compiler/Parser — The brain of the frontend, containing parsing logic and AST node definitions.
• /compiler/Errors — Custom error utility tracking.
• /test — Sample .ionx source files and the main runner entry point.

What's Next for Ion Plus?

Ion+ is still in its early stages. The Lexer and Parser ( not complete ) working beautifully, and the AST is growing every day. The next major hurdles include expanding statement handling, expression parsing, full language semantics, and eventually building out the execution backend.

In future this language will replace traditional C/C++ heap or other slow old logic with new arena based memory infrastructure which will remove traditional error causes like Double Free, Memory Leak etc.

Building a programming language and learning compiler design from scratch has been an incredible journey. If you are interested in parsing, ASTs, or language design, I would love for you to check it out!

Help and Advice Needed for improving my language!!

Star the repo on GitHub: YashAryaPersonal/ion-plus

Let's connect: You can find me on Discord here.