From 0b4cdf7b28b3855d2dee18d91e9319d66a8ec25f Mon Sep 17 00:00:00 2001 From: _ov4 Date: Sun, 26 Jul 2026 11:15:57 +0000 Subject: docs: add AGENTS/ architecture docs for AI-assisted development ARCH.md holds the overview (pipeline, build/run, layout); details are split into CONVENTIONS.md, EXTENDING.md, SUBSYSTEMS.md and PITFALLS.md. All claims were verified against the source, most also empirically by compiling and running test programs. Co-Authored-By: Claude Fable 5 --- AGENTS.md | 3 ++ AGENTS/ARCH.md | 66 +++++++++++++++++++++++++++++++++++++++++++ AGENTS/CONVENTIONS.md | 77 +++++++++++++++++++++++++++++++++++++++++++++++++++ AGENTS/EXTENDING.md | 37 +++++++++++++++++++++++++ AGENTS/PITFALLS.md | 41 +++++++++++++++++++++++++++ AGENTS/SUBSYSTEMS.md | 29 +++++++++++++++++++ 6 files changed, 253 insertions(+) create mode 100644 AGENTS.md create mode 100644 AGENTS/ARCH.md create mode 100644 AGENTS/CONVENTIONS.md create mode 100644 AGENTS/EXTENDING.md create mode 100644 AGENTS/PITFALLS.md create mode 100644 AGENTS/SUBSYSTEMS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..6f7c200 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,3 @@ +Refer to all markdown files in `AGENTS/` + +@AGENTS/ARCH.md diff --git a/AGENTS/ARCH.md b/AGENTS/ARCH.md new file mode 100644 index 0000000..63087ac --- /dev/null +++ b/AGENTS/ARCH.md @@ -0,0 +1,66 @@ +# Architecture + +`cpsc` compiles CIE A-Level 9618 pseudocode to LLVM IR. Classic three-stage +Kaleidoscope-style design, C++17, no external deps beyond LLVM 18 and libc. + +``` +stdin ──> Lexer ──> Parser ──> vector> ──> CodeGen ──> textual LLVM IR on stderr + (getchar) (recursive (flat program list; (one llvm::Module "cps_module", + descent) FunctionDefAST mixed in) everything else inside i32 main()) +``` + +Fixed value representation (ABI — much code infers types backwards from these): +INTEGER = `i64`, REAL = `double`, BOOLEAN = `i1`, CHAR = `i8`, STRING = opaque `ptr` +to a NUL-terminated buffer. + +## Build & run + +The compiler stops at textual IR. Making a runnable binary is manual: + +```bash +mkdir -p build && cd build && cmake -DLLVM_DIR=/usr/lib/llvm-18/cmake .. && make # build cpsc +./cpsc < prog.cps 2> prog.ll # stdin is the ONLY input; IR goes to STDERR +clang-18 prog.ll -o prog && ./prog +``` + +Three facts about the driver (`tools/driver/main.cc`, 17 lines) that break naive automation: + +- **No argv.** `./cpsc file.cps` blocks forever on stdin; you must redirect `<`. +- **IR and error messages share stderr.** A failed parse still dumps (partial) IR, + prefixed by `Error:` lines that make clang reject the file. +- **Exit code is always 0**, even on errors. Grepping stderr for `Error:` catches most + failures but has false negatives: some parse paths return nullptr without printing + anything (an identifier statement not followed by `<-`/`[`, an unrecognized statement + token), the keyword builtins' arity checks print without the `Error:` prefix + (`LENGTH expects 1 arg`), and the identifier-route builtins + (ASC/CHR/IS_NUM/NUM_TO_STR/STR_TO_NUM) fail arity checks silently. Those statements + just vanish from otherwise-valid IR. The only reliable check is running the program + and diffing its output. + +Build pin: **LLVM 18 only.** LLVM 19 breaks compilation (`ArithmeticHandler.cc` uses +`llvm::Module` methods relying on LLVM 18's transitive `#include`; fix would be adding +`llvm/IR/Module.h`). + +## Layout + +| Path | Contents | +|---|---| +| `include/cps/` | All headers, included as `"cps/Foo.h"` | +| `lib/Lexer/Lexer.cc` | Entire lexer: one function, `Lexer::gettok()` | +| `lib/Parser/Parser.cc` | Statements, expressions, precedence table | +| `lib/Parser/ParserFunction.cc` | FUNCTION/PROCEDURE/CALL/RETURN parsing | +| `lib/CodeGen/CodeGen.cc` | Hub: dispatch, coercion, control flow, INPUT/OUTPUT, main() | +| `lib/CodeGen/*Handler.cc`, `FunctionGen.cc`, `RuntimeCheck.cc`, `TypeSystem.cc` | Helpers owned by CodeGen | +| `CMakeLists.txt` | 3 static libs (CPSLexer, CPSParser, CPSCodeGen) + `cpsc`. **New .cc files must be added to the source lists here** | + +AST nodes live in `include/cps/AST.h` (+ `FunctionAST.h`); they are pure data with +virtual destructors only — no codegen/visitor methods. + +## Detail docs + +| File | Read it when | +|---|---| +| `AGENTS/CONVENTIONS.md` | Before writing any code — cross-module invariants whose violation breaks code you didn't touch | +| `AGENTS/EXTENDING.md` | Adding a statement, builtin, operator, type, runtime check, or libc dependency — exact recipes | +| `AGENTS/SUBSYSTEMS.md` | Working on arrays, functions/procedures, or runtime checks | +| `AGENTS/PITFALLS.md` | Verified sharp edges (they *will* bite) and the list of unimplemented 9618 features | diff --git a/AGENTS/CONVENTIONS.md b/AGENTS/CONVENTIONS.md new file mode 100644 index 0000000..e3c67de --- /dev/null +++ b/AGENTS/CONVENTIONS.md @@ -0,0 +1,77 @@ +# Load-bearing conventions + +These invariants span modules. Violating any of them breaks code you didn't touch. +(Overview and build instructions: `ARCH.md`.) + +**Tokens are raw `int`.** `Lexer::gettok()` returns negative values for the `Token` +enum (`include/cps/Lexer.h`) and non-negative values for literal ASCII chars (`'('`, +`'+'`, `'<'`). Parser and CodeGen switch on both forms interchangeably; `BinaryExprAST` +stores this same int as its operator. Payloads (identifier text, literal values) arrive +out-of-band via public `Lexer` fields (`IdentifierStr`, `StringVal`, `NumVal`, +`RealVal`, `CharVal`) that must be read before the next `gettok()`. + +**Keywords are UPPERCASE and case-sensitive.** `declare` silently lexes as an +identifier. Assignment is `<-` (ASCII only), equality is single `=`, inequality `<>`, +concatenation `&` (reaches the parser as raw ASCII 38 — no token). No escape sequences +in string/char literals. Comments are `//` to end of line. + +**The lexer can run once per process, from stdin only.** `gettok()` keeps its lookahead +in a function-local `static int LastChar` and calls `getchar()` directly. No re-lexing, +no file paths, and unit tests can't lex two programs in one process. + +**Types travel as uppercase `std::string`** (`"INTEGER"`, `"REAL"`, `"BOOLEAN"`, +`"CHAR"`, `"STRING"`, `"VOID"`) — never an enum. `TypeSystem` resolves the string to +`TypeInfo{LLVMType, ElementSize, Kind, ...}`. The parser's `ParseTypeName` accepts *any* +identifier as a type name except `VOID` (rejected outside a RETURNS clause via its +`AllowVoid` flag); other unknown types only fail (softly) in CodeGen. + +**Fixed IR representation per type.** INTEGER = `i64`, REAL = `double`, BOOLEAN = `i1`, +CHAR = `i8`, STRING = opaque `ptr` to a NUL-terminated buffer. Much code infers +pseudocode types *backwards* from LLVM types (`getExprTypeInfo`, `ArithmeticHandler`'s +`isIntegerTy(64)` tests, BYREF detection), so these widths are ABI. + +**Strings are malloc'd and never freed — by design.** Every string-producing operation +(`&`, MID/LEFT/RIGHT/LCASE/UCASE, NUM_TO_STR, CHR, CHAR/number→STRING coercion) mallocs +a fresh buffer; string literals and BOOLEAN→STRING coercion instead return pointers to +shared module globals, so a STRING value may alias a global. `free` is declared but has +zero call sites. Don't "fix" leaks piecemeal; it's a global design decision. + +**One flat symbol namespace.** Two parallel maps in CodeGen — `NamedValues` +(name → storage `Value*`) and `Symbols` (name → `SymbolInfo{Storage, TypeName, +IsArray}`) — must always be updated in lock-step: via `registerSymbol()`, or directly +but both together as `FunctionGen::createArgumentAllocas` does for parameters. (The +per-type handlers' `emitDeclare` methods set `NamedValues` alone — that is dead code; +don't imitate it.) There is no scope stack. `FunctionGen` saves/clears/restores both +maps around a function body, but `ArrayHandler::ArrayTable` is *not* scoped (see +`PITFALLS.md`). + +**Dispatch is by `dynamic_cast` chain, not virtual methods.** `CodeGen::emitStmt` and +`emitExpr` walk a linear if-chain of casts. **A statement type missing from the chain is +silently ignored** — no error. Adding an AST node always means touching these chains. + +**Error style everywhere: `fprintf(stderr, ...)` then return nullptr/void, and keep +compiling.** No exceptions, no exit, no error counter. Most messages start with +`Error:`; the keyword builtins' arity checks in `emitExpr` lack the prefix, and the +identifier-route builtins' arity checks and several parse failures print nothing at all. +The parser drops a failed statement, skips one token, and continues, so erroneous +statements simply vanish from the program. + +**BYREF is a pointer-typed parameter, inferred structurally at call sites.** +`FunctionGen` emits pointer params for BYREF; call sites (in CodeGen, duplicated for +call-expr and call-stmt) treat *any* pointer-typed callee param as BYREF and require the +argument to be a bare variable whose alloca is passed. The callee side, however, binds +by the *declared* flag. Since STRING itself lowers to `ptr`, the two sides disagree for +STRING: the caller always passes the alloca, but a declared-BYVAL STRING param (the +default) copies that alloca *address* and treats it as the string data pointer — reading +garbage at runtime. Only declared-BYREF STRING parameters work, and STRING arguments can +never be literals/expressions. Function names get no mangling — a pseudocode +`FUNCTION printf` collides with libc. + +**Builtins have two routes and one trap.** LENGTH/MID/RIGHT/LEFT/LCASE/UCASE are +reserved keywords with tokens, normalized back into ordinary `CallExprAST` by the +parser; ASC/CHR/IS_NUM/NUM_TO_STR/STR_TO_NUM need no lexer/parser support at all — +identifiers may contain `_` and `Name(args)` already parses as a call. Both routes are +intercepted by name-string comparison in `emitExpr`'s `CallExprAST` branch *before* the +user-function fallback. The trap: builtin return types are duplicated in +`getExprTypeInfo`'s hard-coded table — forget that second edit and OUTPUT prints the +result with the wrong format, with no compile error. diff --git a/AGENTS/EXTENDING.md b/AGENTS/EXTENDING.md new file mode 100644 index 0000000..62cdf40 --- /dev/null +++ b/AGENTS/EXTENDING.md @@ -0,0 +1,37 @@ +# Extension recipes + +Step-by-step patterns for adding features. Read `CONVENTIONS.md` first — every recipe +below assumes those invariants. + +**New statement:** token in `Lexer.h` enum + keyword match in `Lexer.cc`'s if-chain → +node class in `AST.h` (unique_ptr children, move ctor, raw-ptr getters) → `ParseXxx` in +`Parser.cc` + branch in `ParseStatement` (copy the existing body-loop pattern for block +statements — terminator keyword + tok_eof guard) → `dynamic_cast` branch in +`CodeGen::emitStmt`. Guard every fall-through branch in control-flow codegen with +`if (!Builder->GetInsertBlock()->getTerminator())` — a RETURN in the body may have +already terminated the block. + +**New builtin function:** implement IR emission as a method on the fitting handler +(String/Char/StringConversion), add the name intercept in `emitExpr`'s CallExprAST +section with an arg-count check, **and** add the name to `getExprTypeInfo`'s table. +Identifier-shaped names need nothing in lexer/parser. + +**New binary operator:** token (if multi-char) → one line in `BinopPrecedence` +(Parser ctor; levels: OR=3, AND=5, comparisons=10, `+ - &`=20, `* / DIV MOD`=40; +precedence must be > 0) → branch in `ArithmeticHandler::emitBinaryOp`, which owns *all* +binary ops and dispatches on raw LLVM operand types. + +**New type:** register in `TypeSystem::registerBuiltins` (correct `ElementSize` — +arrays malloc `count * ElementSize`) + `TypeKind` case + `getZeroValue` + +`coerceValueToType` + `emitOutputValue` + INPUT branch in CodeGen + lexer keyword + +`ParseTypeName` case. + +**New runtime check:** message global + `emitXxxCheck` in `RuntimeCheck` (build an i1 +fail-condition, call `emitErrorAndExit` — it splits the block and leaves the builder at +the continuation), then call it from the emit site with the AST node's `getLine()`. +Precedent: `emitIndexCheck`, called per-dimension from ArrayHandler. Note that failure +messages print to **stdout** (`[Fatal] line %d: ...`) and `exit(1)`. + +**New libc dependency:** declare via `getOrInsertFunction` in the owning component's +`setupExternalFunctions()` (re-declaring the same symbol in two handlers is fine — it +dedupes by name). diff --git a/AGENTS/PITFALLS.md b/AGENTS/PITFALLS.md new file mode 100644 index 0000000..3b123fe --- /dev/null +++ b/AGENTS/PITFALLS.md @@ -0,0 +1,41 @@ +# Pitfalls + +## Sharp edges (verified, will bite) + +Every bullet below was confirmed against the source, most also by compiling and running +test programs. + +- **STRING `=` / `<>` compares pointer identity, not contents.** No strcmp anywhere; + two equal heap strings compare unequal. +- **Descending FOR loops never work.** CodeGen picks the loop direction at compile time + by dynamic_casting the STEP to a negative `IntegerExprAST` — but `-1` parses as the + expression `0 - 1` (the lexer has no signed literals), so that branch is dead code and + *every* FOR compiles with an ascending `<=` test. `FOR i <- 5 TO 1 STEP -1` runs zero + iterations (empirically verified). +- **OUTPUT takes exactly one expression.** `OUTPUT a, ", ", b` silently drops everything + after the comma — and if the dropped tail ends in an identifier, error recovery also + swallows the first token of the *next* statement. Concatenate with `&`. Similarly + INPUT only accepts a plain identifier — no `INPUT arr[i]`. +- **Arrays cannot cross function boundaries.** `ArrayTable` is never scoped; metadata + SSA values belong to the declaring function, so referencing another function's array + produces invalid IR, and arrays can't be parameters. +- **`/` always yields REAL** (double FDiv), even for two INTEGERs; integer division is + DIV/MOD only. CHR returns a STRING (2-byte heap buffer), not CHAR. STR_TO_NUM always + returns REAL. NUM_TO_STR formats i64 with sprintf `"%d"` — corrupts values over 32 + bits. +- **`getExprTypeInfo` returns nullptr** for expressions it can't classify (null-check it + at any new call site) and internally defaults to INTEGER for arithmetic binaries and + unknown-function calls; OUTPUT substitutes INTEGER for a nullptr result — so wrong + inference prints garbage instead of erroring. +- The lexer emits no newline tokens; the grammar is purely keyword-delimited. Bare + `RETURN` is only recognized right before ENDIF/ELSE/ENDFUNCTION/ENDPROCEDURE. +- `verifyFunction` results are ignored; invalid modules still print. +- The FOR loop variable must be pre-DECLAREd as INTEGER; FOR reuses that alloca. + +## Not implemented (9618 features) + +CASE OF, CONSTANT, TYPE/ENDTYPE (records/enums/pointers), file handling +(OPENFILE/READFILE/WRITEFILE/CLOSEFILE/EOF), DATE, multi-expression OUTPUT, INPUT into +array elements, string escape sequences. `README.md` lists `type` and `file` as the +intended next work. No tests and no CI exist; verify changes by compiling sample +programs end-to-end (command sequence in `ARCH.md`). diff --git a/AGENTS/SUBSYSTEMS.md b/AGENTS/SUBSYSTEMS.md new file mode 100644 index 0000000..2910bb3 --- /dev/null +++ b/AGENTS/SUBSYSTEMS.md @@ -0,0 +1,29 @@ +# Subsystem notes + +Deep-dives on the three CodeGen helper subsystems. (Invariants: `CONVENTIONS.md`; +known bugs: `PITFALLS.md`.) + +**Arrays** (`ArrayHandler`): a DECLAREd array is one malloc'd flat buffer (memset to +zero), row-major, any rank, any bounds. Bounds and multipliers are *compile-time SSA +values* cached in `ArrayMetadata` (global `ArrayTable` map, keyed by name). Every access +goes through `getMetadata` → per-dimension `emitIndexCheck` → `computeFlatIndex` → +`getElementPointer`; never hand-roll indexing. STRING elements read as null until +written; every read path must replicate the null-guard `select` to the empty string. +`OUTPUT arr` (and partially-indexed accesses) is intercepted by +`ArrayHandler::tryEmitArrayOutput` — the first branch of OUTPUT dispatch, emitting +nested print loops — so any rework of OUTPUT must preserve that interception. + +**Functions** (`FunctionGen`): plain LLVM functions, external linkage, exact pseudocode +name. Params are `tuple`. BYVAL args are copied into +entry-block allocas; BYREF args bind the incoming pointer directly as the variable's +storage. Missing RETURNS defaults the return type to INTEGER; procedures use "VOID". +**Definitions must precede first call**: calling a not-yet-defined function fabricates +an extern declaration (i64 return, param types guessed from the arguments), and when the +real definition arrives, `emitFunctionDef` reuses that declaration by name without +checking the signature — the body attaches to the wrong prototype and the module is +invalid. Body emission is inverted: CodeGen injects a `StmtEmitter` callback, and must +restore its own insert point afterwards. + +**Runtime checks** (`RuntimeCheck`): only array bounds checking is actually wired up. +`emitDivZeroCheck` exists but has **zero call sites** — division by zero (for `/`, DIV +and MOD) is currently unchecked; the TODO sits in `ArithmeticHandler.cc`. -- cgit v1.3.1-4-g156e