diff options
| author | _ov4 <ov4@ov4.org> | 2026-10-27 14:07:24 +0000 |
|---|---|---|
| committer | _ov4 <ov4@ov4.org> | 2026-10-27 14:07:24 +0000 |
| commit | caa88a9b36d9e1dfae40ecb31aee05bc2891fa17 (patch) | |
| tree | d41168dc0799c6f8b033a3d6fdb2c2c72aac2b42 /AGENTS/CONVENTIONS/LEXING.md | |
| parent | 5d37ebcc02f6ca2ce1ced93f803dd1daf634d866 (diff) | |
docs(preprocessor): document switches and library regions
Add manuals and aliases for all five PP directives, replace the retired #LIB
page, and update compiler and shared-editor architecture notes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Diffstat (limited to 'AGENTS/CONVENTIONS/LEXING.md')
| -rw-r--r-- | AGENTS/CONVENTIONS/LEXING.md | 13 |
1 files changed, 12 insertions, 1 deletions
diff --git a/AGENTS/CONVENTIONS/LEXING.md b/AGENTS/CONVENTIONS/LEXING.md index 81b01d1..654d7ae 100644 --- a/AGENTS/CONVENTIONS/LEXING.md +++ b/AGENTS/CONVENTIONS/LEXING.md @@ -20,7 +20,8 @@ identifier. Assignment is `<-` (ASCII only), equality is single `=`, inequality concatenation `&` (reaches the parser as raw ASCII 38 — no token). Identifiers use ASCII case-insensitive lookup/comparison through `IdentifierMap`, `IdentifierSet` and `sameIdentifier`; retain source spelling in ASTs and diagnostics. This includes -functions, types, enum values, fields and builtins, but not C_TYPE import strings. +functions, types, enum values, fields and builtins, but not C_TYPE import strings +or preprocessor switch names. Integer tokens carry a checked unsigned magnitude through lookahead; the parser applies the sign and i64 range check, including the minimum value. String/char literals support C++17 byte escapes and Unicode escapes (UTF-8 in STRING, ASCII only @@ -32,3 +33,13 @@ reserved in every user identifier, including EXTERN declarations. **The lexer reads a caller-owned stream.** Construct it with `Lexer(std::istream &)`; the stream must outlive the lexer. Lookahead, line number and error state belong to each instance. The driver opens the source file; stdin is not used for compilation. + +**Preprocessing precedes lexing.** `preprocess(std::istream &)` returns masked source, +per-line __LIB states and an error flag. PP_DEFINE/PP_UNDEF change case-sensitive, +file-local switches; PP_IFDEF/PP_IFNDEF/PP_ENDIF maintain a nested conditional stack. +Directives occupy a line, with optional leading whitespace and trailing // comments. +Disabled ordinary source is never tokenized; DEFINE/UNDEF in it have no effect. +Masking preserves byte lengths, CRLF and original source line numbers. The driver +owns the processed stream passed to Lexer. PP_* keywords remain reserved outside +valid directives. There is no macro expansion, value, PP_ELSE or switch propagation +through INCLUDE; line-leading # directives are errors. |