From 2baaa531915cefbb857100654e6c7518ee1dbf31 Mon Sep 17 00:00:00 2001 From: _ov4 Date: Fri, 16 Oct 2026 17:50:41 +0000 Subject: docs(tests): drop Python from the test setup Co-Authored-By: Claude Fable 5.1 --- AGENTS/ARCH.md | 4 ++-- AGENTS/TESTING.md | 4 ++-- doc/build.md | 2 +- tests/.gitignore | 1 - tests/README.md | 4 ++-- 5 files changed, 7 insertions(+), 8 deletions(-) delete mode 100644 tests/.gitignore diff --git a/AGENTS/ARCH.md b/AGENTS/ARCH.md index 175a607..83a1923 100644 --- a/AGENTS/ARCH.md +++ b/AGENTS/ARCH.md @@ -256,7 +256,7 @@ source downloads remain available. The Linux compiler and plugin jobs use Ubuntu 24.04 x86_64; the macOS compiler job uses a macOS 15 arm64 runner, and Windows uses Windows Server 2025 x86_64. All four check out the push SHA and run independently after cleanup. All three -compiler jobs use LLVM/Clang/LLD 23, Python, Release and EMBEDDED settings, Pandoc, and the +compiler jobs use LLVM/Clang/LLD 23, Python (build-time), Release and EMBEDDED settings, Pandoc, and the existing `release` target. Linux obtains its toolchain from apt.llvm.org; macOS uses Homebrew's LLVM 23 and builds matching static LLD with the helper above. Windows uses `msys2/setup-msys2@v2` with MINGW64, MSYS2 LLVM/Clang packages, explicit native compiler and LLVM paths, and the @@ -269,7 +269,7 @@ verifies arm64 and rejects non-system dynamic libraries. The plugin uses Node.js 22, `npm ci --ignore-scripts` and `npm run package`, then checks VSIX contents. Each job uploads directly after its own checks pass; failure in one does not block the others. Compiler jobs enable -BUILD_TESTING, install Perl and run all CTest cases with the push SHA as seed before +BUILD_TESTING, install Perl for the tests and run all CTest cases with the push SHA as seed before their package smoke checks and uploads. Failed tests retain inputs and logs as Actions artifacts for 14 days; test summaries appear even on failure. See TESTING.md. Cleanup and uploads check main's current SHA to avoid modifying nightly for a diff --git a/AGENTS/TESTING.md b/AGENTS/TESTING.md index 7403aa1..0dfc472 100644 --- a/AGENTS/TESTING.md +++ b/AGENTS/TESTING.md @@ -1,12 +1,12 @@ # Compiler testing -Compiler tests live in `tests/`; run instructions and findings are in its README and FAILURES files. `BUILD_TESTING` defaults to OFF and adds a Perl dependency only when enabled. Python is already a compiler build dependency. CMake builds independent C++ reference executables with the same host ABI and registers one CTest entry per `case.json`, plus harness and generator checks. +Compiler tests live in `tests/`; run instructions and findings are in its README and FAILURES files. `BUILD_TESTING` defaults to OFF and needs only Perl (core modules) when enabled. Python remains a compiler build dependency. CMake builds independent C++ reference executables with the same host ABI and registers one CTest entry per `case.json`, plus harness and generator checks. Each case directory has `program.cps` and `case.json`. Suite i adds `expected.txt`, optional `input.txt`, input files under `files/`, expected text-file contents under `expected-files/`, and independently written trace notes with human-review status. Suite ii adds `reference.cc` and `generate.pl`; the generator accepts seed and random count, emits the total record count followed by one ASCII record per line, and each program emits one result line per record. Suite iii stores an `errors` list of `{reason: regex, line: integer}` in its manifest. A cause and its line must appear on the same error diagnostic. EOF after a final newline is the following source line. Optional manifest fields are `exit_code` for expected program status (default 0) and `timeout` for execution/reference seconds (default 30; file-intensive cases use 180). Compilation has a 120-second timeout. Do not make a known compiler defect pass by changing the expectation, adding an exemption or comparing only a permissive substring of program output. -The Python runner accepts `--case`, `--cpsc`, `--run-mode`, `--perl`, `--reference`, `--seed`, `--count` and `--results`. CMake provides paths, including executable suffixes, and caches `CPSC_TEST_SEED` and `CPSC_TEST_COUNT`. The default seed is HEAD read at execution, not configure time. CI overrides it with the checked-out push SHA. Both native and enabled runner paths are tested independently; NONE skips only runner execution. +The Perl runner (`tests/run.pl`) accepts `--case`, `--cpsc`, `--run-mode`, `--perl`, `--reference`, `--seed`, `--count` and `--results`. CMake provides paths, including executable suffixes, and caches `CPSC_TEST_SEED` and `CPSC_TEST_COUNT`. The default seed is HEAD read at execution, not configure time. CI overrides it with the checked-out push SHA. Both native and enabled runner paths are tested independently; NONE skips only runner execution. All commands run without a shell and retain their exact argv, working directory, input, status and output under the case result directory. Each execution path has its own files. Existing results for that case are replaced on re-run. Avoid concurrent runs of the same case into the same results directory. Tests compare CRLF-normalized bytes without stripping whitespace. Runtime diagnostics on stdout are part of fixed expectations; successful compilation and ordinary programs require empty stderr. diff --git a/doc/build.md b/doc/build.md index 65390ed..6f80236 100644 --- a/doc/build.md +++ b/doc/build.md @@ -32,7 +32,7 @@ instead. Removing the build directory also removes its downloaded dependencies. All three execution modes need LLVM, Clang and LLD 23 development libraries, Python 3.9 or newer, and the native C/C++ development libraries to build, including `NONE`. Users of an `EMBEDDED` executable do not need LLVM or `lli`. -Compiler tests are optional (`-DBUILD_TESTING=ON`) and additionally need Perl with its core Digest::SHA module. See [the test instructions](../tests/README.md) for running CTest and replaying random inputs. CI enables the complete suite on every compiler platform. +Compiler tests are optional (`-DBUILD_TESTING=ON`) and need only Perl with core modules (Digest::SHA, JSON::PP). See [the test instructions](../tests/README.md) for running CTest and replaying random inputs. CI enables the complete suite on every compiler platform. CMake strictly requires LLVM development libraries and both C/C++ compilers to have major version 23. The compilers must be upstream Clang; GCC and AppleClang diff --git a/tests/.gitignore b/tests/.gitignore deleted file mode 100644 index c18dd8d..0000000 --- a/tests/.gitignore +++ /dev/null @@ -1 +0,0 @@ -__pycache__/ diff --git a/tests/README.md b/tests/README.md index dd1407a..e7e9d42 100644 --- a/tests/README.md +++ b/tests/README.md @@ -1,12 +1,12 @@ # Compiler tests -Enable tests in an existing configured build (Perl with core Digest::SHA is additionally required): +Enable tests in an existing configured build (tests need only Perl with core modules): ```sh cmake -S . -B build -DBUILD_TESTING=ON cmake --build build --parallel 2 ctest --test-dir build --output-on-failure --parallel 2 -python3 tests/summarize.py build/test-results +perl tests/summarize.pl build/test-results ``` Use `ctest --test-dir build -L '^ii$' --output-on-failure` for one suite, or `-R '^i/arrays$'` for one case. CTest returns failure if any case fails; it continues through the other cases. [Historical failures and coverage gaps](FAILURES.md) are intentionally visible, with no expected-failure exemptions. -- cgit v1.3.1-4-g156e