From ec0ae47dd798e48e2409cbc544e17e2b13f550eb Mon Sep 17 00:00:00 2001 From: _ov4 Date: Tue, 13 Oct 2026 11:21:13 +0000 Subject: docs: document local dependency downloads on all build platforms Co-Authored-By: Claude Fable 5.1 --- AGENTS/ARCH.md | 31 ++++++-- doc/build.md | 218 ++++++++++++++++++++++++++++++++++++++++++++------------- 2 files changed, 193 insertions(+), 56 deletions(-) diff --git a/AGENTS/ARCH.md b/AGENTS/ARCH.md index baa8a74..e5801a3 100644 --- a/AGENTS/ARCH.md +++ b/AGENTS/ARCH.md @@ -26,14 +26,28 @@ The compiler emits textual IR. Initial configuration must explicitly select development libraries to build; NONE still generates IR. Native binaries remain manual: ```bash -mkdir -p build && cd build -cmake -DCPSC_RUN_MODE=EMBEDDED -DLLVM_DIR=/usr/lib/llvm-18/cmake .. && make -./cpsc prog.cps -o prog.ll # default without -o: prog.ir beside the source -./cpsc prog.cps -r # execute using the selected build mode -./cpsc prog.cps -r -o prog.ir # save IR, then run it +# First download dependencies as documented in doc/build.md. +cmake -S . -B build -DCPSC_RUN_MODE=EMBEDDED \ + -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ +cmake --build build +build/cpsc prog.cps -o prog.ll # default without -o: prog.ir beside the source +build/cpsc prog.cps -r # execute using the selected build mode +build/cpsc prog.cps -r -o prog.ir # save IR, then run it clang-18 prog.ll -o prog && ./prog ``` +CMake, make and Clang are system tools. Other build dependencies are downloaded +to the configured build directory's `deps/packages/`. The modular +`cmake/LocalDependencies.cmake` entry point extracts Debian, MSYS2 and Homebrew +binary packages into `deps/root/`, then searches each dependency locally before +falling back to system libraries. Local LLVM overrides a cached system `LLVM_DIR`. +Platform modules relocate package paths only inside the extraction directory; +Homebrew bottles also receive local Cellar/opt links and Mach-O relocation/signing. +Package hashes skip unchanged extractions; a changed package set rebuilds the +managed root. Downloads belong to the package-manager commands in `doc/build.md`, +not to CMake. The Windows Pandoc ZIP is downloaded directly from its official +release because MSYS2 has no Pandoc package. Only the Linux build is verified. + Driver behavior (`tools/driver/main.cc`): - One source file, with optional `-o path` and `-r`; options may precede or follow the source. @@ -68,7 +82,10 @@ On Linux, `make -C build install` builds the manual and calls the POSIX `install.sh` to copy `cpsc` and `doc/cpsc.1` into `CMAKE_INSTALL_PREFIX/bin` and `CMAKE_INSTALL_PREFIX/share/man/man1` (default prefix `/usr/local`). The script also supports `--prefix`, `--source-dir` and `DESTDIR` staging. Ordinary builds -do not require Pandoc; `doc`, installation and release packaging do. +do not require Pandoc; `doc`, installation and release packaging do. The modular +`cmake/Manual.cmake` target defers Pandoc discovery until manual generation, prefers +the extracted executable, and sets its local runtime/data paths as needed. System +Pandoc remains the fallback. `make -C build release` packages the executable, manual, installer and license as `build/cpsc-VERSION-SYSTEM-ARCH.tar.xz`, retaining the configured execution mode. `cmake/Release.cmake` reads HEAD at packaging time; VERSION is the commit's UTC @@ -117,6 +134,8 @@ untested.) | `lib/CodeGen/*Handler.cc`, `FunctionGen.cc`, `RuntimeCheck.cc`, `TypeSystem.cc` | Helpers owned by CodeGen | | `CMakeLists.txt` | 3 static libs (CPSLexer, CPSParser, CPSCodeGen) + `cpsc`. **New .cc files must be added to the source lists here** | | `tools/driver/EmbeddedRunner.cc`, `cmake/EmbeddedLLVM.cmake` | EMBEDDED-only MCJIT runner and static support-library selection | +| `cmake/LocalDependencies.cmake`, `cmake/*Packages.cmake` | Local package extraction, platform relocation and dependency discovery | +| `cmake/Manual.cmake`, `cmake/GenerateManual.cmake.in` | Optional manual generation with local-first Pandoc discovery | | `install.sh`, `install.ps1`, `cmake/Install.cmake.in`, `cmake/Release.cmake` | Artifact installation and versioned binary release packaging | AST nodes live in `include/cps/AST.h` (+ `FunctionAST.h`, `FileAST.h`, `TypeAST.h`); diff --git a/doc/build.md b/doc/build.md index 7e4cc7c..18165b7 100644 --- a/doc/build.md +++ b/doc/build.md @@ -1,44 +1,108 @@ # Building cpsc -Run these commands from the project root unless stated otherwise. For installation and release packaging, see [install.md](install.md). +Run these commands from the project root. For installation and release packaging, +see [install.md](install.md). ## Dependencies -These are build dependencies. All three modes use LLVM to generate IR, so even -`NONE` requires LLVM development libraries when building the compiler. -Users of an `EMBEDDED` executable do not need to install LLVM or `lli`. +Install CMake, make and Clang as system tools. Download the other dependencies to +`build/deps/packages/`; CMake extracts them into `build/deps/root/` during +configuration. Each dependency is searched locally first, then in the system. +Local LLVM takes precedence over a previously cached system `LLVM_DIR`. + +Unchanged packages are not extracted again. Keep one version of each package in +the download directory; remove old archives before downloading a different +version. CMake rebuilds its extraction directory when the package set changes. +If you use a different CMake build directory, download into its `deps/packages/` +instead. Removing the build directory also removes its downloaded dependencies. + +All three execution modes need LLVM development libraries to build, including +`NONE`. Users of an `EMBEDDED` executable do not need LLVM or `lli`. +LLVM 18 is the tested version; newer versions remain untested. + +### Linux (Debian/Ubuntu) + +Use a distribution repository that provides LLVM 18. Install the build tools: -Linux ```bash sudo apt-get update -sudo apt-get install -y \ - build-essential \ - cmake \ - llvm-18-dev \ - zlib1g-dev \ - libzstd-dev \ - libtinfo-dev +sudo apt-get install -y --no-install-recommends build-essential cmake clang-18 +``` + +Download LLVM and its support packages without installing them: + +```bash +mkdir -p build/deps/packages +( + cd build/deps/packages + apt-get download \ + llvm-18-dev llvm-18 llvm-18-runtime llvm-18-tools llvm-18-linker-tools \ + libllvm18 libclang-cpp18 libz3-dev libz3-4 \ + libffi-dev libffi8 libxml2-dev libxml2 \ + zlib1g-dev zlib1g libzstd-dev libzstd1 libncurses-dev libtinfo6 +) ``` -The last three packages supply static support archives for `EMBEDDED` when the -LLVM package uses them. `PATH` additionally needs `lli` at run time, for example -from `llvm-18` with `/usr/lib/llvm-18/bin` in PATH. Clang is only needed if you -want to turn the emitted IR into a native executable yourself. +`apt-get download` also downloads explicitly named packages that are already +installed. The LLVM companion packages satisfy its CMake imports, while the +development support packages supply the static archives used by `EMBEDDED`. ---- +### Windows (MSYS2 MINGW64) + +Run these commands in the MSYS2 **MINGW64** shell. Update MSYS2 first, reopening +the shell and repeating the update if requested: -mingw-w64 for Windows ```bash pacman -Syu pacman -S --needed \ - mingw-w64-x86_64-toolchain \ mingw-w64-x86_64-cmake \ + mingw-w64-x86_64-make \ + mingw-w64-x86_64-clang +``` + +Download the dependencies to the project build directory: + +```bash +mkdir -p build/deps/packages +pacman -Sw --noconfirm --cachedir "$PWD/build/deps/packages" \ mingw-w64-x86_64-llvm \ - mingw-w64-x86_64-clang \ + mingw-w64-x86_64-llvm-libs \ + mingw-w64-x86_64-llvm-tools \ mingw-w64-x86_64-lld \ - mingw-w64-x86_64-make + mingw-w64-x86_64-libffi \ + mingw-w64-x86_64-libxml2 \ + mingw-w64-x86_64-libiconv \ + mingw-w64-x86_64-zlib \ + mingw-w64-x86_64-zstd ``` +`-Sw` downloads packages without installing them; omit `--needed` here so that +already installed dependencies are also downloaded. CMake keeps the packages' +`mingw64/` layout under `build/deps/root/`. MSYS2 supplies its current LLVM +version. Windows builds have only been reviewed statically. + +### macOS (Homebrew) + +Use Clang and make from the Xcode Command Line Tools. If they are missing, run +`xcode-select --install`. Install CMake with Homebrew: + +```bash +brew install cmake +``` + +Download the binary bottles to the build directory: + +```bash +mkdir -p build/deps/packages +HOMEBREW_CACHE="$PWD/build/deps/packages" brew fetch --force-bottle \ + llvm@18 zlib zstd ncurses lz4 xz +``` + +This requires binary bottles available for your macOS version and CPU; source +archives are not used. CMake extracts the bottles into a local `Cellar/`, creates +local `opt/` links and relocates their CMake and dynamic-library paths using the +Command Line Tools. The macOS build has only been reviewed statically. + ## Build You must explicitly choose `-DCPSC_RUN_MODE=PATH`, `EMBEDDED`, or `NONE` on @@ -50,54 +114,108 @@ initial configuration; there is no default. The choice is saved in CMake's cache | `EMBEDDED` | Run with the statically linked LLVM engine inside `cpsc`; no PATH lookup. | | `NONE` | Reject `-r`; compiling to IR still works. | -Linux +Linux: + ```bash -cmake -S . -B build -DCPSC_RUN_MODE=EMBEDDED \ - -DLLVM_DIR=/usr/lib/llvm-18/cmake -DCMAKE_BUILD_TYPE=Release +cmake -S . -B build -DCPSC_RUN_MODE=EMBEDDED -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_C_COMPILER=clang-18 -DCMAKE_CXX_COMPILER=clang++-18 cmake --build build -j4 ``` -Distribute `build/cpsc` for `EMBEDDED`. LLVM, the project's libraries and the -compression/terminal support libraries are linked statically; a missing required -archive is a configure error. On Linux with GCC/Clang, the C++ runtime is static -too. The tested Debian/LLVM 18 executable needs only the system libc, libm and -dynamic loader. It runs pseudocode with `-r` as one executable, without LLVM -libraries or `lli` installed. Build for the intended operating system and CPU. +MSYS2 MINGW64: ---- +```bash +cmake -S . -B build -G "MinGW Makefiles" \ + -DCPSC_RUN_MODE=EMBEDDED -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ +mingw32-make -C build -j4 +``` + +macOS: -mingw-w64 for Windows ```bash -mkdir build && cd build/ -export CMAKE_GENERATOR="MinGW Makefiles" -cmake -DCPSC_RUN_MODE=EMBEDDED .. -mingw32-make +cmake -S . -B build -DCPSC_RUN_MODE=EMBEDDED -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_C_COMPILER=/usr/bin/clang -DCMAKE_CXX_COMPILER=/usr/bin/clang++ +cmake --build build -j4 ``` -MinGW uses static linking for `EMBEDDED`; this mode has been verified on Linux, -and the Windows build has not been tested. +Local packages do not need `-DLLVM_DIR`. When no local LLVM is present, CMake uses +the system search; you can specify `-DLLVM_DIR=/path/to/lib/cmake/llvm` for a +system installation outside its usual search paths. + +Distribute `build/cpsc` (Windows: `build/cpsc.exe`) for `EMBEDDED`. LLVM and its +required compression/terminal support libraries are linked statically; a missing +required archive is a configure error. Linux also links the C++ runtime statically, +and MinGW uses static linking. The tested Debian/LLVM 18 executable needs only +the system libc, libm and dynamic loader. Build for the intended OS and CPU. + +For `PATH` mode, add the extracted `lli` directory to your shell's PATH after +configuration. On Linux, also expose its downloaded runtime libraries: + +```bash +export PATH="$PWD/build/deps/root/usr/lib/llvm-18/bin:$PATH" +export LD_LIBRARY_PATH="$PWD/build/deps/root/usr/lib/$(cc -print-multiarch)${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" +``` + +MSYS2 MINGW64: + +```bash +export PATH="$PWD/build/deps/root/mingw64/bin:$PATH" +``` + +macOS: + +```bash +export PATH="$PWD/build/deps/root/opt/llvm@18/bin:$PATH" +``` ## Manual -The command-line quick reference is in [cpsc.md](cpsc.md). Generating the manual requires Pandoc; ordinary builds do not. +The quick reference is in [cpsc.md](cpsc.md). Pandoc is required only for the +`doc`, installation and release targets. Download it for your platform: + +Linux: -From the project root on Linux: +```bash +( + cd build/deps/packages + apt-get download pandoc pandoc-data \ + libffi8 libgmp10 liblua5.4-0 libnuma1 libyaml-0-2 zlib1g +) +``` + +macOS: ```bash -sudo apt-get install pandoc -cmake -S . -B build -DCPSC_RUN_MODE=EMBEDDED -make -C build doc -man ./build/doc/cpsc.1 +HOMEBREW_CACHE="$PWD/build/deps/packages" brew fetch --force-bottle pandoc gmp ``` -You can also run `make doc` inside the configured `build/` directory. -The output is `build/doc/cpsc.1` on Unix and `build/doc/cpsc.html` on Windows, -and is regenerated when the Markdown source changes. On Windows, install Pandoc, -then generate and open the standalone HTML manual from PowerShell: +Windows has no MSYS2 Pandoc package. Download the official x86_64 ZIP from +PowerShell, without running an installer: ```powershell +New-Item -ItemType Directory -Force build/deps/packages | Out-Null +$release = Invoke-RestMethod https://api.github.com/repos/jgm/pandoc/releases/latest +$asset = $release.assets | Where-Object { $_.name -like 'pandoc-*-windows-x86_64.zip' } +Invoke-WebRequest -UseBasicParsing $asset.browser_download_url -OutFile "build/deps/packages/$($asset.name)" +``` + +The ZIP is provided on the [official Pandoc download page](https://pandoc.org/installing.html). + +Reconfigure using your platform's build command above, then generate the manual: + +```bash cmake --build build --target doc -Start-Process .\build\doc\cpsc.html ``` -The HTML contains its own styles and can be read offline in a browser. +CMake selects local Pandoc first, sets its local runtime/data paths as needed, +and falls back to Pandoc in the system PATH. The output is `build/doc/cpsc.1` on +Unix and `build/doc/cpsc.html` on Windows. It is regenerated when the source or +downloaded packages change. + +On Unix, open it with `man ./build/doc/cpsc.1`. On Windows, open the standalone, +offline HTML from PowerShell: + +```powershell +Start-Process .\build\doc\cpsc.html +``` -- cgit v1.3.1-4-g156e