Toolchain

Hey Toolchain & System Integration

A centralized, adaptive orchestrator for NixOS, bridging Janet-based logic, NixOS module system, and Zsh performance.

🏗️ Architecture Overview

The toolchain is designed to be WM-agnostic and declarative. It centralizes duplicated logic (volume, brightness, screenshots, hooks) into a unified system that adapts at runtime to the current environment.

Core Components

• hey (Janet): The primary entry point. A high-level CLI that handles subcommand routing, state management, and hook dispatching.
• lib/hey/ (Janet Modules): Core libraries for command definition (cmd.janet), path resolution (init.janet), and system utilities.
• lib/zsh/ (Zsh Functions): Autoloaded helpers that provide instant access to system state (WM name, mode) for shell scripts.
• bin/ (Unified Scripts): The implementation layer. Adaptive scripts that perform the actual work (locking, OSD, etc.).
• justfile: A high-level task runner providing a human-friendly menu for hey and bootstrap tasks.

🧬 System-Level Integration (NixOS)

hey is a foundational layer of the NixOS configuration, not just a standalone script.

1. The hey Input Argument

In lib/nixos.nix, a special hey object is created and passed as a module argument to all NixOS modules.

• Attributes: hey.dir, hey.binDir, hey.configDir, hey.modulesDir, hey.themesDir, etc.
• Usage: Allows modules to reference dotfile assets without hardcoding paths.

2. Pure Hermetic Flake

The flake is fully hermetic — zero --impure, zero getEnv, zero HEYENV. Build-time path resolution uses toString self (the flake store path), which contains all config files, modules, and themes. No environment variables are needed for evaluation.

Where host/user/theme come from

The Janet (flake) function reads $HOST and $USER for CLI runtime context (default host for sync, path resolution). Theme is read purely from info.json (generated by modules/hey.nix). These are Janet runtime concerns — not Nix evaluation inputs.

3. Module: modules/hey.nix

Handles the system plumbing:

• Package: hey is built as a Nix derivation (packages/hey/) during nixos-rebuild. The compiled binary and all Janet modules live in the nix store, ensuring hey is available on every host (including remote VPS targets) without activation-time compilation.
• PATH: Local dev builds ($JANET_TREE/bin) take priority for login shells. The nix-store hey is installed via environment.systemPackages (available at /run/current-system/sw/bin/hey), ensuring it is on the PATH for all processes — including niri’s spawn, systemd services, and non-login shells. The local dev build at ~/.local/bin/hey (from the wrapped jpm) appears earlier in the system PATH, so it takes precedence when present. See path.md for the complete system PATH construction reference.
• FPath: Adds lib/zsh/ to the Zsh fpath for autoloading functions.
• Metadata: Generates ~/.local/share/hey/info.json for scripts to consume.

4. Hermetic Target Architecture & Custom Library Isolation

To maintain strict hermetic evaluation and prevent target architecture contamination, the codebase isolates custom library helpers from the orchestrating host system.

• Dynamic Re-instantiation: While the global lib in flake.nix is loaded early using a deterministic host system (x86_64-linux) for flake checking and CLI routing, the NixOS module system requires architecture-appropriate helper outputs. In lib/nixos.nix, when building the target host, hey.lib is dynamically re-instantiated using the target host’s specific pkgs (e.g., native x86_64-linux for lab-matrix, or cross-compiled pkgs targeting aarch64-linux for sbc-opi5p).
• Derivation Isolation: This dynamic rebinding guarantees that helper functions which package files or wrap binaries (e.g., wrapFakeHome, mkLauncherEntry, mkWrapper, and mkOutOfStoreSymlink) construct derivations targeting the exact target system architecture.
• Validation: Consequently, querying the top-level derivation (e.g., via nix derivation show) shows complete system separation:
  • lab-matrix (x86_64-linux) system contains zero references to aarch64 paths.
  • sbc-opi5p (aarch64-linux) system evaluates cleanly using aarch64 target inputs, cross-compiled without native x86_64-linux package leakage.

🐚 Shell & App Integration

Zsh Bridge (lib/zsh/)

Every shell script has access to hey.* functions:

• hey.wm.name: Detects ‘hypr’, ‘niri’, or ‘bspwm’.
• hey.wm.mode: Detects ‘dms’ or ‘traditional’ UI modes.
• hey.log: Unified logging with levels and colors.

App-Level (Theming)

• Data Flow: modules/themes/apps.nix uses theme colors to generate config for apps (Alacritty, Waybar, Rofi).
• Dynamic Updates: bin/termcolors.sh pushes live color updates to running terminals.

📖 Operator Manual

The hey toolchain is categorized by operational lifecycle. For most daily tasks, the justfile provides the fastest interface.

🛠️ Provisioning & Deployment (Day 0)

These commands are used when setting up new hardware or performing low-level disk operations.

🔄 System Maintenance (Day 1)

Commands for keeping your system updated and clean.

🧪 Development & Verification (Day 2)

Tools for refactoring modules and building custom images.

Eval Profiling (hey check profile)

Uses Nix’s built-in sampling profiler (--eval-profiler flamegraph, Nix 2.30+) to measure evaluation time and identify slow paths. Produces a collapsed-stack profile file compatible with flamegraph.pl, inferno-flamegraph, and speedscope.app.

# Basic profile (output: <host>.nix.profile)
hey check profile --host lab-matrix

# With frequency control and custom output
hey check profile --host lab-matrix --frequency 200 --output my.profile

# Open interactive flamegraph viewer (uses speedscope CLI, best experience)
hey check profile --host lab-matrix --open

# Auto-generate interactive SVG flamegraph
hey check profile --host lab-matrix --flamegraph

# Generate SVG and open in browser
hey check profile --host lab-matrix --flamegraph --open

# Open speedscope interactive viewer explicitly
hey check profile --host lab-matrix --speedscope

The profile file uses the collapsed-stack format (same as flamegraph.pl input) — each line is a semicolon-separated call chain. Two visualization options:

• speedscope (--speedscope / --open): Interactive web-based viewer. The nixpkgs speedscope CLI reads the profile directly (no JSON conversion needed) and opens a local interactive UI in the browser. Supports flamegraph, time-order, and sandwich views with search and zoom.
• Flamegraph SVG (--flamegraph): Interactive self-contained SVG file (click to zoom, hover for details, search). The inferno-flamegraph (Rust, fast) or flamegraph.pl (Perl) renderer converts the profile to an SVG you can open in any browser — no server needed.

All three packages (inferno, flamegraph, speedscope) are included in modules/dev/nix.nix so they’re available on any host with modules.dev.nix.enable = true.

📊 Information & Introspection

Commands for inspecting the state of the system, flake, or network.

Note: hey info closure evaluates the current state of your dotfiles repository, allowing you to preview the size of a configuration before you sync or deploy it.

🔍 Deep Dive: Under the Hood

To effectively use and extend the toolchain, it is important to understand how hey resolves your intent into action.

1. How hey <subcommand> Works (Janet Dispatch)

The hey binary (a Janet script) uses a declarative dispatcher to route commands.

• Subcommand Registry: When you run hey sync, the dispatcher looks for a registered keyword :sync.
• Dynamic Loading: It calls (cmd 'sync), which uses Janet’s import to load bin/hey.d/sync.janet on demand. This keeps the tool fast by only loading the code required for the current task.
• Type-Safe Options: Each subcommand uses the defcmd macro, which automatically parses flags (like --fast or --host) and validates their types before the script logic even runs.

2. How hey .<script> Works (The “Dot” Syntax)

The dot syntax is a shortcut for executing “binscripts”—standalone shell or Janet scripts located in the bin/ directory.

When hey sees a command starting with a ., it performs a scoped search using the following priority:

1. Host-Specific: hosts/$HOST/bin/ (Machine-specific overrides).
2. WM-Specific: config/$WM/bin/ (Compositor-aware behavior).
3. Global: bin/ (The project-wide implementation).

Example: Running hey .lock will first check if id3-eniac has a custom locking script, then check if there is a Niri-specific locking script, before falling back to the default bin/lock.zsh. This allows for incredible portability across different hardware and desktop environments.

3. How hey @<app> Works (Rofi & Orchestration)

The @ prefix is a specialized dispatcher for application-specific tools and interactive menus, primarily used for Rofi integration.

When hey sees a command starting with @, it searches for scripts in config/$APP/bin/.

Example: Running hey @rofi wifimenu

1. Resolution: hey maps @rofi to the directory $DOTFILES_HOME/config/rofi/bin/.
2. Execution: It searches for wifimenu within that directory, supporting multiple backends:
   • Janet (.janet): Used for complex logic (e.g., wifimenu.janet, audiomenu.janet). hey automatically calls the main function with arguments (provided the script uses defmain with upscope).
   • Zsh (.zsh / .sh): Used for shell-heavy tasks (e.g., powermenu.zsh, appmenu.sh).
3. UI Integration: These scripts typically use the hey/rofi library to render interactive menus that integrate with the system’s theme and window manager.

🚨 Developer Note: Clean Capture When writing Janet scripts for @ commands, be aware that the $<_ macro captures both stdout and stderr. System utilities often print warnings (like locale issues) to stderr, which will corrupt JSON parsing. Always use this pattern for machine-readable output:

($<_ sh -c "pactl -f json list sinks 2>/dev/null")

** Launcher Integration**: The exec strings defined in NixOS modules (like modules/desktop/apps/rofi.nix) leverage this syntax:

exec = "hey @rofi wifimenu"; # Calls the wifimenu janet script

4. How the Hook System Works

Hooks are triggered via hey hook <event>. This is the primary way the system reacts to environmental changes (e.g., hey hook battery).

Resolution Logic: The hook command (in bin/hey.d/hook.janet) searches for scripts named after the event. It doesn’t just run one script; it searches through a precedence hierarchy and executes the most specific one it finds:

1. hosts/$HOST/hooks/: Highest priority. Used for machine-specific reactions (e.g., unique power-saving steps for a laptop).
2. config/$WM/hooks/: Medium priority. Used for compositor-specific logic (e.g., sending a specific IPC command to Niri).
3. bin/hooks/: Global priority. Standardized logic used by the entire fleet.
4. ~/.local/share/hey/hooks.d/: Generated hooks. Used by the Nix module system to inject logic into hooks based on enabled modules (e.g., adding a specific service restart to the reload hook).

4. Integration with Zsh (lib/zsh/)

The toolchain provides “shell primitives” that scripts can use to remain environment-aware without being brittle.

• hey.wm.name: Returns the current WM string (fetched via XDG_CURRENT_DESKTOP).
• hey.wm.mode: Returns the desktop mode (e.g., dms or traditional).
• hey.vars: A lightweight key-value store (using JSON files in $XDG_STATE_HOME/hey/vars.json) that allows scripts to persist state across reboots or shared between different languages (Janet and Zsh).

5. The Build and Activation Lifecycle (hey build hey vs Nix package)

There are two independent ways hey gets onto your system. They produce separate, self-contained binaries — neither depends on or calls the other.

Production: Nix Package (packages/hey/)

The hey binary is built as a hermetic Nix derivation during nixos-rebuild. All Janet dependencies are pre-fetched with pinned revisions, compiled on the build host, and the result (binary + modules) is a nix store path copied to every host as part of the system closure.

This means hey is available immediately after deployment — no activation-time jpm deps or jpm install needed. This is the solution to the remote-VPS problem where slow C compilation (sqlite3) caused system.userActivationScripts.initHey to be killed by systemd before the binary was built.

The Nix derivation (packages/hey/default.nix):

1. Creates a temporary $JANET_TREE in the build sandbox
2. Fetches all Janet dependencies (spork, sh, posix-spawn, judge, cmd, sqlite3) at pinned git revisions with explicit SRI hashes
3. Compiles each dependency with jpm install into the sandbox tree
4. Symlinks lib/hey/ (the project’s own Janet modules) into the tree
5. Compiles bin/hey with jpm install --build-type=release --optimize=2
6. Installs the compiled binary to $out/bin/hey and all modules to $out/lib/

The nix store output structure:

/nix/store/<hash>-hey-0.1.0/
├── bin/
│   └── hey          # Compiled Janet image (native + jimage)
└── lib/
    ├── hey/         # init.janet, lib.janet, cmd.janet, sys.janet, ...
    ├── spork/       # path, json, rawterm, ...
    ├── cmd/         # arg-parser, param-parser, ...
    ├── judge/       # test runner modules
    ├── sh.janet     # shell DSL
    ├── posix-spawn.janet
    ├── sqlite3.so   # Native C module
    └── .manifests/  # jpm metadata

Development: scripts/build_hey.zsh

For local iteration, the standalone build script compiles hey directly into $JANET_TREE/bin/ (= ~/.local/share/janet/jpm_tree/bin/). It mirrors the Nix build logic but runs outside the sandbox, using the host’s Janet and GCC.

PATH precedence (set in modules/hey.nix via environment.extraInit):

1. $JANET_TREE/bin/hey                 ← local dev build (scripts/build_hey.zsh)
2. /run/current-system/sw/bin/hey      ← compiled system package
3. $DOTFILES_HOME/bin/hey              ← source-wrapper (fallback)

environment.extraInit is sourced by /etc/set-environment, which feeds both login shells and systemd user services (swayidle, niri’s spawn). $JANET_TREE/bin is prepended first; $DOTFILES_HOME/bin is appended last. The system hey at /run/current-system/sw/bin is already in the base PATH from the PAM environment — it sits naturally between the two.

This order gives:

• jpm_tree first: local dev changes take immediate effect after scripts/build_hey.zsh
• System package second: stable, hermetic, works on all hosts including remote VPS
• Source-wrapper last: fallback for helper scripts (.zsh, .janet) in bin/

When the jpm tree is deleted, the system package becomes the primary hey. The source-wrapper is always available as a last-resort fallback.

To rebuild after editing Janet source:

./scripts/build_hey.zsh       # or: hey build hey

Independence Guarantee

Each binary is a fully self-contained Janet image carrying its own compiled lib/hey/ modules. At runtime, each independently resolves its exec-path by scanning JANET_PATH entries (see Exec-Path Resolution). Neither binary shells out to the other — they are peers that happen to share the same source code at different compilation times.

Path Resolution Strategy (Dev vs System)

Both dev and system hey use the same compile-time-first, DOTFILES_HOME-fallback strategy for resolving source file paths. This ensures each binary finds files relative to its own build context without cross-contamination.

dispatch macro (lib/hey/init.janet):

(if (path/exists? compile-time-abs-path)   ; dev: working dir ✓
    compile-time-abs-path                   ; system: /build/hey_src/... ✗
    (path :home "bin/hey"))                 ; fallback: $DOTFILES_HOME/bin/hey

defcmd-1 macro (lib/hey/cmd.janet):

(fn []
  (if (path/exists? compile-time-abs-path)  ; e.g. ../dotfiles_dev/bin/hey.d/gc.janet
      compile-time-abs-path
      (path/join (or (os/getenv "DOTFILES_HOME") "/") "bin/hey.d/gc.janet")))

Resolution table:

Recursive hey invocation (hey/hey!/hey? macros):

(or (path/find "hey")          ; finds current binary via exec-path
    (path :bin "hey"))         ; fallback: $DOTFILES_HOME/bin/hey

path/find searches exec-path which includes the JANET_TREE bin (dev) or /run/current-system/sw/bin (system). This prevents the dev binary from accidentally spawning the nix store source script.

with-envvars (child process environment):

(with-envvars ["PATH" (string/join exec-path ":")
              "DOTFILES_HOME" (path :home)]
  ...)

Before dispatching any subcommand, dispatch-1 re-exports DOTFILES_HOME for child processes. The value is read from the current environment via (path :home) → (flake :path) → (os/getenv "DOTFILES_HOME").

Runtime Requirements

Common Pitfalls

DOTFILES_HOME stale across reboots: DOTFILES_HOME is set to the nix store source path by NixOS environment.variables. After hey sync, log out and back in to pick up the new value. Tmux/screen sessions preserve the old value.

$DOTFILES_HOME/bin/hey is the source script, not the compiled binary: The source at $DOTFILES_HOME/bin/hey is a #!/usr/bin/env janet script. On a fresh system (before any dev build), running this directly works but is slower than the compiled binary. The compiled system binary at /run/current-system/sw/bin/hey has higher PATH priority.

(path :bin "hey") is not the running binary: This resolves to $DOTFILES_HOME/bin/hey — the flake source script, NOT the compiled dev or system binary. Use (path/find "hey") to locate the actual running hey via exec-path.

XDG Path Compile-Time Caching Bug

Symptom: hey vars get caffeine, hey hook idle, hey info, hey @rofi appmenu or any hey subcommand that touches XDG_RUNTIME_DIR, XDG_DATA_HOME, or spawns child processes fails with stale build-sandbox paths when run from the compiled system binary (/run/current-system/sw/bin/hey).

Root cause: Janet’s jpm install creates a compiled ELF image by loading all modules into memory and snapshotting the environment. Module-level def, var, and delay expressions are evaluated at image creation time in the build sandbox. Three structures in lib/hey/lib.janet were affected:

Impact of each bug:

Fix pattern: Use (defn name [] ...) — a pure function with no mutable state. No var, no delay, no def for any value derived from os/getenv or filesystem reads. The function is compiled to bytecode but evaluated fresh at each call, using the runtime host environment.

Why the source-wrapper worked despite the bug: The source-wrapper (bin/hey, #!/usr/bin/env janet) loads lib/hey/ modules via JANET_PATH at runtime. Janet interprets .janet source files, so (delay ...) fibers are created but not forced until first access — which happens at runtime with correct env vars. Only the compiled ELF binary (/run/current-system/sw/bin/hey) had the compile-time values baked in.

Testing trap: The dev build (./scripts/build_hey.zsh) runs from source and will not reproduce these bugs — all os/getenv calls see the real host environment. Always verify the nix-built binary:

nix build .#hey && result/bin/hey info           # must return real data, not {}
nix build .#hey && result/bin/hey exec /tmp/test_path.sh  # must find rofi

Also in packages/hey/default.nix: --optimize=2 (not 3) to prevent aggressive constant folding of runtime values.

6. Exec-Path Resolution {#6-exec-path-resolution}

When hey spawns child processes (Zsh scripts, system commands, hooks), it constructs a PATH from the exec-path function defined in lib/hey/lib.janet. This function is called at runtime (not at module load time) and produces a valid search path for each invocation.

Historical note: exec-path was originally a def evaluated at module load time. In compiled executables (/run/current-system/sw/bin/hey), Janet evaluates def at image creation time during jpm install, capturing build-sandbox environment variables (USER=nix-build, temp dirs). This caused /etc/profiles/per-user/nix-build/bin to appear in PATH instead of the correct runtime user. It was converted to (defn exec-path [] ...) — a pure function called at runtime. See §XDG Path Compile-Time Caching Bug for the full pattern.

The Problem: (dyn :syspath) Is Unreliable

Janet’s (dyn :syspath) dynamic variable is set by the source loader during module resolution. Its default value is the last entry of the JANET_PATH environment variable. This has a critical consequence:

JANET_PATH = /nix/store/<hash>-hey-0.1.0/lib : ~/.local/share/janet/jpm_tree/lib
                                                        ↑
                                              (dyn :syspath) = this one (LAST)

On a host where no local build has been done, ~/.local/share/janet/jpm_tree/bin/ does not exist. If exec-path blindly trusts (dyn :syspath) and calls os/realpath on <syspath>/../bin, it crashes with ENOENT — even though the nix store bin/ is perfectly valid and available.

This is the exact failure mode that rendered hey unusable on VPS targets after hey ops deploy.

The Solution: Iterate JANET_PATH

Instead of trusting (dyn :syspath), exec-path iterates all JANET_PATH entries, computes ../bin for each, and returns the first one that exists as a directory:

(some |(let [b (path/join $ "../bin")]
        (when (path/directory? b) b))
      (string/split ":" (or (os/getenv "JANET_PATH") "")))

spork/path/join internally calls posix/normalize, which resolves . and .. components via a PEG parser — no filesystem call, no crash on missing paths. Only the final path/directory? check touches the filesystem, and it returns nil on missing directories instead of throwing.

Resolution Order

JANET_PATH is ordered by modules/hey.nix as:

${heyPackage}/lib          ← nix store (always exists, always has bin/)
${janetTreeDir}/lib        ← local jpm_tree (may or may not have bin/)

Since some returns the first match, the resolution is:

In all cases, each hey binary resolves the bin/ adjacent to its own lib/ in the filesystem — local hey → local bin/, system hey → system bin/.

Fallback Chain

After the JANET_PATH scan, exec-path appends additional fallback entries:

1. (path/xdg :bin) — $XDG_BIN_HOME (~/.local/bin)
2. hey/path file — If ~/.local/share/hey/path exists, its contents (colon-separated paths) are included. This allows per-host path customization.
3. System paths — /run/wrappers/bin, /etc/profiles/per-user/<user>/bin, /run/current-system/sw/bin — ensure core system tools are reachable even in minimal environments (systemd services, cron jobs).
4. $PATH — The process’s own environment PATH, as a final fallback.

The full exec-path is deduplicated via Janet’s distinct. At runtime, path/find searches exec-path for executables, falling back to the dynamic $PATH only if the static entries are exhausted.

7. Matugen Theme Pipeline Verification {#7-matugen-theme-pipeline-verification}

When DMS mode (modules.desktop.dms.enable = true) is active, modules/desktop/shell/matugen.nix generates ~/.config/matugen/config.toml with 11+ template entries. DMS calls matugen image internally on wallpaper changes and at startup. The hey reload --theme hook calls dms matugen generate.

Matugen 4.0.0 Breaking Changes

The nixpkgs matugen package jumped from 3.x to 4.0.0, introducing breaking changes:

Strict Verification Protocol

After any change to matugen templates (config/matugen/templates/*), matugen Nix module (modules/desktop/shell/matugen.nix), or updating the matugen nixpkgs version:

1. Scan for filters: grep -r "set_alpha\|format:\|to_color" config/matugen/templates/ — Only hex_alpha | set_alpha: N and hex lookups are safe. Never use | set_alpha: X | format: "rgba" on a hex color without .hex_alpha or | to_color.
2. Build test config: Create a temporary config.toml with ALL templates pointing to the source files (not nix store paths).
3. Run full generation:

   matugen image <any-image> --config /tmp/test-config.toml --source-color-index 0

   Exit code must be 0. No panics.
4. Verify outputs: Every template must produce a non-empty output file. Zero {{...}} placeholders must leak through (fully rendered).
5. Check rofi: grep "hex_alpha\|rgba\|hsla" on rofi output — must use #RRGGBBAA format (8-digit hex), not rgba() CSS function (which rofi 2.x parses differently from GTK CSS).
6. Restart DMS: After hey sync, systemctl restart --user dms. Check logs:

   journalctl --user -u dms --since "1 min ago" | grep -i "matugen\|theme"

   Must show “Matugen worker completed successfully”, no “Matugen failed” or panics.
7. Test rofi menus: hey @rofi appmenu must launch without theme errors.

Template Filter Reference (matugen 4.0.0)

Rule of thumb: Use .hex for opaque colors, .hex_alpha | set_alpha: N for transparency. Avoid the | format: "rgba" pipe unless you start from a string literal with | to_color first.

🛠️ Workflows

1. The “Safe Refactor”

When modifying a shared module or library:

1. Modify the code.
2. Verify: just check-all (Evaluates every host in the fleet to catch recursion).
3. Apply: just sync.

4. The “Matugen Template Change”

When editing files in config/matugen/templates/ or modules/desktop/shell/matugen.nix:

1. Modify the template or Nix module.
2. Verify: Run the full matugen generation test (see §7).
3. Sync: hey sync to rebuild the nix store template paths.
4. Restart DMS: systemctl restart --user dms.
5. Check logs: Confirm “Theme generation completed” with no panics.
6. Test rofi: hey @rofi appmenu must launch without “Failed to open theme” errors.

2. Provisioning a New Host

1. Boot from a NixOS installer ISO.
2. just disko format --host <name> (Prepare disks).
3. just install --host <name> (Deploy system).

3. Testing in a VM

To test a configuration without touching your hardware:

• hey build vm --host <name>
• This generates a script to launch the configuration in QEMU.