Ai Ml

AI & Machine Learning Configuration

This repository provides first-class support for AI/ML development via local LLMs and hardware acceleration.

Service Architecture

Modules: modules/services/ai/ollama.nix + modules/services/ai/open-webui.nix

Open-WebUI was extracted from ollama.nix into its own standalone module. This allows Open-WebUI to run on any host (with or without a local Ollama backend) by setting modules.services.ai.open-webui.ollamaUrl.

flake.nix                    Nixpkgs (ollama-cuda / ollama-rocm / ollama-cpu)
    │                              │
    ▼                              ▼
host configuration        modules.profiles.hardware
    │                     ["gpu/nvidia/pascal", ...]
    ▼                              │
ollama.nix ─── auto-detects GPU ──┘
    │
    ├─ ollamaPackage  = ollama-cuda  (NVIDIA, CUDA arch from profile)
    │                 │ ollama-rocm  (AMD)
    │                 │ ollama-cpu   (fallback)
    │
    ├─ services.ollama  host=127.0.0.1, port=11434
    │   loadModels = [declared model list]   ← auto-pulled on boot
    │
    └─ persistence      ~/.ollama models survive impermanence wipes

open-webui.nix (standalone — can connect to remote Ollama)
    │
    ├─ services.open-webui  host=127.0.0.1, port=8080
    │   ollamaUrl = "http://127.0.0.1:11434"      ← configurable
    │   environmentFile = /run/agenix/open-webui-env  ← admin credentials
    │
    ├─ nginx vhost  ollama.alienzj.org
    └─ oauth2 SSO  (optional, oauth2.enable)

Open-WebUI Admin User

Open-WebUI stores user accounts in a SQLite database at /var/lib/open-webui/data/webui.db. The admin account is created from environment variables only on first startup when the database is empty:

These are stored in the agenix secret open-webui-env.age and loaded via systemd EnvironmentFile.

Critical behavior: Once webui.db exists — even with zero users — the env vars are ignored. This creates a deadlock if:

1. Open-WebUI starts without the admin env vars → DB created, no admin
2. ENABLE_SIGNUP = "false" blocks web registration
3. Admin env vars are added later → ignored because DB already exists

Recovery: Delete the database and restart:

systemctl stop open-webui
rm /var/lib/open-webui/data/webui.db
systemctl start open-webui
# Admin is recreated from env vars on fresh start

Verification: Check logs for admin creation:

journalctl -u open-webui --no-pager | grep -i 'admin account'
# Expected: "Admin account created successfully: <email>"

Test login directly on localhost (bypasses nginx SSO gate):

curl -s http://127.0.0.1:8080/api/v1/auths/signin \
  -H "Content-Type: application/json" \
  -d '{"email":"[email protected]","password":"<password>"}'
# Returns JSON with "token" on success, 401 on failure

GPU detection (ollama.nix:20-32): reads modules.profiles.hardware from the host config, maps GPU generation to CUDA architecture (Kepler=30, Pascal=61, Turing=75, Ampere=86, Lovelace=89), and overrides ollama-cuda with the correct cudaArches.

Model Catalog

Declared in modules.services.ai.ollama.models and auto-pulled via services.ollama.loadModels. Additional models can be pulled manually with ollama pull <name>.

Declarative (Nix-managed)

Manual (user-pulled)

These are present on the system but not managed by Nix (pulled via ollama pull):

Usage

# List installed models with sizes
$ ollama list
NAME                   ID              SIZE      MODIFIED
translategemma:4b      c49d986b0764    3.3 GB    4 hours ago
gemma4:e2b             7fbdbf8f5e45    7.2 GB    4 hours ago
llama3.2:3b            a80c4f17acd5    2.0 GB    4 hours ago
...

# Pull a model not in Nix config
$ ollama pull qwen3:14b

# Remove an unused model
$ ollama rm gemma4:latest

Terminal AI Tools (hey .<command>)

All tools run against local Ollama models — no network, no telemetry, private.

hey .translate — Translation

# English → Chinese (default)
$ hey .translate "The quick brown fox jumps over the lazy dog"
那只敏捷的棕色狐狸跳过了那只懒狗

# Chinese → English (--reverse / -r)
$ hey .translate -r "今天天气真好，适合出去走走"
The weather is really nice today, perfect for going out for a walk

# Explicit source/target
$ hey .translate -s Japanese -t English "こんにちは"
Hello

# Pipe from stdin
$ echo "请帮我翻译这段话" | hey .translate -r
Please help me translate this passage

# Pipe from clipboard
$ wl-paste | hey .translate -r

Rofi integration: Mod + Alt + T opens an interactive translation menu.

hey .commit-gen — Commit Messages

$ git add -A
$ hey .commit-gen
refactor(idle): add swayidle resume command to prevent unlock relock

# Sarcastic version
$ hey .commit-gen --funny
fix: finally stopped the infinite lock loop, you're welcome future me

# Copy to clipboard automatically
$ hey .commit-gen --copy

Uses staged diff if available, otherwise unstaged diff. Limited to 100 lines to avoid overwhelming the model.

hey .fortune — Fortune / Tip

$ hey .fortune
 _______________________________________
< A good programmer is someone who looks
< both ways before crossing a one-way
< street. A great one just rewrites the
< street.
 ---------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

$ hey .fortune --tip
 _______________________________________
< Use `nix flake metadata` to inspect
< locked inputs without evaluating the
< flake.
 ---------------------------------------

Renders with cowsay + lolcat if installed, otherwise plain colored text.

hey .roast — System Roast

$ hey .roast
🔥 Preparing the roast...

NixOS with 47 open windows? You've got more floating terminals
than functioning brain cells — and that load average suggests
your CPU is about to unionize and demand better working conditions.

Sends the roast as a desktop notification in addition to terminal output.