diff options
Diffstat (limited to 'AGENTS.md')
| -rw-r--r-- | AGENTS.md | 58 |
1 files changed, 58 insertions, 0 deletions
diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..caddcd5 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,58 @@ +# AGENTS.md + +Guidance for AI coding agents working in this repository. + +## What this is + +`strata-docker`: a bash wrapper (`strata-docker`) that builds and runs the +Intel Arc (SYCL) engine of [Strata](https://github.com/Niko1221/Strata) on a +Slackware host, using upstream's own Docker route. Target: Arc B580 (12 GB, +`xe` driver), model Gyro-M. Host needs only the `xe` driver, Docker and +Python 3.10+ with venv. No oneAPI, Level Zero or compute runtime on the host: +those live in the `strata-sycl-dev` image. Keep it a thin wrapper, do not +reimplement anything upstream's scripts already do. + +## How upstream's Intel route works + +- There is no prebuilt Intel engine in Strata's releases. The engine is built + from source in the `strata-sycl-dev` image (`Strata/sycl/tools/Dockerfile`, + based on a community llama.cpp SYCL image, about 13 GB). +- `setup.sh --backend sycl` runs `setup.py` in a `.venv` on the host, which + hands over to `sycl/setup_intel.py`. That writes the config and + `run-<model>.sh`. The server (`sycl/serve/server_intel.py`) runs on the + host. Only the engine runs in Docker, through `sycl/serve/strata-sycl.sh`. +- `strata-sycl.sh` mounts the checkout's parent directory at `/work`. Hence + the layout: `Strata/` (checkout) and `Strata-data/` (models, about 92 GB for + Gyro-M) both inside this repo, gitignored. Symlinks pointing outside the + mount do not work in the container. +- B580 is PCI id `e20b`, AOT target `bmg-g21`. `setup_intel.py` knows it. +- Gyro is the one family whose experts are not streamed into VRAM: it needs + 35 GB of RAM and VRAM together (Gyro-M). The host has about 30 GiB RAM and a + 12 GB card, so close other programs before starting it. +- Gyro on Arc was tested upstream only on an A750 (`i915`). The B580 is + untested natively; one WSL2 B580 report saw device loss. +- On `xe`, asking for more VRAM than the card has does not fail: buffers + spill into RAM and the host can livelock. Never raise `--context` or lower + `--vram-reserve-mib` past what setup's fit rule picked. +- Pictures are off by default on Battlemage (they reset a B70). Do not pass + `--vision yes`. +- `docker build --pull` matters: a stale base can carry an ocloc that crashes + on Battlemage kernels (upstream #3219). `build.sh` checks for it. + +## Conventions + +- Pin the upstream tag in `ref` (`STRATA_REF` overrides). Bump it + deliberately, then rerun `image` (if `sycl/tools/Dockerfile` changed) and + `build`. +- Placeholders only in code, docs, tests and commits. + +## Testing + +No test suite. Check by hand: `bash -n strata-docker`, then `./strata-docker +build` must end with `BUILD EXIT 0` and `errors: 0`, and `./strata-docker +setup --check` lists what fits. + +## Repo facts + +- License: GPLv2 only (`LICENSE`, header in each source file). +- Work directly on master. |
