1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
|
# AGENTS.md
Guidance for AI coding agents working in this repository.
## What this is
`strata-docker`: a bash wrapper (`strata-docker`) that builds and runs the
Intel Arc (SYCL) engine of [Strata](https://github.com/Niko1221/Strata) on a
Slackware host, using upstream's own Docker route. Target: Arc B580 (12 GB,
`xe` driver), model Gyro-M. Host needs only the `xe` driver, Docker and
Python 3.10+ with venv. No oneAPI, Level Zero or compute runtime on the host:
those live in the `strata-sycl-dev` image. Keep it a thin wrapper, do not
reimplement anything upstream's scripts already do.
## How upstream's Intel route works
- There is no prebuilt Intel engine in Strata's releases. The engine is built
from source in the `strata-sycl-dev` image (`Strata/sycl/tools/Dockerfile`,
based on a community llama.cpp SYCL image, about 13 GB).
- `setup.sh --backend sycl` runs `setup.py` in a `.venv` on the host, which
hands over to `sycl/setup_intel.py`. That writes the config and
`run-<model>.sh`. The server (`sycl/serve/server_intel.py`) runs on the
host. Only the engine runs in Docker, through `sycl/serve/strata-sycl.sh`.
- `strata-sycl.sh` mounts the checkout's parent directory at `/work`. Hence
the layout: `Strata/` (checkout) and `Strata-data/` (models, about 92 GB for
Gyro-M) both inside this repo, gitignored. Symlinks pointing outside the
mount do not work in the container.
- B580 is PCI id `e20b`, AOT target `bmg-g21`. `setup_intel.py` knows it.
- Gyro is the one family whose experts are not streamed into VRAM: it needs
35 GB of RAM and VRAM together (Gyro-M). The host has about 30 GiB RAM and a
12 GB card, so close other programs before starting it.
- Gyro on Arc was tested upstream only on an A750 (`i915`). The B580 is
untested natively; one WSL2 B580 report saw device loss.
- Measured here (v0.2.2, B580, 31 GiB RAM): on `xe` the engine runs with
`--stream-experts` and `STRATA_VERIFY_NO_HOST`, so every expert must sit in
the VRAM cache or a pinned RAM mirror (capped at MemAvailable minus 4 GiB).
Gyro-M does not fit: 4.6 GiB VRAM cache + 21.6 GiB mirror left 5,309
experts out and the engine REFUSED. setup's "35 GB of VRAM + RAM: fits"
check is wrong for this path. Gyro-S fits: 3,062 experts in VRAM, 21,514
mirrored (21 GiB), about 6 GiB of RAM left, decode 22-29 tok/s.
- Gyro-S is not stable here: 2 of about 6 requests hit
`UR_RESULT_ERROR_DEVICE_LOST` in the verify (speculative decoding) step,
the engine segfaulted (139) and the chat showed "verify error". The server
restarts the engine and the card recovered each time.
- Likely cause, not verified: device spins on host flags are capped at
20,000 reads on B-series cards (`sycl/include/strata/sycl_doorbell.hpp`),
tuned on a B70 that holds every expert in VRAM. Here 21 GiB of experts
come from the RAM mirror over PCIe, so a wait can outrun the cap. Upstream
documents the same 139 crash pattern for a cold first request
(`docs/INTEL.md`, `STRATA_SYCL_SPIN_MAX`).
- Untried next steps: (1) `"env": {"STRATA_SPIN_MAX": "2000000"}` in the
config, read at engine start. Risk: on `xe` an overlong spin can wedge the
GPU until reboot, and the B580 drives the desktop. (2) Drop `--spec`,
`--spec-min-p` and `--mtp` from the config to avoid the draft verify, at
lower decode speed.
- Status 2026-10-22: testing stopped. Gyro-M deleted. Gyro-S (55 GiB) and
the shared draft layer remain in `Strata-data/`.
- On `xe`, asking for more VRAM than the card has does not fail: buffers
spill into RAM and the host can livelock. Never raise `--context` or lower
`--vram-reserve-mib` past what setup's fit rule picked.
- Upstream bug (v0.2.2): `setup_intel.py` tells `setup.py` the host has
1024 GB of RAM (`S.ram_gb = lambda: fake_ram`), so on the `xe` path
`setup.py` adds `--ple-io ram` to the Gyro config. That loads the 54 GB
n-gram table into RAM and would OOM this host. The Intel code itself says
Gyro's table must stay on the SSD (it only guards the `i915` branch). After
every `setup` run, check `Strata/strata-gyro-gyro-m.json` and drop
`--ple-io ram` from `args`. Start the model with
`Strata/run-gyro-gyro-m.sh`, which does not rewrite the config.
- Pictures are off by default on Battlemage (they reset a B70). Do not pass
`--vision yes`.
- `docker build --pull` matters: a stale base can carry an ocloc that crashes
on Battlemage kernels (upstream #3219). `build.sh` checks for it.
## Conventions
- Pin the upstream tag in `ref` (`STRATA_REF` overrides). Bump it
deliberately, then rerun `image` (if `sycl/tools/Dockerfile` changed) and
`build`.
- Placeholders only in code, docs, tests and commits.
## Testing
No test suite. Check by hand: `bash -n strata-docker`, then `./strata-docker
build` must end with `BUILD EXIT 0` and `errors: 0`, and `./strata-docker
setup --check` lists what fits.
## Repo facts
- License: GPLv2 only (`LICENSE`, header in each source file).
- Work directly on master.
|
