diff options
Diffstat (limited to 'docs/superpowers')
| -rw-r--r-- | docs/superpowers/plans/2026-10-28-generat3d-v1.md | 2057 |
1 files changed, 2057 insertions, 0 deletions
diff --git a/docs/superpowers/plans/2026-10-28-generat3d-v1.md b/docs/superpowers/plans/2026-10-28-generat3d-v1.md new file mode 100644 index 0000000..31663f8 --- /dev/null +++ b/docs/superpowers/plans/2026-10-28-generat3d-v1.md @@ -0,0 +1,2057 @@ +# generat3D v1 Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** A Python CLI that reads a Genesis 9 `.duf` scene plus a Daz content library and writes the `.dbz` file Diffeomorphic's "DBZ File" mesh fitting needs, with no Daz Studio involved. + +**Architecture:** `library` resolves DSON URLs to files across library roots (case-insensitive, like Windows). `scene` loads the `.duf`, the figure `.dsf` and every morph `.dsf` into a figure plus a property graph. `formula` evaluates the graph (stack-machine formulas, sum and mult stages, clamps). `evaluate` turns final values into morphed vertices and bone rest data. `dbz` writes the result. Correctness is proven against `.dbz` files Daz itself exported in a throwaway Windows VM. + +**Tech Stack:** Python >= 3.11 (stdlib `tomllib`, `unittest`), numpy. libvirt/QEMU (session mode) and mtools for the reference VM. + +Spec: `docs/superpowers/specs/2026-10-28-generat3d-v1-design.md`. + +--- + +## Conventions for every task + +- Work on `master`. Run all commands from the repo root. +- Every `.py` file starts with this header, then a blank line, then code: + +```python +# generat3D - evaluate Daz DSON scenes and write .dbz files +# Copyright (C) 2026 Danilo M. <danix@danix.xyz> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program; if not, write to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +``` + + Code blocks below omit the header to save space; add it when creating the file. JSON fixtures get no header. +- Commits are GPG-signed (global config). Never disable signing. Every commit message ends with the trailer line `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>` (shown as a second `-m`). +- Never commit Daz content: no `.dbz` from Daz, no `.dsf` from a product, nothing under `corpus/`. Run `git status --short` before each commit. +- No em dashes in code, comments, docs or commits. + +## File map + +| file | responsibility | +|---|---| +| `generat3d/__init__.py` | `Error`, `warn`, `__version__` | +| `generat3d/config.py` | library roots from `-L` or `config.toml` | +| `generat3d/library.py` | DSON URL parsing, file lookup, DSON reading | +| `generat3d/formula.py` | property graph, formula stack machine, splines | +| `generat3d/scene.py` | `.duf` + `.dsf` loading into `Scene` (figure, nodes, morphs, graph) | +| `generat3d/evaluate.py` | final values to vertices and rest data | +| `generat3d/dbz.py` | `.dbz` document build and atomic write | +| `generat3d/__main__.py` | CLI | +| `tests/test_*.py` | unittest suites | +| `tests/data/lib/...` | hand-written synthetic content library (no Daz content) | +| `tools/export_corpus.dsa` | batch export inside Daz Studio | +| `fixtures/scenes/*.duf` | corpus scene recipes authored in Daz | + +--- + +### Task 1: Package skeleton and config + +**Files:** +- Create: `generat3d/__init__.py` +- Create: `generat3d/config.py` +- Create: `tests/__init__.py` (empty, no header needed) +- Test: `tests/test_config.py` + +- [ ] **Step 1: Create the package base** + +`generat3d/__init__.py`: + +```python +import sys + +__version__ = "0.1.0" + + +class Error(Exception): + """Fatal problem. The CLI prints it and exits 1.""" + + +def warn(msg): + print(f"generat3d: warning: {msg}", file=sys.stderr) +``` + +Create empty `tests/__init__.py`. + +- [ ] **Step 2: Write the failing tests** + +`tests/test_config.py`: + +```python +import os +import tempfile +import unittest +from pathlib import Path + +from generat3d import Error +from generat3d.config import config_path, library_roots + + +class ConfigTest(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + self.dir = Path(self.tmp.name) + self.lib = self.dir / "lib" + self.lib.mkdir() + self.cfg = self.dir / "config.toml" + + def tearDown(self): + self.tmp.cleanup() + + def write(self, text): + self.cfg.write_text(text) + + def test_cli_roots_win_over_config(self): + self.write('library_roots = ["/nonexistent"]\n') + self.assertEqual(library_roots([str(self.lib)], self.cfg), [self.lib]) + + def test_roots_from_config(self): + self.write(f'library_roots = ["{self.lib}"]\n') + self.assertEqual(library_roots([], self.cfg), [self.lib]) + + def test_no_roots_anywhere(self): + with self.assertRaisesRegex(Error, "no library roots"): + library_roots([], self.cfg) + + def test_missing_root_is_fatal(self): + with self.assertRaisesRegex(Error, "library root not found"): + library_roots([str(self.dir / "nope")], self.cfg) + + def test_unknown_key_is_fatal(self): + self.write(f'library_roots = ["{self.lib}"]\nlibary = 1\n') + with self.assertRaisesRegex(Error, "unknown keys: libary"): + library_roots([], self.cfg) + + def test_bad_type_is_fatal(self): + self.write('library_roots = "/x"\n') + with self.assertRaisesRegex(Error, "list of strings"): + library_roots([], self.cfg) + + def test_bad_toml_is_fatal(self): + self.write("library_roots = [\n") + with self.assertRaisesRegex(Error, "config.toml"): + library_roots([], self.cfg) + + def test_xdg_config_home(self): + old = os.environ.get("XDG_CONFIG_HOME") + os.environ["XDG_CONFIG_HOME"] = "/tmp/xdg" + try: + self.assertEqual(config_path(), Path("/tmp/xdg/generat3d/config.toml")) + finally: + if old is None: + del os.environ["XDG_CONFIG_HOME"] + else: + os.environ["XDG_CONFIG_HOME"] = old +``` + +- [ ] **Step 3: Run tests to verify they fail** + +Run: `python -m unittest tests.test_config -v` +Expected: ERROR, `ModuleNotFoundError: No module named 'generat3d.config'` + +- [ ] **Step 4: Implement `generat3d/config.py`** + +```python +import os +import tomllib +from pathlib import Path + +from . import Error + +KEYS = {"library_roots"} + + +def config_path(): + base = os.environ.get("XDG_CONFIG_HOME") or os.path.expanduser("~/.config") + return Path(base) / "generat3d" / "config.toml" + + +def library_roots(cli_roots, path=None): + """Library roots from -L if given (replacing the config), else from the config file.""" + path = Path(path) if path else config_path() + roots = list(cli_roots) + if not roots and path.exists(): + try: + with open(path, "rb") as f: + data = tomllib.load(f) + except tomllib.TOMLDecodeError as e: + raise Error(f"{path}: {e}") from None + unknown = sorted(set(data) - KEYS) + if unknown: + raise Error(f"{path}: unknown keys: {', '.join(unknown)}") + roots = data.get("library_roots", []) + if not isinstance(roots, list) or not all(isinstance(r, str) for r in roots): + raise Error(f"{path}: library_roots must be a list of strings") + if not roots: + raise Error(f"no library roots: pass -L or set library_roots in {path}") + out = [] + for r in roots: + p = Path(r).expanduser() + if not p.is_dir(): + raise Error(f"library root not found: {p}") + out.append(p) + return out +``` + +- [ ] **Step 5: Run tests to verify they pass** + +Run: `python -m unittest tests.test_config -v` +Expected: 8 tests, OK + +- [ ] **Step 6: Commit** + +```bash +git add generat3d/__init__.py generat3d/config.py tests/__init__.py tests/test_config.py +git commit -m "Add package base and config loading" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" +``` + +--- + +### Task 2: Library: URL parsing, lookup, DSON reading + +**Files:** +- Create: `generat3d/library.py` +- Test: `tests/test_library.py` + +Facts: DSON URLs look like `node:/data/Daz%203D/x.dsf#id?prop`. The `node:` part is optional, `#id` may stand alone for a local reference. Paths are percent-encoded. Daz runs on case-insensitive filesystems, so content files often differ in case from the URLs that reference them. Files are plain or gzipped JSON. + +- [ ] **Step 1: Write the failing tests** + +`tests/test_library.py`: + +```python +import gzip +import json +import tempfile +import unittest +from pathlib import Path + +from generat3d import Error +from generat3d.library import Library, parse_url, read_dson + + +class ParseUrlTest(unittest.TestCase): + def test_full(self): + self.assertEqual( + parse_url("lShldrBend:/data/Daz%203D/G9.dsf#lShldrBend?center_point/x"), + ("lShldrBend", "/data/Daz 3D/G9.dsf", "lShldrBend", "center_point/x")) + + def test_path_and_id(self): + self.assertEqual(parse_url("/data/a%20b.dsf#m1"), ("", "/data/a b.dsf", "m1", "")) + + def test_local_ref(self): + self.assertEqual(parse_url("#hip"), ("", "", "hip", "")) + + def test_node_with_local_ref(self): + self.assertEqual(parse_url("Fig:#Tall?value"), ("Fig", "", "Tall", "value")) + + +class LibraryTest(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + base = Path(self.tmp.name) + self.r1, self.r2 = base / "r1", base / "r2" + (self.r1 / "data/Vendor/Fig/Morphs").mkdir(parents=True) + (self.r2 / "data/vendor/fig").mkdir(parents=True) + (self.r1 / "data/Vendor/Fig/Morphs/A.dsf").write_text('{"a": 1}') + (self.r2 / "data/vendor/fig/B.dsf").write_bytes(gzip.compress(b'{"b": 2}')) + (self.r2 / "data/vendor/fig/Bad.dsf").write_text("{nope") + self.lib = Library([self.r1, self.r2]) + + def tearDown(self): + self.tmp.cleanup() + + def test_exact_match(self): + self.assertEqual(self.lib.find("/data/Vendor/Fig/Morphs/A.dsf"), + self.r1 / "data/Vendor/Fig/Morphs/A.dsf") + + def test_case_insensitive_second_root(self): + self.assertEqual(self.lib.find("/data/Vendor/Fig/B.dsf"), + self.r2 / "data/vendor/fig/B.dsf") + + def test_missing(self): + self.assertIsNone(self.lib.find("/data/Nope.dsf")) + + def test_load_gzip_and_cache(self): + doc = self.lib.load("/data/Vendor/Fig/B.dsf") + self.assertEqual(doc, {"b": 2}) + self.assertIs(self.lib.load("/data/vendor/fig/b.dsf"), doc) + + def test_load_missing_is_fatal(self): + with self.assertRaisesRegex(Error, "not found in library roots: /data/Nope.dsf"): + self.lib.load("/data/Nope.dsf") + + def test_broken_json_names_file(self): + with self.assertRaisesRegex(Error, "Bad.dsf"): + self.lib.load("/data/Vendor/Fig/Bad.dsf") + + def test_walk_merges_roots(self): + self.assertEqual(sorted(self.lib.walk("/data/Vendor/Fig")), + ["/data/Vendor/Fig/Morphs/A.dsf", + "/data/vendor/fig/B.dsf", "/data/vendor/fig/Bad.dsf"]) + + def test_read_dson_plain(self): + p = Path(self.tmp.name) / "x.duf" + p.write_text(json.dumps({"x": 1})) + self.assertEqual(read_dson(p), {"x": 1}) +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `python -m unittest tests.test_library -v` +Expected: ERROR, `ModuleNotFoundError: No module named 'generat3d.library'` + +- [ ] **Step 3: Implement `generat3d/library.py`** + +```python +import gzip +import json +import os +from pathlib import Path +from urllib.parse import unquote + +from . import Error + + +def parse_url(url): + """Split a DSON URL "node:/path/file.dsf#id?prop" into (node, path, id, prop).""" + node = "" + if not url.startswith(("/", "#")) and ":" in url: + node, url = url.split(":", 1) + url, _, prop = url.partition("?") + path, _, ident = url.partition("#") + return unquote(node), unquote(path), unquote(ident), prop + + +def read_dson(path): + """Parse a plain or gzipped DSON file.""" + try: + data = Path(path).read_bytes() + if data[:2] == b"\x1f\x8b": + data = gzip.decompress(data) + return json.loads(data) + except (OSError, EOFError, ValueError) as e: + raise Error(f"{path}: {e}") from None + + +def _find_nocase(base, parts, want_dir=False): + for part in parts: + try: + names = {n.lower(): n for n in os.listdir(base)} + except OSError: + return None + name = names.get(part.lower()) + if name is None: + return None + base = base / name + return base if (base.is_dir() if want_dir else base.is_file()) else None + + +class Library: + def __init__(self, roots): + self.roots = [Path(r) for r in roots] + self._docs = {} + + def _locate(self, rel, want_dir): + for root in self.roots: + p = root / rel + if p.is_dir() if want_dir else p.is_file(): + yield root, p + continue + p = _find_nocase(root, rel.split("/"), want_dir) + if p: + yield root, p + + def find(self, path): + """File for a library path like "/data/x.dsf", first root wins, or None.""" + return next((p for _, p in self._locate(path.strip("/"), False)), None) + + def load(self, path): + """Parsed DSON document for a library path. Error if missing or broken.""" + key = path.lower() + if key not in self._docs: + f = self.find(path) + if f is None: + raise Error(f"not found in library roots: {path}") + self._docs[key] = read_dson(f) + return self._docs[key] + + def walk(self, path): + """Library paths of every .dsf under a library directory, across all roots.""" + seen = {} + for root, base in self._locate(path.strip("/"), True): + for f in sorted(base.rglob("*")): + if f.is_file() and f.suffix.lower() == ".dsf": + lp = "/" + f.relative_to(root).as_posix() + seen.setdefault(lp.lower(), lp) + return list(seen.values()) +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `python -m unittest tests.test_library -v` +Expected: 12 tests, OK + +- [ ] **Step 5: Commit** + +```bash +git add generat3d/library.py tests/test_library.py +git commit -m "Add DSON URL parsing and library lookup" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" +``` + +--- + +### Task 3: Formula graph + +**Files:** +- Create: `generat3d/formula.py` +- Test: `tests/test_formula.py` + +Facts (DSON spec, Diffeomorphic `formula.py`): a formula has `output` (a property URL), optional `stage` (`sum`, the default, or `mult`) and `operations`, a stack program. Ops: `push` with `val` or `url`, `add`, `sub`, `mult`, `div`, `neg`, and the splines (Task 4). A property's final value is `(base + sum of sum-stage results) * product of mult-stage results`, then clamped to `[min, max]` if the property is clamped. `formula.py` never sees URLs: `scene.py` (Task 5) rewrites `push url` into `push key`. A formula whose input key is unknown is dropped, as Daz drops formulas whose inputs come from products that are not installed. + +Unverified until the corpus (Task 11): clamping applies to the final value, not only to the user's dial. + +- [ ] **Step 1: Write the failing tests** + +`tests/test_formula.py`: + +```python +import unittest + +from generat3d import Error +from generat3d.formula import Graph + + +def push(key): + return {"op": "push", "key": key} + + +def val(v): + return {"op": "push", "val": v} + + +class GraphTest(unittest.TestCase): + def setUp(self): + self.g = Graph() + self.g.add_prop("a", 0.5) + self.g.add_prop("b") + self.g.add_prop("c", 0.0, 0.0, 1.0, clamped=True) + + def test_base_values(self): + self.assertEqual(self.g.evaluate(), {"a": 0.5, "b": 0.0, "c": 0.0}) + + def test_sum_stage(self): + self.g.add_formula("b", [push("a"), val(2), {"op": "mult"}]) + self.g.add_formula("b", [push("a"), val(1), {"op": "add"}]) + self.assertEqual(self.g.evaluate()["b"], 1.0 + 1.5) + + def test_mult_stage(self): + self.g.set_value("b", 3.0) + self.g.add_formula("b", [push("a")], stage="mult") + self.assertEqual(self.g.evaluate()["b"], 1.5) + + def test_chain_and_clamp(self): + self.g.add_formula("b", [push("a"), val(4), {"op": "mult"}]) + self.g.add_formula("c", [push("b")]) + out = self.g.evaluate() + self.assertEqual(out["b"], 2.0) + self.assertEqual(out["c"], 1.0) + + def test_sub_div_neg(self): + self.g.add_formula("b", [val(1), push("a"), {"op": "sub"}, val(4), {"op": "div"}, + {"op": "neg"}]) + self.assertEqual(self.g.evaluate()["b"], -0.125) + + def test_unknown_input_drops_formula(self): + self.g.add_formula("b", [push("missing")]) + self.g.add_formula("b", [val(7)]) + self.assertEqual(self.g.evaluate()["b"], 7) + + def test_output_without_prop_is_ignored(self): + self.g.add_formula("nowhere", [val(1)]) + self.assertNotIn("nowhere", self.g.evaluate()) + + def test_cycle_is_fatal(self): + self.g.add_formula("a", [push("b")]) + self.g.add_formula("b", [push("a")]) + with self.assertRaisesRegex(Error, "formula cycle: a -> b -> a"): + self.g.evaluate() + + def test_unknown_op_is_fatal(self): + self.g.add_formula("b", [val(1), {"op": "pow"}], source="/data/x.dsf#m") + with self.assertRaisesRegex(Error, r"/data/x.dsf#m: unknown formula operation 'pow'"): + self.g.evaluate() + + def test_unknown_stage_is_fatal(self): + with self.assertRaisesRegex(Error, "unknown formula stage 'avg'"): + self.g.add_formula("b", [val(1)], stage="avg") + + def test_stack_errors_are_fatal(self): + self.g.add_formula("b", [{"op": "add"}], source="f") + with self.assertRaisesRegex(Error, "f: malformed formula for b"): + self.g.evaluate() + + def test_leftover_stack_is_fatal(self): + self.g.add_formula("b", [val(1), val(2)], source="f") + with self.assertRaisesRegex(Error, "f: malformed formula for b"): + self.g.evaluate() +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `python -m unittest tests.test_formula -v` +Expected: ERROR, `ModuleNotFoundError: No module named 'generat3d.formula'` + +- [ ] **Step 3: Implement `generat3d/formula.py`** + +```python +import math +import operator + +from . import Error + +STAGES = ("sum", "mult") +BINARY = {"add": operator.add, "sub": operator.sub, + "mult": operator.mul, "div": operator.truediv} +SPLINES = () # filled in by Task 4 + + +class Prop: + __slots__ = ("value", "lo", "hi", "clamped") + + def __init__(self, value, lo, hi, clamped): + self.value, self.lo, self.hi, self.clamped = value, lo, hi, clamped + + +class Graph: + """Properties keyed by string, plus the formulas that drive them.""" + + def __init__(self): + self.props = {} + self.formulas = {} # output key -> [(stage, ops, source)] + + def add_prop(self, key, value=0.0, lo=-math.inf, hi=math.inf, clamped=False): + self.props[key] = Prop(value, lo, hi, clamped) + + def set_value(self, key, value): + self.props[key].value = value + + def add_formula(self, output, ops, stage="sum", source="?"): + if stage not in STAGES: + raise Error(f"{source}: unknown formula stage {stage!r} for {output}") + self.formulas.setdefault(output, []).append((stage, ops, source)) + + def evaluate(self): + """Final value of every property.""" + done = {} + for key in self.props: + self._value(key, done, []) + return done + + def _value(self, key, done, path): + if key in done: + return done[key] + if key in path: + raise Error("formula cycle: " + " -> ".join(path[path.index(key):] + [key])) + prop = self.props.get(key) + if prop is None: + return None + path.append(key) + total, factor = prop.value, 1.0 + for stage, ops, source in self.formulas.get(key, ()): + r = self._run(key, ops, done, path, source) + if r is None: + continue + if stage == "sum": + total += r + else: + factor *= r + path.pop() + v = total * factor + if prop.clamped: + v = min(max(v, prop.lo), prop.hi) + done[key] = v + return v + + def _run(self, key, ops, done, path, source): + stack = [] + try: + for op in ops: + name = op["op"] + if name == "push": + if "key" in op: + v = self._value(op["key"], done, path) + if v is None: + return None + stack.append(v) + else: + stack.append(op["val"]) + elif name in BINARY: + b, a = stack.pop(), stack.pop() + stack.append(BINARY[name](a, b)) + elif name == "neg": + stack.append(-stack.pop()) + elif name in SPLINES: + stack.append(_spline_op(name, stack)) + else: + raise Error(f"{source}: unknown formula operation {name!r} for {key}") + except (IndexError, KeyError, TypeError, ZeroDivisionError, ValueError): + raise Error(f"{source}: malformed formula for {key}") from None + if len(stack) != 1: + raise Error(f"{source}: malformed formula for {key}") + return stack[0] + + +def _spline_op(name, stack): + raise NotImplementedError # Task 4 +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `python -m unittest tests.test_formula -v` +Expected: 12 tests, OK + +- [ ] **Step 5: Commit** + +```bash +git add generat3d/formula.py tests/test_formula.py +git commit -m "Add formula graph with stages, clamps and cycle detection" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" +``` + +--- + +### Task 4: Spline operations + +**Files:** +- Modify: `generat3d/formula.py` (replace `SPLINES = ()` and `_spline_op`) +- Test: `tests/test_formula.py` (append class) + +Facts (Diffeomorphic `formula.py` `evalMainOper`): a spline formula is `push <input>`, one `push val` per knot, `push val <knot count>`, then the op. Knots are `[x, y]` (constant, linear) or `[x, y, t, c, b]` (tcb). Outside the knot range the end values hold. + +Unverified until the corpus: Daz's exact TCB parameterization. This uses standard Kochanek-Bartels with uniform per-segment parameterization. + +- [ ] **Step 1: Append the failing tests to `tests/test_formula.py`** + +```python +from generat3d.formula import spline + + +def spline_ops(kind, knots): + return ([{"op": "push", "key": "a"}] + [val(k) for k in knots] + + [val(len(knots)), {"op": kind}]) + + +class SplineTest(unittest.TestCase): + LINE = [[0, 0], [1, 1], [2, 2], [3, 3]] + + def test_outside_range_holds_ends(self): + for kind in ("spline_constant", "spline_linear", "spline_tcb"): + self.assertEqual(spline(kind, -1, self.LINE), 0) + self.assertEqual(spline(kind, 9, self.LINE), 3) + + def test_constant(self): + self.assertEqual(spline("spline_constant", 1.7, [[0, 5], [1, 6], [2, 7]]), 6) + + def test_linear(self): + self.assertAlmostEqual(spline("spline_linear", 0.25, [[0, 0], [1, 4]]), 1.0) + + def test_tcb_hits_knots(self): + knots = [[0, 0, 0, 0, 0], [1, 3, 0, 0, 0], [2, 1, 0, 0, 0]] + self.assertEqual(spline("spline_tcb", 1, knots), 3) + + def test_tcb_zero_params_is_catmull_rom(self): + knots = [k + [0, 0, 0] for k in self.LINE] + self.assertAlmostEqual(spline("spline_tcb", 1.5, knots), 1.5) + + def test_spline_in_graph(self): + g = Graph() + g.add_prop("a", 0.5) + g.add_prop("b") + g.add_formula("b", spline_ops("spline_linear", [[0, 0], [1, 10]])) + self.assertAlmostEqual(g.evaluate()["b"], 5.0) +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `python -m unittest tests.test_formula -v` +Expected: ERROR, `ImportError: cannot import name 'spline'` + +- [ ] **Step 3: Implement** + +In `generat3d/formula.py` add `import bisect` to the imports, replace `SPLINES = () # filled in by Task 4` with: + +```python +SPLINES = ("spline_constant", "spline_linear", "spline_tcb") +``` + +and replace the `_spline_op` stub with: + +```python +def _spline_op(name, stack): + n = int(stack.pop()) + if n < 1 or n > len(stack) - 1: + raise IndexError + knots = stack[-n:] + del stack[-n:] + return spline(name, stack.pop(), knots) + + +def spline(kind, x, knots): + """Evaluate a DSON spline at x. Knots are [x, y] or [x, y, t, c, b], sorted by x.""" + xs = [k[0] for k in knots] + ys = [k[1] for k in knots] + if x <= xs[0]: + return ys[0] + if x >= xs[-1]: + return ys[-1] + i = bisect.bisect_right(xs, x) - 1 + if kind == "spline_constant": + return ys[i] + s = (x - xs[i]) / (xs[i + 1] - xs[i]) + if kind == "spline_linear": + return ys[i] + s * (ys[i + 1] - ys[i]) + # ponytail: Kochanek-Bartels with uniform segment parameterization; + # if the corpus shows Daz scales tangents by knot spacing, add that here. + m0 = _tcb_tangent(knots, ys, i, outgoing=True) + m1 = _tcb_tangent(knots, ys, i + 1, outgoing=False) + s2, s3 = s * s, s * s * s + return ((2 * s3 - 3 * s2 + 1) * ys[i] + (s3 - 2 * s2 + s) * m0 + + (-2 * s3 + 3 * s2) * ys[i + 1] + (s3 - s2) * m1) + + +def _tcb_tangent(knots, ys, k, outgoing): + t, c, b = (list(knots[k][2:5]) + [0, 0, 0])[:3] + d_in = ys[k] - ys[max(k - 1, 0)] + d_out = ys[min(k + 1, len(ys) - 1)] - ys[k] + if outgoing: + return (1 - t) * ((1 + b) * (1 + c) * d_in + (1 - b) * (1 - c) * d_out) / 2 + return (1 - t) * ((1 + b) * (1 - c) * d_in + (1 - b) * (1 + c) * d_out) / 2 +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `python -m unittest tests.test_formula -v` +Expected: 18 tests, OK + +- [ ] **Step 5: Commit** + +```bash +git add generat3d/formula.py tests/test_formula.py +git commit -m "Add constant, linear and TCB spline formula operations" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" +``` + +--- + +### Task 5: Scene loading + +**Files:** +- Create: `generat3d/scene.py` +- Create: `tests/data/lib/data/Test/Fig/Fig.dsf` +- Create: `tests/data/lib/data/Test/Fig/Morphs/Test/Tall.dsf` +- Create: `tests/data/lib/data/Test/Fig/Morphs/Test/Wide.dsf` +- Create: `tests/data/lib/data/Test/Fig/Morphs/Test/Broken.dsf` +- Create: `tests/data/scene.duf` +- Test: `tests/test_scene.py` + +Facts: +- `.duf` `scene.nodes[]` entries have `id`, `url` (to a node in a `.dsf` `node_library`), `name`, `label`, optional `geometries[0].url` and, for followers (eyes, mouth, clothing), `conform_target`. Bones of the figure also appear as scene nodes. +- Node types in `node_library`: `figure`, `bone`, others (lights, cameras, props). +- Node channels `center_point`, `end_point`, `orientation` are lists of `{"id": "x", "value": v}` (scene files may carry `current_value`). +- Morph `.dsf`: `modifier_library[]` entries with `id`, `parent`, `channel` (`value`, `min`, `max`, `clamped`), optional `morph` (`vertex_count`, `deltas.values` as `[index, dx, dy, dz]`, optional `hd_url`) and `formulas`. +- Dial values in a `.duf`: `scene.modifiers[].channel.current_value`, or `scene.animations[]` with `url` like `Fig:#Tall?value` and `keys` `[[0, v]]`. +- Daz loads every `.dsf` under the figure's `Morphs` directory (sibling of the figure `.dsf`). +- Property keys: `"{node}:{id}?{channel}"`. Node is the bone id for bone channels and the figure id for everything else. +- Pose channels (`rotation`, `translation`, `scale`, `general_scale`) are not graph properties. Formula inputs from them become rest constants (0, scale 1). + +Spec additions decided here (not in the spec, flag in the final report): more than one non-follower figure in a scene is fatal in v1; a pose in the `.duf` is ignored with one warning. + +- [ ] **Step 1: Write the synthetic library** + +`tests/data/lib/data/Test/Fig/Fig.dsf`: + +```json +{ + "file_version": "0.6.0.0", + "node_library": [ + {"id": "Fig", "name": "Fig", "label": "Test Figure", "type": "figure", + "rotation_order": "XYZ", + "center_point": [{"id": "x", "value": 0}, {"id": "y", "value": 0}, {"id": "z", "value": 0}], + "end_point": [{"id": "x", "value": 0}, {"id": "y", "value": 10}, {"id": "z", "value": 0}], + "orientation": [{"id": "x", "value": 0}, {"id": "y", "value": 0}, {"id": "z", "value": 0}]}, + {"id": "hip", "name": "hip", "label": "Hip", "type": "bone", "parent": "#Fig", + "rotation_order": "YZX", + "center_point": [{"id": "x", "value": 0}, {"id": "y", "value": 5}, {"id": "z", "value": 0}], + "end_point": [{"id": "x", "value": 0}, {"id": "y", "value": 8}, {"id": "z", "value": 0}], + "orientation": [{"id": "x", "value": 0}, {"id": "y", "value": 0}, {"id": "z", "value": 0}]} + ], + "geometry_library": [ + {"id": "Fig-geom", + "vertices": {"count": 3, "values": [[0, 0, 0], [1, 0, 0], [0, 1, 0]]}, + "polygon_material_groups": {"count": 1, "values": ["Skin"]}, + "polylist": {"count": 1, "values": [[0, 0, 0, 1, 2]]}} + ] +} +``` + +`tests/data/lib/data/Test/Fig/Morphs/Test/Tall.dsf`: + +```json +{ + "file_version": "0.6.0.0", + "modifier_library": [ + {"id": "Tall", "parent": "/data/Test/Fig/Fig.dsf#Fig", + "channel": {"id": "value", "type": "float", "value": 0, "min": 0, "max": 1, "clamped": true}, + "morph": {"vertex_count": 3, "deltas": {"count": 1, "values": [[1, 0, 2, 0]]}}, + "formulas": [ + {"output": "hip:/data/Test/Fig/Fig.dsf#hip?center_point/y", + "operations": [{"op": "push", "url": "Fig:/data/Test/Fig/Morphs/Test/Tall.dsf#Tall?value"}, + {"op": "push", "val": 2}, {"op": "mult"}]}, + {"output": "Fig:/data/Test/Fig/Morphs/Test/Wide.dsf#Wide?value", + "operations": [{"op": "push", "url": "Fig:/data/Test/Fig/Morphs/Test/Tall.dsf#Tall?value"}, + {"op": "push", "val": 0.5}, {"op": "mult"}]}, + {"output": "Fig:/data/Test/Fig/Morphs/Test/Wide.dsf#Wide?value", + "operations": [{"op": "push", "url": "hip:/data/Test/Fig/Fig.dsf#hip?rotation/x"}]} + ]} + ] +} +``` + +(The third formula reads a pose channel, which is 0 at rest, so it adds nothing.) + +`tests/data/lib/data/Test/Fig/Morphs/Test/Wide.dsf`: + +```json +{ + "file_version": "0.6.0.0", + "modifier_library": [ + {"id": "Wide", "parent": "/data/Test/Fig/Fig.dsf#Fig", + "channel": {"id": "value", "type": "float", "value": 0, "min": 0, "max": 1, "clamped": true}, + "morph": {"vertex_count": 3, "deltas": {"count": 2, "values": [[0, -1, 0, 0], [1, 1, 0, 0]]}}} + ] +} +``` + +`tests/data/lib/data/Test/Fig/Morphs/Test/Broken.dsf` (deliberately invalid): + +``` +{not json +``` + +`tests/data/scene.duf`: + +```json +{ + "file_version": "0.6.0.0", + "scene": { + "nodes": [ + {"id": "Fig", "url": "/data/Test/Fig/Fig.dsf#Fig", "name": "Fig", "label": "Test Figure"}, + {"id": "hip", "url": "/data/Test/Fig/Fig.dsf#hip", "name": "hip", "parent": "#Fig"} + ], + "animations": [ + {"url": "Fig:/data/Test/Fig/Morphs/Test/Tall.dsf#Tall?value", "keys": [[0, 1]]} + ] + } +} +``` + +- [ ] **Step 2: Write the failing tests** + +`tests/test_scene.py`: + +```python +import contextlib +import io +import json +import tempfile +import unittest +from pathlib import Path + +from generat3d import Error +from generat3d.library import Library +from generat3d.scene import load + +DATA = Path(__file__).parent / "data" + + +def load_quiet(duf, extra_roots=()): + err = io.StringIO() + with contextlib.redirect_stderr(err): + scene = load(duf, Library([DATA / "lib", *extra_roots])) + return scene, err.getvalue() + + +class SceneTest(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + self.dir = Path(self.tmp.name) + + def tearDown(self): + self.tmp.cleanup() + + def duf(self, scene): + p = self.dir / "s.duf" + p.write_text(json.dumps({"scene": scene})) + return p + + def base_nodes(self): + return [{"id": "Fig", "url": "/data/Test/Fig/Fig.dsf#Fig", "name": "Fig"}] + + def test_figure_and_nodes(self): + scene, _ = load_quiet(DATA / "scene.duf") + fig = scene.figure + self.assertEqual((fig.id, fig.name, fig.label), ("Fig", "Fig", "Test Figure")) + self.assertEqual([n.id for n in fig.nodes], ["Fig", "hip"]) + self.assertEqual(fig.nodes[1].rotation_order, "YZX") + self.assertEqual(fig.material_groups, ["Skin"]) + self.assertEqual(fig.verts.tolist(), [[0, 0, 0], [1, 0, 0], [0, 1, 0]]) + + def test_morphs_auto_loaded_and_broken_file_warns(self): + scene, err = load_quiet(DATA / "scene.duf") + self.assertEqual(sorted(scene.figure.morphs), ["Fig:Tall?value", "Fig:Wide?value"]) + self.assertIn("Broken.dsf", err) + + def test_dial_from_animations(self): + scene, _ = load_quiet(DATA / "scene.duf") + self.assertEqual(scene.graph.props["Fig:Tall?value"].value, 1) + + def test_dial_from_modifiers(self): + scene, _ = load_quiet(self.duf({"nodes": self.base_nodes(), "modifiers": [ + {"id": "Tall", "url": "/data/Test/Fig/Morphs/Test/Tall.dsf#Tall", + "parent": "#Fig", "channel": {"id": "value", "current_value": 0.25}}]})) + self.assertEqual(scene.graph.props["Fig:Tall?value"].value, 0.25) + + def test_bone_channels_are_props(self): + scene, _ = load_quiet(DATA / "scene.duf") + self.assertEqual(scene.graph.props["hip:hip?center_point/y"].value, 5) + self.assertEqual(scene.graph.props["Fig:Fig?end_point/y"].value, 10) + + def test_formulas_rewritten_to_keys(self): + scene, _ = load_quiet(DATA / "scene.duf") + ops = [f[1] for f in scene.graph.formulas["Fig:Wide?value"]] + self.assertEqual(ops[0][0], {"op": "push", "key": "Fig:Tall?value"}) + self.assertEqual(ops[1][0], {"op": "push", "val": 0.0}) + + def test_missing_dialed_morphs_listed_and_fatal(self): + mods = [{"id": m, "url": f"/data/Test/Fig/Morphs/Test/{m}.dsf#{m}", "parent": "#Fig", + "channel": {"id": "value", "current_value": 1}} for m in ("Gone1", "Gone2")] + with self.assertRaisesRegex(Error, "(?s)Gone1.dsf.*Gone2.dsf"): + load_quiet(self.duf({"nodes": self.base_nodes(), "modifiers": mods})) + + def test_missing_undialed_morph_is_fine(self): + mods = [{"id": "Gone", "url": "/data/Test/Fig/Morphs/Test/Gone.dsf#Gone", + "parent": "#Fig", "channel": {"id": "value", "current_value": 0}}] + load_quiet(self.duf({"nodes": self.base_nodes(), "modifiers": mods})) + + def test_missing_figure_is_fatal(self): + with self.assertRaisesRegex(Error, "Nope.dsf"): + load_quiet(self.duf({"nodes": [{"id": "X", "url": "/data/Nope.dsf#X"}]})) + + def test_no_figure_is_fatal(self): + with self.assertRaisesRegex(Error, "no figure"): + load_quiet(self.duf({"nodes": []})) + + def test_two_figures_is_fatal(self): + nodes = self.base_nodes() + [{"id": "Fig2", "url": "/data/Test/Fig/Fig.dsf#Fig"}] + with self.assertRaisesRegex(Error, "one figure"): + load_quiet(self.duf({"nodes": nodes})) + + def test_follower_skipped_with_warning(self): + nodes = self.base_nodes() + [{"id": "Shirt", "url": "/data/Test/Fig/Fig.dsf#Fig", + "name": "Shirt", "conform_target": "#Fig"}] + scene, err = load_quiet(self.duf({"nodes": nodes})) + self.assertEqual(scene.figure.id, "Fig") + self.assertIn("Shirt", err) + + def test_pose_ignored_with_warning(self): + nodes = self.base_nodes() + [{"id": "hip", "url": "/data/Test/Fig/Fig.dsf#hip", + "rotation": [{"id": "x", "current_value": 30}]}] + _, err = load_quiet(self.duf({"nodes": nodes})) + self.assertIn("pose", err) +``` + +- [ ] **Step 3: Run tests to verify they fail** + +Run: `python -m unittest tests.test_scene -v` +Expected: ERROR, `ModuleNotFoundError: No module named 'generat3d.scene'` + +- [ ] **Step 4: Implement `generat3d/scene.py`** + +```python +import math +import posixpath +from dataclasses import dataclass, field +from pathlib import Path + +import numpy as np + +from . import Error, warn +from .formula import Graph +from .library import parse_url, read_dson + +AXES = "xyz" +REST_CHANNELS = ("center_point", "end_point", "orientation") +POSE_REST = {"rotation": 0.0, "translation": 0.0, "scale": 1.0, "general_scale": 1.0} + + +def key(node, ident, channel): + return f"{node}:{ident}?{channel}" + + +@dataclass +class Node: + """The figure root or one of its bones.""" + id: str + name: str + label: str + rotation_order: str + + +@dataclass +class Figure: + id: str + name: str + label: str + verts: np.ndarray + material_groups: list + nodes: list # nodes[0] is the figure root, then bones in file order + morphs: dict = field(default_factory=dict) # value key -> (indices, deltas) + + +@dataclass +class Scene: + path: Path + figure: Figure + graph: Graph + + +def _channel(node, name): + comps = {c["id"]: c.get("current_value", c.get("value", 0.0)) for c in node.get(name, [])} + return [comps.get(a, 0.0) for a in AXES] + + +def _lookup(library, doc, url, section): + """Resolve url to (file path, entry in that file's section). Local refs use doc.""" + _, path, ident, _ = parse_url(url) + src = library.load(path) if path else doc + for entry in src.get(section, []): + if entry.get("id") == ident: + return path, entry + raise Error(f"{url}: no {section} entry {ident!r}") + + +class _Loader: + def __init__(self, path, library): + self.path = Path(path) + self.library = library + self.doc = read_dson(self.path) + self.graph = Graph() + + def run(self): + nodes = self.doc.get("scene", {}).get("nodes", []) + figs = [] + posed = False + for n in nodes: + if "url" not in n: + continue + fpath, asset = _lookup(self.library, self.doc, n["url"], "node_library") + kind = asset.get("type") + if kind == "bone": + posed = posed or any(v for ch in ("rotation", "translation") + for v in _channel(n, ch)) + elif kind != "figure": + warn(f"skipped scene node {n.get('name', n['id'])!r} ({kind or 'no type'})") + elif "conform_target" in n: + warn(f"skipped follower {n.get('name', n['id'])!r} (not handled in v1)") + else: + figs.append((n, fpath, asset)) + if not figs: + raise Error(f"{self.path}: no figure in scene") + if len(figs) > 1: + raise Error(f"{self.path}: v1 handles one figure per scene, found {len(figs)}") + self.figure = self._figure(*figs[0]) + self.ids = {n.id for n in self.figure.nodes} | {n.name for n in self.figure.nodes} + self._morphs(figs[0][1]) + posed = self._dials() or posed + if posed: + warn("pose in scene ignored: the .dbz holds rest data") + return Scene(self.path, self.figure, self.graph) + + def _figure(self, inst, fpath, asset): + fdoc = self.library.load(fpath) + geoms = inst.get("geometries", []) + if geoms and "url" in geoms[0]: + _, geom = _lookup(self.library, fdoc, geoms[0]["url"], "geometry_library") + elif fdoc.get("geometry_library"): + geom = fdoc["geometry_library"][0] + else: + raise Error(f"{fpath}: figure has no geometry") + bones = [n for n in fdoc.get("node_library", []) if n.get("type") == "bone"] + nodes = [] + for n in [asset] + bones: + nodes.append(Node(n["id"], n.get("name", n["id"]), n.get("label", n["id"]), + n.get("rotation_order", "XYZ"))) + for ch in REST_CHANNELS: + for a, v in zip(AXES, _channel(n, ch)): + self.graph.add_prop(key(n["id"], n["id"], f"{ch}/{a}"), v) + for f in n.get("formulas", []): + self._formula(f, fpath) + return Figure(asset["id"], inst.get("name", asset.get("name", asset["id"])), + inst.get("label", asset.get("label", asset["id"])), + np.array(geom["vertices"]["values"], dtype=float), + list(geom.get("polygon_material_groups", {}).get("values", [])), + nodes) + + def _key(self, url): + node, _, ident, channel = parse_url(url) + if node not in self.ids or node == self.figure.name: + node = self.figure.id + return key(node, ident, channel) + + def _formula(self, f, source): + ops = [] + for op in f.get("operations", []): + if op.get("op") == "push" and "url" in op: + channel = parse_url(op["url"])[3].split("/")[0] + if channel in POSE_REST: + ops.append({"op": "push", "val": POSE_REST[channel]}) + else: + ops.append({"op": "push", "key": self._key(op["url"])}) + else: + ops.append(op) + self.graph.add_formula(self._key(f["output"]), ops, f.get("stage", "sum"), source) + + def _modifiers(self, path, doc): + nverts = len(self.figure.verts) + for mod in doc.get("modifier_library", []): + parent = parse_url(mod.get("parent", ""))[2] + if parent and parent != self.figure.id: + continue + ch = mod.get("channel", {}) + k = key(self.figure.id, mod["id"], "value") + self.graph.add_prop(k, ch.get("value", 0.0), ch.get("min", -math.inf), + ch.get("max", math.inf), ch.get("clamped", False)) + morph = mod.get("morph") + if morph: + if "hd_url" in morph: + warn(f"{path}#{mod['id']}: HD morph detail ignored") + d = morph.get("deltas", {}).get("values", []) + if morph.get("vertex_count", nverts) != nverts: + warn(f"{path}#{mod['id']}: vertex count mismatch, morph skipped") + elif d: + arr = np.array(d, dtype=float) + self.figure.morphs[k] = (arr[:, 0].astype(int), arr[:, 1:4]) + for f in mod.get("formulas", []): + self._formula(f, f"{path}#{mod['id']}") + + def _morphs(self, fpath): + morph_dir = posixpath.join(posixpath.dirname(fpath), "Morphs") + for path in self.library.walk(morph_dir): + try: + doc = self.library.load(path) + except Error as e: + warn(f"skipped unreadable morph file: {e}") + continue + self._modifiers(path, doc) + + def _dials(self): + """Apply .duf dial values. Returns True if the scene animates a pose channel.""" + scene = self.doc.get("scene", {}) + values, missing, posed = {}, [], False + for m in scene.get("modifiers", []): + if "url" not in m: + continue + k = key(self.figure.id, parse_url(m["url"])[2], "value") + v = m.get("channel", {}).get("current_value", 0.0) + if k not in self.graph.props: + path = parse_url(m["url"])[1] + if path and self.library.find(path): + self._modifiers(path, self.library.load(path)) + if k in self.graph.props: + values[k] = v + elif v: + missing.append(m["url"]) + for a in scene.get("animations", []): + channel = parse_url(a["url"])[3] + v = a.get("keys", [[0, 0.0]])[0][1] + if channel.split("/")[0] in POSE_REST: + posed = posed or bool(v) + elif channel == "value": + k = self._key(a["url"]) + if k in self.graph.props: + values[k] = v + elif v: + missing.append(a["url"]) + if missing: + raise Error("missing morphs used by the scene:\n " + "\n ".join(missing)) + for k, v in values.items(): + self.graph.set_value(k, v) + return posed + + +def load(path, library): + """Load a .duf scene and everything its figure needs.""" + return _Loader(path, library).run() +``` + +- [ ] **Step 5: Run tests to verify they pass** + +Run: `python -m unittest tests.test_scene -v` +Expected: 13 tests, OK + +- [ ] **Step 6: Run the whole suite** + +Run: `python -m unittest -v` +Expected: all tests OK + +- [ ] **Step 7: Commit** + +```bash +git add generat3d/scene.py tests/test_scene.py tests/data +git commit -m "Load .duf scenes: figure, bones, morphs, dials and formulas" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" +``` + +--- + +### Task 6: Evaluate + +**Files:** +- Create: `generat3d/evaluate.py` +- Test: `tests/test_evaluate.py` + +Expected values for `tests/data/scene.duf`: Tall = 1, Wide = 0.5 (formula), hip `center_point/y` = 5 + 2 = 7. Vertex 0 = `[0,0,0] + 0.5*[-1,0,0]`, vertex 1 = `[1,0,0] + 1*[0,2,0] + 0.5*[1,0,0]`, vertex 2 unchanged. + +- [ ] **Step 1: Write the failing test** + +`tests/test_evaluate.py`: + +```python +import contextlib +import io +import unittest +from pathlib import Path + +from generat3d.evaluate import evaluate +from generat3d.library import Library +from generat3d.scene import load + +DATA = Path(__file__).parent / "data" + + +class EvaluateTest(unittest.TestCase): + def test_synthetic_scene(self): + with contextlib.redirect_stderr(io.StringIO()): + scene = load(DATA / "scene.duf", Library([DATA / "lib"])) + verts, rest = evaluate(scene) + self.assertEqual(verts.tolist(), [[-0.5, 0, 0], [1.5, 2, 0], [0, 1, 0]]) + self.assertEqual(rest["hip"]["center_point"], [0, 7, 0]) + self.assertEqual(rest["hip"]["end_point"], [0, 8, 0]) + self.assertEqual(rest["Fig"]["end_point"], [0, 10, 0]) + self.assertEqual(scene.figure.verts.tolist()[0], [0, 0, 0]) # base untouched +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `python -m unittest tests.test_evaluate -v` +Expected: ERROR, `ModuleNotFoundError: No module named 'generat3d.evaluate'` + +- [ ] **Step 3: Implement `generat3d/evaluate.py`** + +```python +from .scene import AXES, REST_CHANNELS, key + + +def evaluate(scene): + """Morphed rest vertices and per-node rest channels {node id: {channel: [x, y, z]}}.""" + values = scene.graph.evaluate() + fig = scene.figure + verts = fig.verts.copy() + for k, (idx, deltas) in fig.morphs.items(): + w = values.get(k, 0.0) + if w: + verts[idx] += w * deltas # indices are unique within one morph + rest = {n.id: {ch: [values[key(n.id, n.id, f"{ch}/{a}")] for a in AXES] + for ch in REST_CHANNELS} + for n in fig.nodes} + return verts, rest +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `python -m unittest tests.test_evaluate -v` +Expected: 1 test, OK + +- [ ] **Step 5: Commit** + +```bash +git add generat3d/evaluate.py tests/test_evaluate.py +git commit -m "Evaluate scenes into morphed vertices and rest data" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" +``` + +--- + +### Task 7: `.dbz` build and write + +**Files:** +- Create: `generat3d/dbz.py` +- Test: `tests/test_dbz.py` + +Facts (Diffeomorphic `export_to_blender.dsa` and `dbzfile.py`, commit 78afc2e): +- Gzipped JSON. `application` must be one of `export_basic_data`, `export_to_blender`, `export_highdef_to_blender`. +- `figures[]` ends with `{"name": "dummy", "num verts": 0}`. Entries without `center_point` are skipped by the reader. +- Figure entry: `name`, `id`, `label`, `center_point`, `end_point`, `orientation`, `rotation_order`, `class` (`DzFigure`), `material groups`, `num verts`, `vertices`, `bones` (each with the node fields plus `origin`, which is the center point). +- `ws_transform` is optional for the reader: without it, head = `center_point` and the world matrix is identity, which is exactly the rest state. v1 omits it (Task 11 confirms Daz writes identity + center point at rest). +- `orientation`: the reader accepts 4 values (quaternion `x, y, z, w`) or 3 (Euler, radians). Daz writes a quaternion. DSON stores Euler degrees. Conversion order is unverified until Task 11; this uses XYZ order. + +- [ ] **Step 1: Write the failing tests** + +`tests/test_dbz.py`: + +```python +import contextlib +import gzip +import io +import json +import math +import tempfile +import unittest +from pathlib import Path + +from generat3d.dbz import build, orientation_quat, write +from generat3d.library import Library +from generat3d.scene import load + +DATA = Path(__file__).parent / "data" + + +class DbzTest(unittest.TestCase): + def setUp(self): + with contextlib.redirect_stderr(io.StringIO()): + self.doc = build(load(DATA / "scene.duf", Library([DATA / "lib"])), [DATA / "lib"]) + + def test_header(self): + self.assertEqual(self.doc["application"], "export_to_blender") + self.assertEqual(self.doc["figures"][-1], {"name": "dummy", "num verts": 0}) + + def test_figure_entry(self): + fig = self.doc["figures"][0] + self.assertEqual((fig["name"], fig["id"], fig["label"], fig["class"]), + ("Fig", "Fig", "Test Figure", "DzFigure")) + self.assertEqual(fig["num verts"], 3) + self.assertEqual(fig["vertices"][1], [1.5, 2.0, 0.0]) + self.assertEqual(fig["material groups"], ["Skin"]) + self.assertEqual(fig["orientation"], [0.0, 0.0, 0.0, 1.0]) + self.assertNotIn("ws_transform", fig) + + def test_bones(self): + hip = self.doc["figures"][0]["bones"][0] + self.assertEqual(hip["name"], "hip") + self.assertEqual(hip["center_point"], [0, 7, 0]) + self.assertEqual(hip["origin"], [0, 7, 0]) + self.assertEqual(hip["rotation_order"], "YZX") + + def test_orientation_quat(self): + x, y, z, w = orientation_quat([90, 0, 0]) + self.assertAlmostEqual(x, math.sqrt(0.5)) + self.assertAlmostEqual(w, math.sqrt(0.5)) + self.assertAlmostEqual(y, 0) + self.assertAlmostEqual(z, 0) + + def test_write_is_gzip_json_and_atomic(self): + with tempfile.TemporaryDirectory() as d: + out = Path(d) / "s.dbz" + write(self.doc, out) + self.assertEqual(json.loads(gzip.decompress(out.read_bytes())), self.doc) + self.assertEqual([p.name for p in Path(d).iterdir()], ["s.dbz"]) + + def test_failed_write_leaves_nothing(self): + with tempfile.TemporaryDirectory() as d: + out = Path(d) / "s.dbz" + with self.assertRaises(TypeError): + write({"bad": object()}, out) + self.assertEqual(list(Path(d).iterdir()), []) +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `python -m unittest tests.test_dbz -v` +Expected: ERROR, `ModuleNotFoundError: No module named 'generat3d.dbz'` + +- [ ] **Step 3: Implement `generat3d/dbz.py`** + +```python +import gzip +import json +import math +import os +import tempfile +from pathlib import Path + +from . import __version__ +from .evaluate import evaluate + + +def orientation_quat(deg): + """DSON orientation (Euler degrees, XYZ) to a quaternion [x, y, z, w].""" + hx, hy, hz = (math.radians(d) / 2 for d in deg) + cx, sx = math.cos(hx), math.sin(hx) + cy, sy = math.cos(hy), math.sin(hy) + cz, sz = math.cos(hz), math.sin(hz) + # q = qz * qy * qx (X applied first) + return [sx * cy * cz - cx * sy * sz, + cx * sy * cz + sx * cy * sz, + cx * cy * sz - sx * sy * cz, + cx * cy * cz + sx * sy * sz] + + +def _node(n, r): + return {"name": n.name, "id": n.id, "label": n.label, + "center_point": r["center_point"], "end_point": r["end_point"], + "orientation": orientation_quat(r["orientation"]), + "rotation_order": n.rotation_order} + + +def build(scene, roots): + """The .dbz document for a loaded scene.""" + verts, rest = evaluate(scene) + fig = scene.figure + root, bones = fig.nodes[0], fig.nodes[1:] + entry = _node(root, rest[root.id]) + entry.update({ + "class": "DzFigure", + "material groups": fig.material_groups, + "num verts": len(verts), + "vertices": verts.tolist(), + "bones": [_node(b, rest[b.id]) | {"origin": rest[b.id]["center_point"]} + for b in bones], + }) + return {"application": "export_to_blender", + "version": f"generat3d {__version__}", + "filepath": str(scene.path), + "rootpaths": {"content": [str(r) for r in roots]}, + "figures": [entry, {"name": "dummy", "num verts": 0}]} + + +def write(doc, out): + """Write doc as gzipped JSON. A temp file plus rename: no partial file on failure.""" + out = Path(out) + fd, tmp = tempfile.mkstemp(dir=out.parent, prefix=out.name, suffix=".tmp") + try: + with os.fdopen(fd, "wb") as f: + f.write(gzip.compress(json.dumps(doc).encode())) + os.replace(tmp, out) + except BaseException: + os.unlink(tmp) + raise +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `python -m unittest tests.test_dbz -v` +Expected: 6 tests, OK + +- [ ] **Step 5: Commit** + +```bash +git add generat3d/dbz.py tests/test_dbz.py +git commit -m "Build and atomically write .dbz documents" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" +``` + +--- + +### Task 8: CLI + +**Files:** +- Create: `generat3d/__main__.py` +- Test: `tests/test_cli.py` + +- [ ] **Step 1: Write the failing tests** + +`tests/test_cli.py`: + +```python +import contextlib +import gzip +import io +import json +import shutil +import tempfile +import unittest +from pathlib import Path + +from generat3d.__main__ import main + +DATA = Path(__file__).parent / "data" + + +class CliTest(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.TemporaryDirectory() + self.dir = Path(self.tmp.name) + self.duf = self.dir / "scene.duf" + shutil.copy(DATA / "scene.duf", self.duf) + + def tearDown(self): + self.tmp.cleanup() + + def run_main(self, *args): + out, err = io.StringIO(), io.StringIO() + with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err): + code = main(list(args)) + return code, out.getvalue(), err.getvalue() + + def test_default_output_beside_scene(self): + code, out, _ = self.run_main(str(self.duf), "-L", str(DATA / "lib")) + self.assertEqual((code, out), (0, "")) + doc = json.loads(gzip.decompress((self.dir / "scene.dbz").read_bytes())) + self.assertEqual(doc["figures"][0]["name"], "Fig") + + def test_explicit_output(self): + target = self.dir / "other.dbz" + code, _, _ = self.run_main(str(self.duf), "-L", str(DATA / "lib"), "-o", str(target)) + self.assertEqual(code, 0) + self.assertTrue(target.exists()) + + def test_error_exits_1_without_output(self): + code, _, err = self.run_main(str(self.duf), "-L", str(self.dir / "nope")) + self.assertEqual(code, 1) + self.assertIn("generat3d: error: library root not found", err) + self.assertFalse((self.dir / "scene.dbz").exists()) +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `python -m unittest tests.test_cli -v` +Expected: ERROR, `ModuleNotFoundError: No module named 'generat3d.__main__'` + +- [ ] **Step 3: Implement `generat3d/__main__.py`** + +```python +import argparse +import sys +from pathlib import Path + +from . import Error +from .config import library_roots +from .dbz import build, write +from .library import Library +from .scene import load + + +def main(argv=None): + ap = argparse.ArgumentParser( + prog="generat3d", + description="Evaluate a Daz scene and write the .dbz file for Diffeomorphic.") + ap.add_argument("scene", type=Path, help="the .duf scene") + ap.add_argument("-L", dest="roots", action="append", default=[], metavar="ROOT", + help="content library root, repeatable, searched in order; " + "replaces library_roots from the config file") + ap.add_argument("-o", dest="out", type=Path, metavar="OUT.dbz", + help="output file (default: the scene path with .dbz)") + args = ap.parse_args(argv) + try: + roots = library_roots(args.roots) + scene = load(args.scene, Library(roots)) + write(build(scene, roots), args.out or args.scene.with_suffix(".dbz")) + except Error as e: + print(f"generat3d: error: {e}", file=sys.stderr) + return 1 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `python -m unittest -v` +Expected: all tests OK (3 new) + +- [ ] **Step 5: Smoke test the real entry point** + +Run: `python -m generat3d tests/data/scene.duf -L tests/data/lib -o /tmp/claude-smoke.dbz; echo "exit $?"; python -m generat3d --help | head -3; rm -f /tmp/claude-smoke.dbz` +Expected: a warning line about `Broken.dsf` on stderr, `exit 0`, then the usage line `usage: generat3d [-h] [-L ROOT] [-o OUT.dbz] scene` + +- [ ] **Step 6: Commit** + +```bash +git add generat3d/__main__.py tests/test_cli.py +git commit -m "Add command line interface" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" +``` + +--- + +### Task 9: Reference VM (needs the user) + +**Files:** none in the repo. + +The user must be present: Windows setup and the Daz login are interactive. Confirm the VM directory with the user before creating anything (proposal: `/data/VMs/generat3d`, it has ~214 GB free). The user is in the `kvm` group but not `libvirt`, so use the session connection (`qemu:///session`), which needs no root. + +- [ ] **Step 1: Check tools** + +Run: `which virt-install virt-viewer remote-viewer mformat mcopy` +Expected: `virt-install` plus at least one viewer and both mtools commands. If mtools is missing, ask the user to install it (they build their own packages). If no viewer, use `--graphics vnc` below and any VNC client. + +- [ ] **Step 2: Create disks** + +```bash +VMDIR=/data/VMs/generat3d # as confirmed with the user +mkdir -p "$VMDIR" +qemu-img create -f qcow2 "$VMDIR/win10.qcow2" 80G +truncate -s 32G "$VMDIR/xfer.img" +parted -s "$VMDIR/xfer.img" mklabel msdos mkpart primary fat32 1MiB 100% +mformat -i "$VMDIR/xfer.img@@1M" -F -v XFER :: +mmd -i "$VMDIR/xfer.img@@1M" ::/scenes ::/corpus ::/Diffeomorphic +``` + +Expected: no errors. `mdir -i "$VMDIR/xfer.img@@1M" ::` lists the three directories. + +- [ ] **Step 3: Put Diffeomorphic's Daz scripts on the transfer disk** + +```bash +git clone --depth 1 https://github.com/Diffeomorphic/import_daz.git /tmp/claude-import_daz +git -C /tmp/claude-import_daz log -1 --format=%h +mcopy -i "$VMDIR/xfer.img@@1M" -s /tmp/claude-import_daz/to_daz_studio/Scripts/Diffeomorphic/* ::/Diffeomorphic/ +``` + +Note the commit hash: Task 11 records it with the corpus. + +- [ ] **Step 4: Create the VM and install Windows** + +```bash +virt-install --connect qemu:///session --name generat3d-win10 \ + --memory 8192 --vcpus 4 --os-variant win10 \ + --cdrom "<path to the Windows 10 22H2 ISO, see the user's ISO directory>" \ + --disk path="$VMDIR/win10.qcow2",format=qcow2,bus=sata \ + --disk path="$VMDIR/xfer.img",format=raw,bus=sata \ + --network user,model=e1000 --graphics spice +``` + +The user completes Windows setup (local account, no product key, "I don't have internet" path if offered). In Disk Management the transfer disk should show as a FAT32 volume labeled XFER; note its drive letter (expected `E:` or `D:`). + +- [ ] **Step 5: Install Daz Studio and content (user)** + +In the VM browser the user logs in at daz3d.com (the agent never handles the credentials), downloads Daz Studio and Install Manager, installs both, and installs Genesis 9 Starter Essentials through Install Manager. Record the Daz Studio version from Help > About. + +- [ ] **Step 6: Install the export scripts (user)** + +Copy `<XFER>:\Diffeomorphic` into the Daz content library's `Scripts` folder (Daz Studio > Content Library > right-click a folder > Browse to folder location shows the library path). + +--- + +### Task 10: Corpus authoring + +**Files:** +- Create: `tools/export_corpus.dsa` +- Create: `fixtures/scenes/*.duf` (copied out of the VM) +- Local only (gitignored): `corpus/*.dbz`, plus the host content library + +- [ ] **Step 1: Write `tools/export_corpus.dsa`** + +The Daz-side batch script. The header uses `//` comments. + +```javascript +// generat3D - evaluate Daz DSON scenes and write .dbz files +// Copyright (C) 2026 Danilo M. <danix@danix.xyz> +// +// This program is free software; you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation; either version 2 of the License, or +// (at your option) any later version. +// +// This program is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License along +// with this program; if not, write to the Free Software Foundation, Inc., +// 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +// Batch corpus export, run from Daz Studio's Script IDE pane. +// Loads every .duf in XFER/scenes and writes XFER/corpus/<name>.dbz with +// Diffeomorphic's export_to_blender.dsa, minus its dialogs. + +var XFER = "E:/"; // drive letter of the XFER disk, edit if different +var EXPORT = XFER + "Diffeomorphic/export_to_blender.dsa"; + +function readText(path) { + var f = new DzFile(path); + f.open(DzFile.ReadOnly); + var text = f.read(); + f.close(); + return text; +} + +var src = readText(EXPORT); +src = src.replace(/\ncreateDialog\(\)\s*$/, "\n"); +src = src.replace(/MessageBox\.information\(\s*msg,\s*appName,\s*"&OK"\s*\);/, "print(msg);"); +eval(src); + +var dir = new DzDir(XFER + "scenes"); +var files = dir.entryList(["*.duf"], DzDir.Files); +for (var i = 0; i < files.length; i++) { + Scene.clear(); + Scene.loadScene(XFER + "scenes/" + files[i], DzScene.OpenNew); + exportToBlender(XFER + "corpus/" + files[i].replace(/\.duf$/, ".dbz")); + print("exported " + files[i]); +} +print("done: " + files.length + " scenes"); +``` + +Commit it, then copy it to the transfer disk with the VM shut down: + +```bash +git add tools/export_corpus.dsa +git commit -m "Add Daz Studio batch script for corpus export" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" +mcopy -i "$VMDIR/xfer.img@@1M" tools/export_corpus.dsa ::/ +``` + +- [ ] **Step 2: Copy the content library to the host (user in VM, then host)** + +In the VM the user copies the Daz content library folder (the one Install Manager installed into) to `<XFER>:\library`, then shuts the VM down. On the host: + +```bash +mkdir -p ~/Daz/library # confirm location with the user +mcopy -i "$VMDIR/xfer.img@@1M" -s -n ::/library/* ~/Daz/library/ +mkdir -p ~/.config/generat3d +printf 'library_roots = ["%s"]\n' "$HOME/Daz/library" > ~/.config/generat3d/config.toml +``` + +This is the user's licensed content on the user's own machine. It stays outside the repo. + +- [ ] **Step 3: Pick the morphs for scenes 3 to 5** + +```bash +python - <<'EOF' +from generat3d.config import library_roots +from generat3d.library import Library, parse_url +lib = Library(library_roots([])) +drivers, joints, clamped_targets = set(), set(), {} +for p in lib.walk("/data/Daz 3D/Genesis 9/Base/Morphs"): + try: + doc = lib.load(p) + except Exception: + continue + for m in doc.get("modifier_library", []): + for f in m.get("formulas", []): + out = parse_url(f["output"]) + if out[3] == "value": + drivers.add(m["id"]) + clamped_targets.setdefault(out[2], set()).add(m["id"]) + if out[3].startswith(("center_point", "end_point")): + joints.add(m["id"]) +print("drives other morphs:", sorted(drivers)[:30]) +print("moves joint centers:", sorted(joints)[:30]) +print("targets with 2+ drivers:", [(t, sorted(d)) for t, d in clamped_targets.items() if len(d) > 1][:10]) +EOF +``` + +Choose, preferring user-facing dials (names without `CTRL` or `JCM` prefixes): M1, a plain morph that appears in none of the lists, for scene 2. M2 from "drives other morphs" for scene 3. M3 from "moves joint centers" for scene 4. For scene 5, two drivers of one shared target. Write the picks down for Step 4. + +- [ ] **Step 4: Author the scenes in Daz (user, guided)** + +In the VM, for each scene: File > New, load Genesis 9 from the Content Library, set the dials with Parameters, File > Save As Scene into `<XFER>:\scenes\`: + +| file | content | +|---|---| +| `01_base.duf` | Genesis 9, nothing changed | +| `02a_morph_full.duf` | M1 = 1.0 | +| `02b_morph_half.duf` | M1 = 0.5 | +| `03_erc.duf` | M2 = 1.0 | +| `04_joints.duf` | M3 = 1.0 | +| `05_stacked.duf` | both scene-5 drivers at 1.0 (their sum pushes the shared target past its limit) | +| `06_posed.duf` | `02a` plus a visible pose (bend an arm and a knee) | + +- [ ] **Step 5: Run the batch export (user)** + +In Daz: Window > Panes (Tabs) > Script IDE. Open `<XFER>:\export_corpus.dsa`, check `XFER` matches the drive letter, Execute. Expected output: `exported 01_base.duf` through `exported 06_posed.duf`, then `done: 7 scenes`. If `DzDir.entryList` or `Scene.loadScene` errors, check the method names in Daz's scripting documentation (Help > Scripting) and fix the script on the host. + +Shut the VM down. + +- [ ] **Step 6: Pull scenes and corpus to the host** + +```bash +mkdir -p fixtures/scenes corpus +mcopy -i "$VMDIR/xfer.img@@1M" -n ::/scenes/*.duf fixtures/scenes/ +mcopy -i "$VMDIR/xfer.img@@1M" -n ::/corpus/*.dbz corpus/ +command ls -1 fixtures/scenes corpus +``` + +Expected: 7 `.duf` files and 7 `.dbz` files. + +- [ ] **Step 7: Check the scenes carry no geometry, then commit them** + +```bash +for f in fixtures/scenes/*.duf; do + python -c "import sys; from generat3d.library import read_dson; d = read_dson(sys.argv[1]); print(sys.argv[1], sorted(d), 'geometry_library' in d or 'modifier_library' in d)" "$f" +done +``` + +Expected: each line ends in `False` (no embedded geometry or morph data). If any prints `True`, stop and ask the user: that scene embeds content and must not be committed as is. + +`.duf` files from Daz are gzipped. Commit them as they are: + +```bash +git status --short # must show fixtures/scenes only, never corpus/ +git add fixtures/scenes +git commit -m "Add corpus scene recipes authored in Daz Studio" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" +``` + +--- + +### Task 11: Corpus findings (decision gate) + +**Files:** +- Modify: `docs/superpowers/specs/2026-10-28-generat3d-v1-design.md` (append a "Corpus findings" section) + +Pins down what the code assumed. Stop and report to the user after this task: answers may change later tasks. + +- [ ] **Step 1: Inspect the reference files** + +```bash +python - <<'EOF' +import numpy as np +from pathlib import Path +from generat3d.library import read_dson +for p in sorted(Path("corpus").glob("*.dbz")): + d = read_dson(p) + print(f"== {p.name}: application={d['application']} version={d.get('version')}") + for f in d["figures"]: + if "center_point" not in f: + continue + print(f" {f['name']!r} class={f.get('class')} verts={f.get('num verts')} " + f"bones={len(f.get('bones', []))} keys={sorted(f)}") + fig = next(f for f in d["figures"] if f.get("class") == "DzFigure") + for b in fig["bones"][:3]: + print(" bone", b["name"], "cp", b["center_point"], "orient", b["orientation"], + "ws", b.get("ws_transform")) +a = read_dson("corpus/02a_morph_full.dbz"); b = read_dson("corpus/06_posed.dbz") +fa = next(f for f in a["figures"] if f.get("class") == "DzFigure") +fb = next(f for f in b["figures"] if f.get("class") == "DzFigure") +print("posed vs unposed max vertex diff:", + np.abs(np.array(fa["vertices"]) - np.array(fb["vertices"])).max()) +EOF +``` + +- [ ] **Step 2: Answer each question and record it in the spec** + +Append to the spec: + +```markdown +## Corpus findings + +Daz Studio <version>, Diffeomorphic scripts at <commit>, corpus exported <date>. + +1. ws_transform at rest: <identity rotation + center_point | other: describe>. +2. orientation: <4 values, matches XYZ conversion of DSON degrees | other order: which>. +3. Figure entries per scene: <only Genesis 9 | Genesis 9 plus followers: list names and classes>. +4. Pose: max vertex difference between 02a and 06 is <value>. <rest assumption holds | .dbz is posed>. +``` + +Rules for the answers: +- Q1: if `ws_transform` rotation is not identity or its translation is not `center_point` in an unposed scene, Task 7's omission is wrong: add a task that writes `ws_transform`. +- Q2: compare `orientation` of a bone with nonzero DSON orientation against `orientation_quat` of its `.dsf` values. If no Euler order matches, report and stop. +- Q3: if followers (Genesis 9 Eyes, Mouth, Eyelashes, Tear) appear as `DzFigure` entries with vertices, Task 13 is required for v1. +- Q4: if the difference is above 1e-4, the `.dbz` is posed. Stop: v1 scope needs skinning. Revise the spec with the user before going on. + +- [ ] **Step 3: Commit the findings** + +```bash +git add docs/superpowers/specs/2026-10-28-generat3d-v1-design.md +git commit -m "Record corpus findings in the v1 spec" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" +``` + +- [ ] **Step 4: Report to the user and wait** before Task 12. + +--- + +### Task 12: Reference tests and fix loop + +**Files:** +- Create: `tests/test_reference.py` +- Modify: whichever module the failures point to + +- [ ] **Step 1: Write the reference test** + +`tests/test_reference.py`: + +```python +import contextlib +import io +import unittest +from pathlib import Path + +import numpy as np + +from generat3d import Error +from generat3d.config import library_roots +from generat3d.dbz import build +from generat3d.library import Library, read_dson +from generat3d.scene import load + +REPO = Path(__file__).parent.parent +CORPUS = REPO / "corpus" +SCENES = REPO / "fixtures" / "scenes" +TOL = 1e-4 # cm + + +def check(tc, label, ours, ref): + ours, ref = np.asarray(ours, dtype=float), np.asarray(ref, dtype=float) + tc.assertEqual(ours.shape, ref.shape, f"{label}: shape") + err = np.abs(ours - ref) + bad = int((err > TOL).sum()) + if bad: + at = np.unravel_index(err.argmax(), err.shape) + tc.fail(f"{label}: {bad} values over {TOL}, worst {err.max():.6g} at {at}") + + +def check_quats(tc, label, ours, ref): + ours, ref = np.asarray(ours, dtype=float), np.asarray(ref, dtype=float) + flip = np.abs(ours - ref).max(axis=1) > np.abs(ours + ref).max(axis=1) + check(tc, label, np.where(flip[:, None], -ours, ours), ref) # q and -q are the same rotation + + +@unittest.skipUnless(CORPUS.is_dir(), "corpus/ absent") +class ReferenceTest(unittest.TestCase): + def test_scenes(self): + try: + roots = library_roots([]) + except Error as e: + self.skipTest(str(e)) + refs = sorted(CORPUS.glob("*.dbz")) + self.assertTrue(refs, "corpus/ has no .dbz files") + for ref_path in refs: + with self.subTest(scene=ref_path.stem): + with contextlib.redirect_stderr(io.StringIO()): + ours = build(load(SCENES / f"{ref_path.stem}.duf", Library(roots)), roots) + ref = {f["name"]: f for f in read_dson(ref_path)["figures"]} + for fig in ours["figures"][:-1]: + want = ref[fig["name"]] + check(self, f"{fig['name']} vertices", fig["vertices"], want["vertices"]) + mine = {b["name"]: b for b in fig["bones"]} + theirs = {b["name"]: b for b in want["bones"]} + self.assertEqual(sorted(mine), sorted(theirs), "bone names") + names = sorted(mine) + for ch in ("center_point", "end_point"): + check(self, f"bones {ch}", [mine[n][ch] for n in names], + [theirs[n][ch] for n in names]) + check_quats(self, "bones orientation", + [mine[n]["orientation"] for n in names], + [theirs[n]["orientation"] for n in names]) +``` + +- [ ] **Step 2: Run it** + +Run: `python -m unittest tests.test_reference -v` +Expected at first: some subtests FAIL. Each failure names the scene, the quantity, the worst error and its index. + +- [ ] **Step 3: Fix loop, one scene at a time, in order 01 to 06** + +For each failing scene: find the cause with superpowers:systematic-debugging before changing code. Likely causes, by scene: +- 01: geometry choice, bone list or order, orientation convention (Task 11 Q2). +- 02a/02b: delta application, morph auto-loading paths. +- 03: formula key normalization in `scene._key` (node names vs ids), stages. +- 04: joint center formulas, `end_point`. +- 05: clamp semantics (final value vs dial only). If Daz clamps only the dial, change `Graph._value` to clamp `prop.value` before formulas and add a unit test for it. +- 06: should equal 02a (Task 11 Q4). + +Every fix gets a unit test with a hand-written snippet in `tests/` that fails before the fix. Commit each fix separately: + +```bash +git add <files> +git commit -m "<what was wrong and how it is fixed>" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" +``` + +- [ ] **Step 4: All green** + +Run: `python -m unittest -v` +Expected: all tests OK, reference subtests included (not skipped). + +- [ ] **Step 5: Commit the test** + +```bash +git add tests/test_reference.py +git commit -m "Add reference tests against Daz-exported corpus" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" +``` + +--- + +### Task 13: Follower figures (only if Task 11 Q3 found them) + +**Files:** +- Modify: `generat3d/scene.py`, `generat3d/evaluate.py`, `generat3d/dbz.py` +- Test: `tests/test_scene.py`, `tests/data/lib/data/Test/Fig/Eyes/...` (new synthetic follower) + +Genesis 9's eyes, mouth, eyelashes and tear are separate figures conformed to the body. Without them the face morphs move the face but not the eyes. This task is a spec change for v1: confirm with the user before starting. Its concrete design depends on Task 11's findings, so write the detailed steps then, using the same TDD shape as Task 5. Starting point, all to be verified against the corpus: +- Each follower is a figure with its own `.dsf`, its own `Morphs` directory and `conform_target` pointing at the body. +- Follower morph properties whose channel has `"auto_follow": true` take the value of the body property with the same id. Model it as one extra sum-stage formula per such property: `push key(body_id, id, "value")`. +- Follower bones take the body bone values of the same name. +- Keys stay unambiguous by prefixing follower keys with the follower's figure id, as for the body. + +Done when the follower entries in the corpus pass the same reference checks as the body. + +--- + +### Task 14: Blender acceptance and docs + +**Files:** +- Modify: `README.md`, `AGENTS.md` + +- [ ] **Step 1: Blender acceptance (user, once Blender is available)** + +On the host: `python -m generat3d fixtures/scenes/04_joints.duf -o /tmp/claude-04.dbz`, then copy `fixtures/scenes/04_joints.duf` and the `.dbz` to a scratch directory with the same base name. In Blender with Diffeomorphic: Easy Import of the `.duf` with Mesh Fitting = DBZ File. Repeat with the Daz-exported `corpus/04_joints.dbz`. The two imports must look the same: shape, joint positions when posed in Blender. + +- [ ] **Step 2: Update README usage** + +In `README.md`: change "Status: in design." to "Status: v1, Genesis 9 figures." In "Usage", remove "Planned interface:" and add after the config block: + +```markdown +Example: + + python -m generat3d ~/scenes/alice.duf + +writes `~/scenes/alice.dbz`. In Blender, Easy Import `alice.duf` with +Diffeomorphic and set Mesh Fitting to "DBZ File". + +Warnings (skipped clothing, ignored pose, unreadable morph files) go to +stderr. Errors exit with status 1 and leave no `.dbz` behind. +``` + +- [ ] **Step 3: Update AGENTS.md Testing** + +Replace the Testing section body with: + +```markdown +`python -m unittest -v` from the repo root. Unit tests use the synthetic +library in `tests/data/lib` and always run. `tests/test_reference.py` +compares against Daz-exported `.dbz` files in the gitignored `corpus/`, +using the library roots from `~/.config/generat3d/config.toml`, and skips +when either is missing. `tools/export_corpus.dsa` regenerates the corpus +inside Daz Studio (see the v1 plan, Tasks 9 and 10). +``` + +Add to "Repo facts": `- Daz Studio, Diffeomorphic commit and findings behind the corpus: spec section "Corpus findings".` + +- [ ] **Step 4: Commit and push** + +```bash +git add README.md AGENTS.md +git commit -m "Document v1 usage and the test setup" -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>" +git push +``` |