From 155529e45c70519e01b5f4d758ccaa8f68b6d10b Mon Sep 17 00:00:00 2001 From: "Danilo M." Date: Wed, 28 Oct 2026 10:09:11 +0100 Subject: Add v1 design spec: DSON figure evaluator Scope (Genesis 9 morphs, formulas, joint centers), .dbz format notes, architecture, CLI and config, error policy, reference corpus produced by Daz in a throwaway Windows VM, and the content licensing rule that keeps Daz geometry out of the repo. Co-Authored-By: Claude Opus 5.5 --- .../specs/2026-10-28-generat3d-v1-design.md | 207 +++++++++++++++++++++ 1 file changed, 207 insertions(+) create mode 100644 docs/superpowers/specs/2026-10-28-generat3d-v1-design.md (limited to 'docs/superpowers/specs/2026-10-28-generat3d-v1-design.md') diff --git a/docs/superpowers/specs/2026-10-28-generat3d-v1-design.md b/docs/superpowers/specs/2026-10-28-generat3d-v1-design.md new file mode 100644 index 0000000..aac991f --- /dev/null +++ b/docs/superpowers/specs/2026-10-28-generat3d-v1-design.md @@ -0,0 +1,207 @@ +# generat3D v1 design: DSON figure evaluator + +Date: 2026-10-28. Status: approved in design session, pending spec review. + +## Goal + +Remove Daz Studio from a Linux pipeline of Daz content, Blender (Diffeomorphic +DAZ Importer), Cycles renders, Ren'Py. Today Diffeomorphic's most accurate +import mode ("Mesh Fitting: DBZ File") needs a `.dbz` file that only Daz +Studio can write, via Diffeomorphic's `export_to_blender.dsa`. generat3D +writes that file natively. + +## Background + +- The DSON format is publicly documented by Daz + (`docs.daz3d.com/doku.php/public/dson_spec/start`). +- Diffeomorphic (GPLv2 or later) already reads DSON. It is reference reading + only, not a dependency. Its formula evaluation is bound to Blender drivers. +- Clean-room rule: no reverse engineering of Daz Studio. Sources are the + spec, Diffeomorphic's source, and reference `.dbz` files exported by Daz. +- Content license: renders of Daz content are covered by the standard + license. The 3D data itself is never redistributed by this project. + +## The `.dbz` format + +Read from `to_daz_studio/Scripts/Diffeomorphic/export_to_blender.dsa` +(Diffeomorphic, commit 78afc2e) and `dbzfile.py`. Gzipped JSON. Top level: +content directory lists and `figures`, an array with one entry per scene +node, terminated by a `{"name": "dummy", "num verts": 0}` entry. A figure +entry has `name`, `id`, `label`, `center_point`, `end_point`, `orientation`, +`rotation_order`, `ws_transform`, `class`, the current-shape mesh (`num +verts`, `vertices`, `uvset`, `uvs`, `faces`, `material groups`) and `bones`, +each bone carrying the same node fields plus `origin`. Diffeomorphic treats +all of this as rest data; posing happens on the Blender armature. + +Assumption to verify with corpus scene 6: the exported mesh is the morphed +rest shape, unaffected by pose. If false, v1 scope grows to include skinning +and this spec is revised. + +The exact layout (key order, number formatting, which header keys +Diffeomorphic requires) is pinned down by diffing against corpus scene 1. + +## Scope + +v1 (tier 1): Genesis 9 figures. Base mesh, shape morphs, formula links +between dials (ERC), formula-driven joint centers (`center_point`, +`end_point`, orientation), limits. Output: the figure's `.dbz` entry. + +Out of v1: + +- Pose and skinning (Blender does it). +- Conforming clothing and hair fitting, auto-generated fit morphs (tier 2, + own spec). Until then Diffeomorphic imports them in its approximate mode. +- HD morphs and geografts (tier 3, own spec). +- dForce, Iray materials (materials come from the `.duf` via Diffeomorphic). +- Genesis 8 and older: not targeted, not rejected. Revisit after v1. +- A debug OBJ export (declined: Blender import is the visual check). + +## Architecture + +``` +scene.duf -> resolve figure .dsf via library roots -> base mesh + skeleton + -> auto-load every .dsf in the figure's Morphs directories + -> apply dial values from .duf -> property graph -> evaluate + -> morphed rest vertices + bone rest data -> scene.dbz +``` + +Daz loads every morph found in a figure's Morphs directories when the figure +loads, whether or not the scene references it, because formulas in those +files can drive other properties. generat3D does the same. + +Package `generat3d`: + +| file | job | depends on | +|---|---|---| +| `library.py` | resolve DSON URIs (`/data/...dsf#id`, percent-encoded) across library roots in order, first match wins; read gzipped or plain JSON; cache | stdlib | +| `config.py` | load `config.toml`, validate | stdlib (`tomllib`) | +| `scene.py` | load `.duf`: figure instance, base geometry, skeleton, auto-discovered morphs, dial values | library | +| `formula.py` | property graph: operation stacks (push, add, mult, spline), sum and multiply stages, min/max clamps, topological evaluation, cycle detection | stdlib | +| `evaluate.py` | final property values to morphed vertices (numpy, sparse deltas) and bone rest data | scene, formula, numpy | +| `dbz.py` | write the `.dbz` in the export script's layout | evaluate | +| `__main__.py` | CLI | all | + +`formula.py` knows nothing about meshes: property ids in, numbers out. Tiers 2 +and 3 reuse `library`, `scene` and `formula` and add consumers next to +`evaluate`. + +## CLI + +``` +python -m generat3d scene.duf [-L ROOT ...] [-o OUT.dbz] +``` + +- Output defaults to `scene.dbz` beside `scene.duf`: Diffeomorphic finds the + `.dbz` by that name. +- `-L` is repeatable and searched in order. When given, it replaces the + config roots (no merging). +- Exit codes: 0 ok, 1 error, 2 usage. +- Success prints nothing to stdout. Warnings and errors go to stderr. + +## Config + +`$XDG_CONFIG_HOME/generat3d/config.toml`, falling back to +`~/.config/generat3d/config.toml`: + +```toml +library_roots = ["/home/you/Daz/My Library"] +``` + +- No roots from `-L` or config: fatal, `no library roots: pass -L or set + library_roots in `. +- A configured or `-L` root that does not exist: fatal. +- Unknown keys: fatal. +- Missing config file is fine when `-L` is given. +- `library_roots` is the only key in v1. + +## Error handling + +Policy: fail loudly. A wrong `.dbz` is worse than none, Diffeomorphic would +import it silently. + +| situation | behavior | +|---|---| +| figure's base `.dsf` unresolved | fatal | +| morph used by the `.duf` with a nonzero value missing | collect all missing URIs, list them, fatal | +| unreadable file in an auto-loaded Morphs directory | warning, skip (Daz does the same) | +| unknown formula operation or stage | fatal, naming property id and file | +| cycle in the formula graph | fatal, printing the cycle; revisit if real content has legitimate cycles | +| malformed JSON or gzip | fatal, with file path | +| non-figure scene nodes (clothing, hair, props) | warning, not written | +| HD morphs or geografts present | warning, ignored | + +The `.dbz` is written to a temporary file in the target directory and +renamed into place, so a failed run never leaves a partial file. + +## Correctness + +Ground truth is Daz itself: each corpus scene is exported to `.dbz` inside a +Windows VM with Diffeomorphic's export script. Each reference file records the +Daz Studio version that produced it. + +Corpus (Genesis 9 Starter Essentials, free with Daz Studio), one feature per +scene: + +| # | scene | proves | +|---|---|---| +| 1 | base G9, all dials 0 | URI resolution, geometry load, bone rest data, `.dbz` layout | +| 2 | one plain morph at 1.0, and a copy at 0.5 | delta application, linear scaling | +| 3 | a morph that drives others through formulas | formula engine, auto-loaded Morphs directories | +| 4 | a morph that moves joint centers | bone `center_point` and `end_point` formulas | +| 5 | several dials stacked, one past its limit | sum and multiply stages, clamps | +| 6 | scene 2, posed | rest-data assumption; expected identical to scene 2 | + +Comparison: max absolute error per vertex coordinate and per bone value, +tolerance 1e-4 cm. A failure reports the worst offender and the count over +tolerance. + +Acceptance for v1: all six scenes within tolerance, and one generated `.dbz` +imported by Diffeomorphic in Blender with DBZ mesh fitting, looking the same +as the Daz-exported one. + +## Tests + +stdlib `unittest`, run with `python -m unittest` from the repo root. + +- Unit tests for `formula.py`, `library.py` and `config.py` on small DSON + snippets written by hand. No Daz content, committed, always run. +- Reference tests diff generat3D output against `corpus/`. They skip with a + notice when `corpus/` is absent. + +## Repo layout + +``` +generat3d/ the package +tests/ unit and reference tests +fixtures/scenes/*.duf committed scene recipes, URIs and dial values only +tools/export_corpus.dsa batch export script, run inside the VM +corpus/ gitignored, reference .dbz files from Daz +docs/superpowers/specs/ design specs +``` + +Reference `.dbz` files contain Daz geometry and are never committed. + +## Reference VM + +Throwaway Windows VM on the host, used only to produce the corpus. + +- libvirt/KVM via `virt-install`, from a local Windows 10 22H2 ISO. Windows 10 + because Windows 11 needs a TPM and `swtpm` is not installed. No activation. +- Emulated SATA disk and e1000 NIC: no virtio ISO on the host, no guest + drivers needed. +- Network on only to download Daz Studio, Install Manager and Genesis 9 + Starter Essentials. The user types the Daz credentials in the VM; the agent + never reads them. +- Transfer: a second raw disk image, FAT32 formatted on the host. The VM + writes the exports there; after shutdown the host loop-mounts it read-only + and copies to `corpus/`. No network share, no SSH into Windows. +- `tools/export_corpus.dsa` opens each `.duf`, runs Diffeomorphic's export + logic, and writes `.dbz` files to the transfer disk. Daz Studio accepts a + script path on its command line, so the batch runs without clicking. + +## Later + +- Tier 2: conforming clothing and hair fitting. +- Tier 3: HD morphs and geografts. +- A Blender sidebar for library browsing and presets (own spec). +- Offer the evaluator upstream to Diffeomorphic once v1 is proven. -- cgit v1.2.3