diff options
| author | Danilo M. <danix@danix.xyz> | 2026-10-30 19:15:03 +0100 |
|---|---|---|
| committer | Danilo M. <danix@danix.xyz> | 2026-10-30 19:15:03 +0100 |
| commit | 69810152d0dfedeaa02a96df73cbc184225a1278 (patch) | |
| tree | 23b1f2c4cd3904831edccc942ab9277179f3121a | |
| parent | 610bf7f3432037f7f8cfb641b05e39e48830f82a (diff) | |
| download | upscale-qt-69810152d0dfedeaa02a96df73cbc184225a1278.tar.gz upscale-qt-69810152d0dfedeaa02a96df73cbc184225a1278.zip | |
Drop imajev suggestion, describe models from models.txt
The vision model picked the right upscaler about half the time on four
test photos, and llama-mtmd-cli cannot turn its thinking off: greedy
decoding loops, larger budgets cost seconds and still miss. The dropdown
now shows a one-line description per model from an optional models.txt
next to the models, so the choice stays with the user but is readable.
Document usage, model descriptions and how to test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| -rw-r--r-- | AGENTS.md | 39 | ||||
| -rw-r--r-- | README.md | 19 | ||||
| -rw-r--r-- | docs/superpowers/specs/2026-10-30-upscale-qt-design.md | 6 | ||||
| -rw-r--r-- | test_upscale_qt.py | 37 | ||||
| -rwxr-xr-x | upscale-qt | 123 |
5 files changed, 116 insertions, 108 deletions
@@ -7,10 +7,10 @@ Guidance for AI coding agents working in this repository. `upscale-qt`: single-file PyQt6 desktop app (`upscale-qt`, no `.py` extension) to upscale an image with an ESRGAN model through `sd-cli` (stable-diffusion.cpp), compare before/after with a slider wipe, then save -as a new file or overwrite the original. A small vision model (imajev, run -one-shot through `llama-mtmd-cli`) suggests which upscaler to use. The only -Python dependency is PyQt6; everything else is stdlib plus the two external -binaries. Keep it a single file and do not add dependencies. +as a new file or overwrite the original. The model dropdown shows a +description per model from an optional `models.txt` in the upscalers +folder. The only Python dependency is PyQt6; everything else is stdlib plus +`sd-cli`. Keep it a single file and do not add dependencies. Design spec: `docs/superpowers/specs/2026-10-30-upscale-qt-design.md`. @@ -18,10 +18,12 @@ Design spec: `docs/superpowers/specs/2026-10-30-upscale-qt-design.md`. - Upscale: `sd-cli -M upscale --upscale-model <model> -i <in> -o <out.png>`. Models are `.pth`/`.safetensors`/`.gguf` files in the upscalers dir. -- Suggestion: `llama-mtmd-cli -m <imajev gguf> --mmproj <mmproj gguf> - --image <in> -p <prompt>`. Its answer is free text and gets matched to a - filename from the upscalers dir; never trust it as a path. -- Both run through `QProcess` so the UI never blocks. +- `sd-cli` runs through `QProcess` so the UI never blocks. +- sd-cli (stb_image) ignores EXIF orientation. `load()` applies it with + `QImageReader.setAutoTransform` and feeds sd-cli an upright temp PNG. +- `models.txt` lives with the models, not in the repo: `filename = what it + is for`, `#` comments. Model names stay out of the repo on purpose, the + personal-data hook flags at least one of them as a real name. - `sd-cli` loads ESRGAN (RRDBNet) and MoSR models only, detected from the weights. SwinIR, HAT, DAT, SPAN, Compact and face restorers do not load. - Model choice, measured on 4x crops of a clean portrait, a landscape and a @@ -32,8 +34,15 @@ Design spec: `docs/superpowers/specs/2026-10-30-upscale-qt-design.md`. blocks), 4xNomos8kSC (artifacts on noisy input), 4xLSDIRplusR (plastic on clean input), 4xLSDIRplusC (close to the foolhardy model, not better). - Timing: an RRDBNet model takes about 25 s per megapixel of input, so a - 10 MP photo takes about 4 minutes and outputs a 160 MP image. imajev - answers in about 7 s. + 10 MP photo takes about 4 minutes and outputs a 160 MP image. +- Model suggestion by a local vision model (imajev 4B via + `llama-mtmd-cli`) was built and removed. Measured on four photos it + picked the right model about half the time. It is a thinking model and + `llama-mtmd-cli` has no switch to turn thinking off (no + `--chat-template-kwargs`, `/no_think` ignored); with `--temp 0` it loops + on `</think> <answer>` until `-n`, without `--jinja` it emits an empty + answer. Do not bring it back without a better model or a server that + can disable thinking. ## Conventions @@ -46,7 +55,15 @@ Design spec: `docs/superpowers/specs/2026-10-30-upscale-qt-design.md`. ## Testing -No test suite yet. + python3 test_upscale_qt.py + +Plain asserts for the helpers plus offscreen smoke tests of `CompareView` +and `Main`. The test sets `XDG_CONFIG_HOME` to a temp dir, so it never +touches the real settings. For an end-to-end check with the real +`sd-cli`, load the script with `SourceFileLoader` under +`QT_QPA_PLATFORM=offscreen`, call `load()` and `upscale()`, wait on +`proc.finished` with a `QEventLoop`, and `grab()` a screenshot. Use a small +image: about 25 s per input megapixel. ## Repo facts @@ -1,18 +1,33 @@ # upscale-qt -Small PyQt6 app to upscale an image with an ESRGAN model through `sd-cli` (stable-diffusion.cpp). A local vision model suggests which upscaler fits the image, a slider wipe compares before and after, and the result can be saved as a new file or written over the original. +Small PyQt6 app to upscale an image with an ESRGAN model through `sd-cli` (stable-diffusion.cpp). A slider wipe compares before and after, and the result can be saved as a new file or written over the original. ## Requirements - Python 3 - PyQt6 - `sd-cli` from stable-diffusion.cpp, plus one or more ESRGAN upscaler models in a folder -- Optional: `llama-mtmd-cli` from llama.cpp and the imajev model with its mmproj, for upscaler suggestions ## Usage ./upscale-qt [image] +- Open an image (Ctrl+O, or pass it as argument). EXIF rotation is applied. +- Pick a model and Upscale (Ctrl+U). Expect roughly 25 seconds per megapixel of input; the result is 4x in each direction. +- The view shows before left of the divider and after right of it. Drag the divider to wipe, drag elsewhere to pan, mouse wheel to zoom, double-click to fit. +- Save As (Ctrl+Shift+S) writes a new file. Overwrite replaces the original after a confirmation, in the same format. EXIF metadata is not kept. +- Settings holds the upscalers folder. + +### Model descriptions + +Put a `models.txt` next to the models to show what each one is for in the dropdown: + + # filename = what it is for + some_model_x4.pth = Clean photos: people, nature + other_model_x4.pth = Noisy or compressed photos + +Models without a line show their filename. + ## License Copyright (C) 2026 Danilo M. <danix@danix.xyz> diff --git a/docs/superpowers/specs/2026-10-30-upscale-qt-design.md b/docs/superpowers/specs/2026-10-30-upscale-qt-design.md index ffcb5da..88f938f 100644 --- a/docs/superpowers/specs/2026-10-30-upscale-qt-design.md +++ b/docs/superpowers/specs/2026-10-30-upscale-qt-design.md @@ -1,5 +1,11 @@ # upscale-qt design +> **Revised 2026-10-30 during implementation:** the imajev model suggestion +> below was built, measured (right about half the time, thinking cannot be +> disabled in `llama-mtmd-cli`) and removed. The dropdown instead shows +> descriptions from an optional `models.txt` in the upscalers folder. Jobs +> carry no state enum yet. See AGENTS.md for the measurements. + ## Goal Pick an image, preview it, upscale it with an ESRGAN model through `sd-cli`, diff --git a/test_upscale_qt.py b/test_upscale_qt.py index 1fbfe2c..cb11346 100644 --- a/test_upscale_qt.py +++ b/test_upscale_qt.py @@ -30,20 +30,6 @@ _loader.exec_module(uq) from PyQt6.QtGui import QColor, QImage, QImageReader # noqa: E402 -NAMES = ['4x_detail.pth', 'RealESRGAN_x4plus.pth', - 'RealESRGAN_x4plus_anime_6B.pth'] - - -def test_match_model(): - m = uq.match_model - # the longer stem must win over its own prefix - assert m('RealESRGAN_x4plus_anime_6B.pth', NAMES) == 'RealESRGAN_x4plus_anime_6B.pth' - assert m(' realesrgan_x4plus\n', NAMES) == 'RealESRGAN_x4plus.pth' - assert m('I suggest 4x_detail.', NAMES) == '4x_detail.pth' - assert m('no idea', NAMES) is None - assert m('', []) is None - - def test_list_models(): d = os.path.join(TMP, 'models') os.makedirs(d) @@ -53,15 +39,25 @@ def test_list_models(): assert uq.list_models(os.path.join(TMP, 'missing')) == [] +def test_descriptions(): + d = os.path.join(TMP, 'models') + with open(os.path.join(d, 'models.txt'), 'w') as f: + f.write('# comment = ignored\n' + 'a.pth = Clean photos: people, nature\n' + '\n' + 'no separator here\n' + ' c.GGUF=Anime \n') + desc = uq.read_descriptions(d) + assert desc == {'a.pth': 'Clean photos: people, nature', 'c.GGUF': 'Anime'} + assert uq.model_label('a.pth', desc) == 'Clean photos: people, nature [a]' + assert uq.model_label('b.safetensors', desc) == 'b.safetensors' + assert uq.read_descriptions(os.path.join(TMP, 'missing')) == {} + + def test_args(): a = uq.sd_args('/m/x.pth', '/in.png', '/out.png') assert a == ['-M', 'upscale', '--upscale-model', '/m/x.pth', '-i', '/in.png', '-o', '/out.png'] - a = uq.llama_args('/m.gguf', '/mm.gguf', '/in.png', NAMES) - assert a[a.index('--image') + 1] == '/in.png' - prompt = a[a.index('-p') + 1] - assert all(n in prompt for n in NAMES) - assert 'noise' in prompt def test_save_over(): @@ -98,7 +94,6 @@ def test_compare_view(): def test_main_window(): w = uq.Main() - w.cfg.setValue('imajev_model', '') # never run the real VLM in tests w.cfg.setValue('upscalers_dir', os.path.join(TMP, 'models')) w.reload_models() assert not w.a_up.isEnabled() # no image yet @@ -110,6 +105,8 @@ def test_main_window(): assert len(w.jobs) == 1 and w.jobs[0].image.size() == img.size() assert os.path.exists(w.jobs[0].input) assert w.a_up.isEnabled() and not w.a_save.isEnabled() + assert w.models.itemText(0) == 'Clean photos: people, nature [a]' + assert w.models.currentData() == 'a.pth' if __name__ == '__main__': @@ -16,7 +16,6 @@ # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. import os -import shutil import sys from dataclasses import dataclass from pathlib import Path @@ -30,17 +29,8 @@ from PyQt6.QtWidgets import (QApplication, QComboBox, QDialog, MODEL_EXTS = ('.pth', '.safetensors', '.gguf') JPEG_QUALITY = 95 -DEFAULTS = { - 'upscalers_dir': '/data/LLM-models/SD/upscalers', - 'imajev_model': '/data/LLM-models/SD/imajev/imajev-4b-Q4_K_M.gguf', - 'imajev_mmproj': '/data/LLM-models/SD/imajev/imajev-4b-mmproj-f16.gguf', -} -PROMPT = ( - 'You pick an image upscaler model. Judge the subject (photo of people, ' - 'landscape, anime, illustration) and also the source quality: noise, ' - 'JPEG compression, blur. Noisy or compressed sources need a denoising ' - 'model, clean sources a detail-preserving one. Answer with exactly one ' - 'filename from this list and nothing else: {names}') +DESCRIPTIONS = 'models.txt' # optional, in the upscalers folder +DEFAULTS = {'upscalers_dir': '/data/LLM-models/SD/upscalers'} def list_models(folder): @@ -51,27 +41,30 @@ def list_models(folder): return [] -def match_model(answer, names): - """Map imajev's free-text answer to one of names, or None. +def read_descriptions(folder): + """Parse models.txt: 'filename = what it is for' per line, # comments.""" + try: + with open(os.path.join(folder, DESCRIPTIONS), encoding='utf-8') as f: + lines = f.read().splitlines() + except OSError: + return {} + out = {} + for line in lines: + name, sep, desc = line.partition('=') + if sep and not line.lstrip().startswith('#'): + out[name.strip()] = desc.strip() + return out + - Longest stem first, so 'x4plus_anime_6B' is not taken for 'x4plus'.""" - answer = answer.lower() - for name in sorted(names, key=lambda n: len(Path(n).stem), reverse=True): - if Path(name).stem.lower() in answer: - return name - return None +def model_label(name, descriptions): + desc = descriptions.get(name) + return f'{desc} [{Path(name).stem}]' if desc else name def sd_args(model, src, out): return ['-M', 'upscale', '--upscale-model', model, '-i', src, '-o', out] -def llama_args(model, mmproj, src, names): - return ['-m', model, '--mmproj', mmproj, '--image', src, - '--temp', '0', '-n', '64', - '-p', PROMPT.format(names=' '.join(names))] - - def save_over(img, path): """Replace path with img in the original's format, never truncating it.""" fmt = bytes(QImageReader.imageFormat(path)).decode() or None @@ -93,7 +86,7 @@ def tail(data, lines=15): class Job: path: str # file the user opened, overwrite target image: QImage # original, EXIF orientation applied - input: str # oriented PNG fed to sd-cli and imajev + input: str # oriented PNG fed to sd-cli model: str = '' out: str = '' result: QImage | None = None @@ -202,8 +195,6 @@ class SettingsDialog(QDialog): self.edits = {k: QLineEdit(str(cfg.value(k, v))) for k, v in DEFAULTS.items()} form = QFormLayout(self) form.addRow('Upscalers folder', self.edits['upscalers_dir']) - form.addRow('imajev model (.gguf)', self.edits['imajev_model']) - form.addRow('imajev mmproj (.gguf)', self.edits['imajev_mmproj']) bb = QDialogButtonBox(QDialogButtonBox.StandardButton.Ok | QDialogButtonBox.StandardButton.Cancel) bb.accepted.connect(self.accept) @@ -226,7 +217,7 @@ class Main(QMainWindow): self.jobs = [] # one job for now, batch mode adds more self.cur = None self.runs = 0 # unique temp file names - self.what = None # 'suggest' or 'upscale' while a process runs + self.what = None # 'upscale' while sd-cli runs self.err = b'' self.view = CompareView() @@ -268,15 +259,18 @@ class Main(QMainWindow): def job(self): return self.jobs[self.cur] if self.cur is not None else None - def names(self): - return [self.models.itemText(i) for i in range(self.models.count())] - def reload_models(self): - keep = self.models.currentText() + keep = self.models.currentData() + folder = self.get('upscalers_dir') + descriptions = read_descriptions(folder) self.models.clear() - self.models.addItems(list_models(self.get('upscalers_dir'))) - if keep: - self.models.setCurrentText(keep) + for name in list_models(folder): + self.models.addItem(model_label(name, descriptions), name) + self.models.setItemData(self.models.count() - 1, name, + Qt.ItemDataRole.ToolTipRole) + i = self.models.findData(keep) + if i >= 0: + self.models.setCurrentIndex(i) self.refresh() def refresh(self): @@ -317,7 +311,6 @@ class Main(QMainWindow): self.view.set_images(img) self.statusBar().showMessage(f'{img.width()}x{img.height()}') self.refresh() - self.suggest() def run(self, what, prog, args, msg): self.what, self.err = what, b'' @@ -325,18 +318,9 @@ class Main(QMainWindow): self.refresh() self.proc.start(prog, args) - def suggest(self): - model, mmproj = self.get('imajev_model'), self.get('imajev_mmproj') - if not (self.names() and os.path.isfile(model) and os.path.isfile(mmproj) - and shutil.which('llama-mtmd-cli')): - return # suggestion is optional, skip quietly - self.run('suggest', 'llama-mtmd-cli', - llama_args(model, mmproj, self.job().input, self.names()), - 'imajev is looking at the image...') - def upscale(self): job = self.job() - job.model = self.models.currentText() + job.model = self.models.currentData() self.runs += 1 job.out = self.tmp.filePath(f'out{self.runs}.png') self.run('upscale', 'sd-cli', @@ -346,43 +330,32 @@ class Main(QMainWindow): def proc_stderr(self): self.err = (self.err + bytes(self.proc.readAllStandardError()))[-8000:] - if self.what == 'upscale': - last = tail(self.err, 1) - if last: - self.statusBar().showMessage(last) + last = tail(self.err, 1) + if last: + self.statusBar().showMessage(last) def proc_done(self, code, status): - what, self.what = self.what, None + self.what = None ok = code == 0 and status == QProcess.ExitStatus.NormalExit job = self.job() - if what == 'suggest': - answer = bytes(self.proc.readAllStandardOutput()).decode(errors='replace') - pick = match_model(answer, self.names()) if ok else None - if pick: - self.models.setCurrentText(pick) - self.statusBar().showMessage(f'imajev suggests {pick}') - else: - self.statusBar().showMessage('imajev gave no usable suggestion') - elif what == 'upscale': - img = QImage(job.out) if ok else QImage() - if img.isNull(): - self.statusBar().showMessage('Upscale failed') - QMessageBox.warning(self, 'Upscale failed', tail(self.err) or f'exit code {code}') - else: - job.result = img - self.view.set_images(job.image, img) - self.statusBar().showMessage( - f'{job.image.width()}x{job.image.height()} -> ' - f'{img.width()}x{img.height()} with {job.model}') + img = QImage(job.out) if ok else QImage() + if img.isNull(): + self.statusBar().showMessage('Upscale failed') + QMessageBox.warning(self, 'Upscale failed', tail(self.err) or f'exit code {code}') + else: + job.result = img + self.view.set_images(job.image, img) + self.statusBar().showMessage( + f'{job.image.width()}x{job.image.height()} -> ' + f'{img.width()}x{img.height()} with {job.model}') self.refresh() def proc_error(self, err): if err != QProcess.ProcessError.FailedToStart: return # crashes also emit finished, handled there - what, self.what = self.what, None + self.what = None prog = self.proc.program() - if what == 'upscale': - QMessageBox.warning(self, 'upscale-qt', f'Cannot start {prog}: is it installed?') + QMessageBox.warning(self, 'upscale-qt', f'Cannot start {prog}: is it installed?') self.statusBar().showMessage(f'Cannot start {prog}') self.refresh() |
