diff options
| -rw-r--r-- | AGENTS.md | 39 | ||||
| -rw-r--r-- | README.md | 19 | ||||
| -rw-r--r-- | docs/superpowers/specs/2026-10-30-upscale-qt-design.md | 6 | ||||
| -rw-r--r-- | test_upscale_qt.py | 37 | ||||
| -rwxr-xr-x | upscale-qt | 123 |
5 files changed, 116 insertions, 108 deletions
@@ -7,10 +7,10 @@ Guidance for AI coding agents working in this repository. `upscale-qt`: single-file PyQt6 desktop app (`upscale-qt`, no `.py` extension) to upscale an image with an ESRGAN model through `sd-cli` (stable-diffusion.cpp), compare before/after with a slider wipe, then save -as a new file or overwrite the original. A small vision model (imajev, run -one-shot through `llama-mtmd-cli`) suggests which upscaler to use. The only -Python dependency is PyQt6; everything else is stdlib plus the two external -binaries. Keep it a single file and do not add dependencies. +as a new file or overwrite the original. The model dropdown shows a +description per model from an optional `models.txt` in the upscalers +folder. The only Python dependency is PyQt6; everything else is stdlib plus +`sd-cli`. Keep it a single file and do not add dependencies. Design spec: `docs/superpowers/specs/2026-10-30-upscale-qt-design.md`. @@ -18,10 +18,12 @@ Design spec: `docs/superpowers/specs/2026-10-30-upscale-qt-design.md`. - Upscale: `sd-cli -M upscale --upscale-model <model> -i <in> -o <out.png>`. Models are `.pth`/`.safetensors`/`.gguf` files in the upscalers dir. -- Suggestion: `llama-mtmd-cli -m <imajev gguf> --mmproj <mmproj gguf> - --image <in> -p <prompt>`. Its answer is free text and gets matched to a - filename from the upscalers dir; never trust it as a path. -- Both run through `QProcess` so the UI never blocks. +- `sd-cli` runs through `QProcess` so the UI never blocks. +- sd-cli (stb_image) ignores EXIF orientation. `load()` applies it with + `QImageReader.setAutoTransform` and feeds sd-cli an upright temp PNG. +- `models.txt` lives with the models, not in the repo: `filename = what it + is for`, `#` comments. Model names stay out of the repo on purpose, the + personal-data hook flags at least one of them as a real name. - `sd-cli` loads ESRGAN (RRDBNet) and MoSR models only, detected from the weights. SwinIR, HAT, DAT, SPAN, Compact and face restorers do not load. - Model choice, measured on 4x crops of a clean portrait, a landscape and a @@ -32,8 +34,15 @@ Design spec: `docs/superpowers/specs/2026-10-30-upscale-qt-design.md`. blocks), 4xNomos8kSC (artifacts on noisy input), 4xLSDIRplusR (plastic on clean input), 4xLSDIRplusC (close to the foolhardy model, not better). - Timing: an RRDBNet model takes about 25 s per megapixel of input, so a - 10 MP photo takes about 4 minutes and outputs a 160 MP image. imajev - answers in about 7 s. + 10 MP photo takes about 4 minutes and outputs a 160 MP image. +- Model suggestion by a local vision model (imajev 4B via + `llama-mtmd-cli`) was built and removed. Measured on four photos it + picked the right model about half the time. It is a thinking model and + `llama-mtmd-cli` has no switch to turn thinking off (no + `--chat-template-kwargs`, `/no_think` ignored); with `--temp 0` it loops + on `</think> <answer>` until `-n`, without `--jinja` it emits an empty + answer. Do not bring it back without a better model or a server that + can disable thinking. ## Conventions @@ -46,7 +55,15 @@ Design spec: `docs/superpowers/specs/2026-10-30-upscale-qt-design.md`. ## Testing -No test suite yet. + python3 test_upscale_qt.py + +Plain asserts for the helpers plus offscreen smoke tests of `CompareView` +and `Main`. The test sets `XDG_CONFIG_HOME` to a temp dir, so it never +touches the real settings. For an end-to-end check with the real +`sd-cli`, load the script with `SourceFileLoader` under +`QT_QPA_PLATFORM=offscreen`, call `load()` and `upscale()`, wait on +`proc.finished` with a `QEventLoop`, and `grab()` a screenshot. Use a small +image: about 25 s per input megapixel. ## Repo facts @@ -1,18 +1,33 @@ # upscale-qt -Small PyQt6 app to upscale an image with an ESRGAN model through `sd-cli` (stable-diffusion.cpp). A local vision model suggests which upscaler fits the image, a slider wipe compares before and after, and the result can be saved as a new file or written over the original. +Small PyQt6 app to upscale an image with an ESRGAN model through `sd-cli` (stable-diffusion.cpp). A slider wipe compares before and after, and the result can be saved as a new file or written over the original. ## Requirements - Python 3 - PyQt6 - `sd-cli` from stable-diffusion.cpp, plus one or more ESRGAN upscaler models in a folder -- Optional: `llama-mtmd-cli` from llama.cpp and the imajev model with its mmproj, for upscaler suggestions ## Usage ./upscale-qt [image] +- Open an image (Ctrl+O, or pass it as argument). EXIF rotation is applied. +- Pick a model and Upscale (Ctrl+U). Expect roughly 25 seconds per megapixel of input; the result is 4x in each direction. +- The view shows before left of the divider and after right of it. Drag the divider to wipe, drag elsewhere to pan, mouse wheel to zoom, double-click to fit. +- Save As (Ctrl+Shift+S) writes a new file. Overwrite replaces the original after a confirmation, in the same format. EXIF metadata is not kept. +- Settings holds the upscalers folder. + +### Model descriptions + +Put a `models.txt` next to the models to show what each one is for in the dropdown: + + # filename = what it is for + some_model_x4.pth = Clean photos: people, nature + other_model_x4.pth = Noisy or compressed photos + +Models without a line show their filename. + ## License Copyright (C) 2026 Danilo M. <danix@danix.xyz> diff --git a/docs/superpowers/specs/2026-10-30-upscale-qt-design.md b/docs/superpowers/specs/2026-10-30-upscale-qt-design.md index ffcb5da..88f938f 100644 --- a/docs/superpowers/specs/2026-10-30-upscale-qt-design.md +++ b/docs/superpowers/specs/2026-10-30-upscale-qt-design.md @@ -1,5 +1,11 @@ # upscale-qt design +> **Revised 2026-10-30 during implementation:** the imajev model suggestion +> below was built, measured (right about half the time, thinking cannot be +> disabled in `llama-mtmd-cli`) and removed. The dropdown instead shows +> descriptions from an optional `models.txt` in the upscalers folder. Jobs +> carry no state enum yet. See AGENTS.md for the measurements. + ## Goal Pick an image, preview it, upscale it with an ESRGAN model through `sd-cli`, diff --git a/test_upscale_qt.py b/test_upscale_qt.py index 1fbfe2c..cb11346 100644 --- a/test_upscale_qt.py +++ b/test_upscale_qt.py @@ -30,20 +30,6 @@ _loader.exec_module(uq) from PyQt6.QtGui import QColor, QImage, QImageReader # noqa: E402 -NAMES = ['4x_detail.pth', 'RealESRGAN_x4plus.pth', - 'RealESRGAN_x4plus_anime_6B.pth'] - - -def test_match_model(): - m = uq.match_model - # the longer stem must win over its own prefix - assert m('RealESRGAN_x4plus_anime_6B.pth', NAMES) == 'RealESRGAN_x4plus_anime_6B.pth' - assert m(' realesrgan_x4plus\n', NAMES) == 'RealESRGAN_x4plus.pth' - assert m('I suggest 4x_detail.', NAMES) == '4x_detail.pth' - assert m('no idea', NAMES) is None - assert m('', []) is None - - def test_list_models(): d = os.path.join(TMP, 'models') os.makedirs(d) @@ -53,15 +39,25 @@ def test_list_models(): assert uq.list_models(os.path.join(TMP, 'missing')) == [] +def test_descriptions(): + d = os.path.join(TMP, 'models') + with open(os.path.join(d, 'models.txt'), 'w') as f: + f.write('# comment = ignored\n' + 'a.pth = Clean photos: people, nature\n' + '\n' + 'no separator here\n' + ' c.GGUF=Anime \n') + desc = uq.read_descriptions(d) + assert desc == {'a.pth': 'Clean photos: people, nature', 'c.GGUF': 'Anime'} + assert uq.model_label('a.pth', desc) == 'Clean photos: people, nature [a]' + assert uq.model_label('b.safetensors', desc) == 'b.safetensors' + assert uq.read_descriptions(os.path.join(TMP, 'missing')) == {} + + def test_args(): a = uq.sd_args('/m/x.pth', '/in.png', '/out.png') assert a == ['-M', 'upscale', '--upscale-model', '/m/x.pth', '-i', '/in.png', '-o', '/out.png'] - a = uq.llama_args('/m.gguf', '/mm.gguf', '/in.png', NAMES) - assert a[a.index('--image') + 1] == '/in.png' - prompt = a[a.index('-p') + 1] - assert all(n in prompt for n in NAMES) - assert 'noise' in prompt def test_save_over(): @@ -98,7 +94,6 @@ def test_compare_view(): def test_main_window(): w = uq.Main() - w.cfg.setValue('imajev_model', '') # never run the real VLM in tests w.cfg.setValue('upscalers_dir', os.path.join(TMP, 'models')) w.reload_models() assert not w.a_up.isEnabled() # no image yet @@ -110,6 +105,8 @@ def test_main_window(): assert len(w.jobs) == 1 and w.jobs[0].image.size() == img.size() assert os.path.exists(w.jobs[0].input) assert w.a_up.isEnabled() and not w.a_save.isEnabled() + assert w.models.itemText(0) == 'Clean photos: people, nature [a]' + assert w.models.currentData() == 'a.pth' if __name__ == '__main__': @@ -16,7 +16,6 @@ # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. import os -import shutil import sys from dataclasses import dataclass from pathlib import Path @@ -30,17 +29,8 @@ from PyQt6.QtWidgets import (QApplication, QComboBox, QDialog, MODEL_EXTS = ('.pth', '.safetensors', '.gguf') JPEG_QUALITY = 95 -DEFAULTS = { - 'upscalers_dir': '/data/LLM-models/SD/upscalers', - 'imajev_model': '/data/LLM-models/SD/imajev/imajev-4b-Q4_K_M.gguf', - 'imajev_mmproj': '/data/LLM-models/SD/imajev/imajev-4b-mmproj-f16.gguf', -} -PROMPT = ( - 'You pick an image upscaler model. Judge the subject (photo of people, ' - 'landscape, anime, illustration) and also the source quality: noise, ' - 'JPEG compression, blur. Noisy or compressed sources need a denoising ' - 'model, clean sources a detail-preserving one. Answer with exactly one ' - 'filename from this list and nothing else: {names}') +DESCRIPTIONS = 'models.txt' # optional, in the upscalers folder +DEFAULTS = {'upscalers_dir': '/data/LLM-models/SD/upscalers'} def list_models(folder): @@ -51,27 +41,30 @@ def list_models(folder): return [] -def match_model(answer, names): - """Map imajev's free-text answer to one of names, or None. +def read_descriptions(folder): + """Parse models.txt: 'filename = what it is for' per line, # comments.""" + try: + with open(os.path.join(folder, DESCRIPTIONS), encoding='utf-8') as f: + lines = f.read().splitlines() + except OSError: + return {} + out = {} + for line in lines: + name, sep, desc = line.partition('=') + if sep and not line.lstrip().startswith('#'): + out[name.strip()] = desc.strip() + return out + - Longest stem first, so 'x4plus_anime_6B' is not taken for 'x4plus'.""" - answer = answer.lower() - for name in sorted(names, key=lambda n: len(Path(n).stem), reverse=True): - if Path(name).stem.lower() in answer: - return name - return None +def model_label(name, descriptions): + desc = descriptions.get(name) + return f'{desc} [{Path(name).stem}]' if desc else name def sd_args(model, src, out): return ['-M', 'upscale', '--upscale-model', model, '-i', src, '-o', out] -def llama_args(model, mmproj, src, names): - return ['-m', model, '--mmproj', mmproj, '--image', src, - '--temp', '0', '-n', '64', - '-p', PROMPT.format(names=' '.join(names))] - - def save_over(img, path): """Replace path with img in the original's format, never truncating it.""" fmt = bytes(QImageReader.imageFormat(path)).decode() or None @@ -93,7 +86,7 @@ def tail(data, lines=15): class Job: path: str # file the user opened, overwrite target image: QImage # original, EXIF orientation applied - input: str # oriented PNG fed to sd-cli and imajev + input: str # oriented PNG fed to sd-cli model: str = '' out: str = '' result: QImage | None = None @@ -202,8 +195,6 @@ class SettingsDialog(QDialog): self.edits = {k: QLineEdit(str(cfg.value(k, v))) for k, v in DEFAULTS.items()} form = QFormLayout(self) form.addRow('Upscalers folder', self.edits['upscalers_dir']) - form.addRow('imajev model (.gguf)', self.edits['imajev_model']) - form.addRow('imajev mmproj (.gguf)', self.edits['imajev_mmproj']) bb = QDialogButtonBox(QDialogButtonBox.StandardButton.Ok | QDialogButtonBox.StandardButton.Cancel) bb.accepted.connect(self.accept) @@ -226,7 +217,7 @@ class Main(QMainWindow): self.jobs = [] # one job for now, batch mode adds more self.cur = None self.runs = 0 # unique temp file names - self.what = None # 'suggest' or 'upscale' while a process runs + self.what = None # 'upscale' while sd-cli runs self.err = b'' self.view = CompareView() @@ -268,15 +259,18 @@ class Main(QMainWindow): def job(self): return self.jobs[self.cur] if self.cur is not None else None - def names(self): - return [self.models.itemText(i) for i in range(self.models.count())] - def reload_models(self): - keep = self.models.currentText() + keep = self.models.currentData() + folder = self.get('upscalers_dir') + descriptions = read_descriptions(folder) self.models.clear() - self.models.addItems(list_models(self.get('upscalers_dir'))) - if keep: - self.models.setCurrentText(keep) + for name in list_models(folder): + self.models.addItem(model_label(name, descriptions), name) + self.models.setItemData(self.models.count() - 1, name, + Qt.ItemDataRole.ToolTipRole) + i = self.models.findData(keep) + if i >= 0: + self.models.setCurrentIndex(i) self.refresh() def refresh(self): @@ -317,7 +311,6 @@ class Main(QMainWindow): self.view.set_images(img) self.statusBar().showMessage(f'{img.width()}x{img.height()}') self.refresh() - self.suggest() def run(self, what, prog, args, msg): self.what, self.err = what, b'' @@ -325,18 +318,9 @@ class Main(QMainWindow): self.refresh() self.proc.start(prog, args) - def suggest(self): - model, mmproj = self.get('imajev_model'), self.get('imajev_mmproj') - if not (self.names() and os.path.isfile(model) and os.path.isfile(mmproj) - and shutil.which('llama-mtmd-cli')): - return # suggestion is optional, skip quietly - self.run('suggest', 'llama-mtmd-cli', - llama_args(model, mmproj, self.job().input, self.names()), - 'imajev is looking at the image...') - def upscale(self): job = self.job() - job.model = self.models.currentText() + job.model = self.models.currentData() self.runs += 1 job.out = self.tmp.filePath(f'out{self.runs}.png') self.run('upscale', 'sd-cli', @@ -346,43 +330,32 @@ class Main(QMainWindow): def proc_stderr(self): self.err = (self.err + bytes(self.proc.readAllStandardError()))[-8000:] - if self.what == 'upscale': - last = tail(self.err, 1) - if last: - self.statusBar().showMessage(last) + last = tail(self.err, 1) + if last: + self.statusBar().showMessage(last) def proc_done(self, code, status): - what, self.what = self.what, None + self.what = None ok = code == 0 and status == QProcess.ExitStatus.NormalExit job = self.job() - if what == 'suggest': - answer = bytes(self.proc.readAllStandardOutput()).decode(errors='replace') - pick = match_model(answer, self.names()) if ok else None - if pick: - self.models.setCurrentText(pick) - self.statusBar().showMessage(f'imajev suggests {pick}') - else: - self.statusBar().showMessage('imajev gave no usable suggestion') - elif what == 'upscale': - img = QImage(job.out) if ok else QImage() - if img.isNull(): - self.statusBar().showMessage('Upscale failed') - QMessageBox.warning(self, 'Upscale failed', tail(self.err) or f'exit code {code}') - else: - job.result = img - self.view.set_images(job.image, img) - self.statusBar().showMessage( - f'{job.image.width()}x{job.image.height()} -> ' - f'{img.width()}x{img.height()} with {job.model}') + img = QImage(job.out) if ok else QImage() + if img.isNull(): + self.statusBar().showMessage('Upscale failed') + QMessageBox.warning(self, 'Upscale failed', tail(self.err) or f'exit code {code}') + else: + job.result = img + self.view.set_images(job.image, img) + self.statusBar().showMessage( + f'{job.image.width()}x{job.image.height()} -> ' + f'{img.width()}x{img.height()} with {job.model}') self.refresh() def proc_error(self, err): if err != QProcess.ProcessError.FailedToStart: return # crashes also emit finished, handled there - what, self.what = self.what, None + self.what = None prog = self.proc.program() - if what == 'upscale': - QMessageBox.warning(self, 'upscale-qt', f'Cannot start {prog}: is it installed?') + QMessageBox.warning(self, 'upscale-qt', f'Cannot start {prog}: is it installed?') self.statusBar().showMessage(f'Cannot start {prog}') self.refresh() |