aboutsummaryrefslogtreecommitdiffstats
path: root/AGENTS.md
diff options
context:
space:
mode:
authorDanilo M. <danix@danix.xyz>2026-10-02 20:59:46 +0200
committerDanilo M. <danix@danix.xyz>2026-10-02 20:59:46 +0200
commit69cc958d714b7cae798aa0b1389c92a308313e0c (patch)
tree66a2d99bbcf4fb6683c003cef94ed7de37c15eb0 /AGENTS.md
downloadtrackcrop-master.tar.gz
trackcrop-master.zip
Add trackcrop: crop a video to a square around a tracked objectHEADmaster
Single-file Python script. The user boxes an object on the first frame, a CSRT tracker follows it, the path is smoothed over ~0.5 s and the square crop is encoded with h264_vaapi, audio copied. --ak820 FPS limits the output to the AJAZZ AK820 Pro screen's 255 frames at FPS, prompting for which window to keep (start, end or a start second) when the clip is longer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Diffstat (limited to 'AGENTS.md')
-rw-r--r--AGENTS.md71
1 files changed, 71 insertions, 0 deletions
diff --git a/AGENTS.md b/AGENTS.md
new file mode 100644
index 0000000..65e1098
--- /dev/null
+++ b/AGENTS.md
@@ -0,0 +1,71 @@
+# trackcrop
+
+Single-file Python script (`trackcrop.py`) that crops a video to a 1:1 square,
+keeping a user-selected object centered. See README.md for usage.
+
+## How it works
+
+1. `cv2.selectROI` on the first frame, the user draws a box.
+2. Pass 1: CSRT tracker (`cv2.TrackerCSRT_create`, needs opencv-contrib) records
+ the object center per frame; when tracking is lost the last center is reused.
+3. Centers are smoothed with a ~0.5s moving average (edge-padded, no lag).
+4. Pass 2: re-read the video, crop an `s x s` square (`s = min(w, h)`) clamped
+ to the frame, pipe raw BGR frames to ffmpeg.
+
+Two passes keep memory flat, frames are never all held in RAM.
+
+`--ak820 FPS` limits the output to the AK820 Pro screen's 255 frames at FPS:
+if the clip is longer, an `input()` prompt picks the window (start, end or a
+start second) before `selectROI`. Both passes skip to the window's first frame
+with `grab()` (exact on any codec, unlike `CAP_PROP_POS_FRAMES` seeking) and
+stop after `count` frames; audio is cut with `-ss`/`-t` on the second ffmpeg
+input.
+
+## Environment facts
+
+- The local ffmpeg is built without `libx264` and without the native `aac`
+ encoder. Encoding uses `h264_vaapi` on `/dev/dri/renderD128`. NVENC, AMF and
+ Vulkan H.264 encoders are listed but do not work on this machine.
+- Audio is stream-copied, so no audio encoder is needed. `--no-audio` drops it.
+- For test clips use `libvpx-vp9` video and `libopus` audio.
+- `drawbox` with a `t` expression did not animate in testing, use `overlay`
+ with `y='...*t'` to make a moving test object.
+
+## Testing
+
+No test suite. Verify end to end headless by generating a synthetic clip with a
+moving red square, stubbing the GUI, and measuring the square's offset from the
+output frame center:
+
+```python
+import sys, runpy, cv2
+cv2.namedWindow = lambda *a: None
+cv2.selectROI = lambda *a: (310, 200, 100, 100) # box around the test object
+cv2.destroyAllWindows = lambda: None
+sys.argv = ["trackcrop.py", "in.mp4", "out.mp4"]
+runpy.run_path("trackcrop.py")
+```
+
+For `--ak820`, also stub the prompt (`builtins.input = lambda p: "e"`), use a
+`testsrc2` background (it has a frame counter), and check the output's first
+frame against the source with `cv2.matchTemplate`: the best match must be the
+window's first frame.
+
+Test clip:
+
+```sh
+ffmpeg -f lavfi -i "color=gray:s=720x1280:d=4:r=30" \
+ -f lavfi -i "color=red:s=100x100:d=4:r=30" -f lavfi -i "sine=d=4" \
+ -filter_complex "[0][1]overlay=x=310:y='200+200*t'" \
+ -c:v libvpx-vp9 -c:a libopus -shortest in.mp4
+```
+
+A solid square is hard for CSRT (no texture), expect some drift late in the
+clip; real footage tracks better.
+
+## Conventions
+
+- Keep it a single file, stdlib plus numpy/opencv only.
+- Every source file carries the GPLv2-only header
+ (`Copyright (C) <year> Danilo M. <danix@danix.xyz>`).
+- Work directly on master.