c0fefc19f16da6e1b95af2a00f36165ae1acc209
27
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 tfaour
|
454c03586e |
Port real epd13in3e driver from vendor code; fix wire-raster stride bug
Vendored the panel's init/LUT/refresh register sequence from three
independent Waveshare reference drivers for this exact panel+controller
(RaspberryPi/c, ESP32, and the ESP32-S3-ePaper-13.3E6 ESP-IDF example),
which all agree byte-for-byte. The epd13in3e.c #error is gone; it
compiles clean and links (verified via /build-firmware ee02).
That vendor code also revealed the panel's SPI wire raster is a native
1200x1600 (portrait), not 1600x1200 as previously assumed -- rotated 90
degrees from the panel's landscape mount/marketing size. The old
assumption wasn't just a rotation bug: 1600x1200 and 1200x1600 don't
share a row stride, so packing at the wrong one would have shredded
images into a repeating diagonal garble on real hardware, not just
displayed them sideways. Fixed with a new PANEL_WIRE_TRANSPOSE in
image_pipeline.py, applied after the existing per-frame
ORIENTATION_TRANSPOSE, with a direction-agnostic regression test that
catches the stride bug specifically (a byte-count check alone can't,
since both orientations pack to the same total size).
A full ee02 build still fails, but no longer because of this driver --
main/{back,next,combo}_button.c call an ESP32-C6-only deep-sleep
GPIO-wakeup API with no ESP32-S3 fallback, a separate pre-existing gap
that was simply hidden behind the panel driver's old #error. See
docs/hardware.md for details; CI's continue-on-error on this board
stays in place until that's fixed too.
|
||
|
tfaour
|
474b92a282 |
Add server-side support for a second panel (13.3in Spectra 6 / EE02) and scaffold its firmware target
Server: Frame.panel_type (new column + migration) is auto-derived from the device's reported board (X-Frame-Board), never user-set -- the panel is a property of the hardware, not a picker in the UI. image_pipeline's packing/render pipeline is parameterized by panel geometry instead of hardcoded 800x480 globals, with the real confirmed 13.3in geometry (1600x1200) registered alongside the original 7.3in panel. Existing 7.3in frames are unaffected (column default + board mapping both resolve to the original panel). Board identifiers are also renamed (devkit/xiao -> devkit_esp32c6/ xiao_esp32c6, plus new "ee02") since the EE02 board also carries a XIAO module -- "xiao" alone stopped disambiguating hardware. The server keeps accepting the legacy bare names indefinitely for already-flashed devices. Firmware: scaffolds a third build target (ee02, ESP32-S3 -- a real chip-target change, not just a same-chip Kconfig variant like xiao) and a new epd13in3e driver component skeleton. The actual panel init/LUT/ refresh register sequence isn't ported from vendor demo code yet (none was available), so that component deliberately fails to compile (#error) rather than risk sending unverified register values to real hardware -- devkit/xiao are unaffected and build identically to before. CI's ee02 build step is continue-on-error for the same reason. |
||
|
tfaour
|
8ea1c53ec3 |
Add experimental HTML/CSS "modern" render style for weather widget
The weather widget's icons/layout are hand-drawn PIL primitives -- clean under quantization but flat, no gradients/shadows. Adds an opt-in render_style="modern" (current/daily modes only) that instead renders a Jinja2 template through a persistent headless-Chromium browser (app/html_render.py), following the approach of Tesserae, an open-source e-ink dashboard targeting this same panel family. Key design points: - The Chromium dependency (Playwright) is lazily imported only when a weather widget actually uses "modern" style, and the background browser itself only launches on first use -- every other widget type, and this one's own classic/hourly/multi_city paths, never pay for it. - No Frame-level dithering setting needed: html_render dithers its own rendered widget to exact palette colors (Bayer/ordered, not Floyd-Steinberg) before compositing, so the shared whole-canvas Floyd-Steinberg pass sees zero quantization error there and leaves it untouched -- same trick draw_text/hand-drawn icons already use. Floyd- Steinberg keeps working unchanged for photos and every other widget. - A "Load calibrated Spectra 6 preset" button in Advanced configuration offers a community-measured palette (data ported from paperlesspaper/epdoptimize, Apache 2.0) as an alternative starting point to the existing idealized DEFAULT_PALETTE_RGB -- fills the existing palette table, doesn't save by itself. Known open risk, not resolved here: a headless Chromium binary is far larger than the ~100MB single-layer limit that already forced this project's pip/npm installs into split layers, and (unlike those) is a single ~180MB file that can't be split across layers by ordinary Dockerfile restructuring. Flagged prominently in server/Dockerfile and docs/widgets.md -- treat this render style as experimental/local-only until that's resolved. |
||
|
tfaour
|
d34eb1bf45 |
Modernize on-panel widget visuals: real typography, theme colors, gutter
Introduces app/panel_style.py, a shared style module every render module now draws through instead of independently duplicating margins/ colors/fonts: Inter Bold/Regular (already vendored, previously only used by widgets/text.py) replace PIL's single-weight bundled default font everywhere else; a per-widget-kind accent color (calendar=blue, tasks=green, weather=black header) replaces plain black-on-white chrome and is centralized in one THEME mapping so a future global theme only needs to touch panel_style.py; a small per-widget gutter separates adjacent widgets without touching grid.py's cell math; header bars, color chips, and the battery icon get rounded corners. Also drops the MUTED gray text color used throughout calendar_render.py and weather_render.py -- a non-palette color that has no close match in the panel's 6-ink palette and dithers into visible speckle once the composited canvas is quantized. Secondary text now reads through size/ weight alone, always exact black. widgets/battery.py and manage_overlay.py's previously-duplicated battery-glyph-drawing code now share one implementation (panel_style. draw_battery_icon). widgets/_shared.py's placeholder image is fixed to use exact palette colors and route through image_pipeline.draw_text, same as everything else -- it was quietly violating both rules already. image_pipeline.draw_widget_border gains an opt-in radius param (default 0, unused by any call site) for a possible future rounded-border setting -- doesn't touch the exact-corner-pixel behavior test_widget_ border.py already pins. Deliberately out of scope: DEFAULT_PALETTE_RGB and the Floyd-Steinberg quantization pipeline are untouched, per the prior reverted measured- palette/OKLab attempt (05b417a/dfe9d701). |
||
|
tfaour
|
dfe9d71971 |
Revert "Quantize with a measured Spectra 6 palette and OKLab-space ordered dithering"
This reverts commit
|
||
|
tfaour
|
05b417a29b |
Quantize with a measured Spectra 6 palette and OKLab-space ordered dithering
DEFAULT_PALETTE_RGB was a guessed approximation of the panel's ink colors (pure sRGB primaries); swap in epdoptimize's measured spectra6 palette instead, which is far more muted/darker, matching how these inks actually look. _quantize now matches against the palette in OKLab space (perceptual distance) instead of PIL's raw-RGB quantize(), with lightness weighted down relative to hue/chroma when selecting the nearest color -- this palette's inks are lit so differently from their sRGB namesakes (muted dark red, bright yellow) that unweighted distance let lightness dominate and mismatch hue (pure red nearest "yellow"). Dithering switched from Floyd-Steinberg error diffusion to a Bayer ordered dither: true error diffusion is an inherently serial per-pixel loop, and doing that in pure Python for a full 800x480 panel took ~1s, blowing past the render-latency budget the "render widgets concurrently" fix (previous commit) exists to protect. The ordered dither finds each pixel's true nearest and second-nearest palette color and mixes between them (via projection onto that segment, not distance ratio) using a tiled Bayer threshold -- fully vectorized, no Python-level pixel loop. |
||
|
tfaour
|
aa4a382c1b |
Add "now displaying" / "up next" preview pair to the frame header
The server now records exactly what was last sent to the device on
every device-facing render (/frame/image, /frame/advance, /frame/back,
and the global hold actions), persisted as Frame.last_displayed_image/
_at and served back via GET /api/frames/{id}/now-displaying. The
header thumbnail is split into that frozen "now displaying" snapshot
and the existing live "up next" re-render, with an arrow between them
-- so editing a layout shows the change immediately on the right while
the left stays exactly what's actually on the panel until the device's
next real wake.
|
||
|
tfaour
|
b15747a604 |
Add per-widget border option (style, thickness, palette color)
A Widget-level property (border_style/border_thickness/border_color_index), not a per-type config field, since every widget type can have one -- drawn once centrally in device.py's _render_widgets before compositing, using an exact panel palette color so it never dithers. Styles: solid, dashed, dotted, and a fancy double-line picture-frame-mat look. Configurable from a shared "Border" card in every widget's gear-icon dialog. |
||
 Thomas Faour
|
35e80c6d1c |
Add a static image widget (PNG/JPEG/GIF/BMP/WEBP/TIFF/PDF upload)
A new widget type showing a single user-uploaded image with no live upstream to poll -- decoded once at upload time (PDF's first page via pypdfium2, BSD-3-Clause/Apache-2.0, no copyleft exposure) into plain RGB PNG bytes, then composed per a crop/stretch/letterbox display mode like the photos widget. |
||
|
tfaour
|
9c8a87e90d |
Add a live preview thumbnail next to the frame name
Small header thumbnail showing exactly what the frame is currently
displaying -- same widget compositor /frame/image uses, handed back as
a plain PNG (image_pipeline.render_panel/render_placeholder gain an
as_png option) instead of packed native-panel bytes. New session-authed
GET /api/frames/{id}/preview exposes it; click-to-refresh plus a slow
60s poll on the frame header so it doesn't hammer Immich/calendar
sources just for a header thumbnail.
|
||
|
tfaour
|
99069ba5fe |
Widget system Phase 3: calendar widgets become size-aware
calendar_render.py's _build_* functions now take a real target box and pick font sizes/margins from three discrete size tiers (nearest pixel-area fit) instead of always laying out at full panel size and resizing after the fact -- a calendar widget placed smaller than the full panel gets an actually-legible layout instead of shrunk text. Month view falls back to agenda below the smallest tier, where 7 columns can no longer stay readable. The old photo-inlay split (inlay_region/_content_region/_paste_inlay) is deleted along with it -- arbitrary widget placement already subsumes what a fixed half-panel split did, and every call site has passed photo_inlay=None since the Phase 2 cutover. Also adds HTTP-level test coverage for GET .../preview/calendar, which had none before this -- it's what caught a stale photo_inlay kwarg left over from the _build signature change that would have TypeError'd on every request. |
||
|
tfaour
|
f48daa71c8 |
Widget system Phase 1: per-type render/action modules
New app/widgets/ package (photos.py, calendar.py, whiteboard.py, plus the WIDGET_TYPES registry) -- the widget-system analogue of routers/device.py's old RENDERERS/ADVANCE_RENDERERS/BACK_RENDERERS, generalized from "one mode owns the whole panel" to "each widget renders into its own region and responds to named button actions." Each module exposes render(db, frame, widget, target_w, target_h) -> Image.Image (never raises -- a widget's own fetch hiccup falls back to a small placeholder rather than taking the whole panel's render down) and an ACTIONS registry for NEXT/BACK button assignment. Supporting changes needed to give the widget modules something to call, all mechanical/behavior-preserving for every existing caller: - image_pipeline.py: render_panel(regions, ...) generalizes render_frame's tail (paste, enhance once, overlay once, quantize once, pack once) from one photo to N regions -- not a restructuring, since calendar mode's photo-inlay feature already pastes a second composed image onto the canvas before that single shared pipeline runs. - photo_queue.py: advance_forced/back_forced/remove_from_rotation/ get_current take an explicit `frame` param now that `cfg` won't always be the Frame itself once photo-queue state moves to PhotoWidgetConfig -- caught a real latent bug while doing this: get_current was reading refresh_interval_s off `cfg`, but that's a frame-level wake-cadence setting, not something that becomes per-widget, so it now reads that off `frame` explicitly instead. - routers/common.py: list_assets/fetch_source_and_faces take album_id/ display_mode directly instead of a whole Frame (both only ever read that one attribute off it); new get_or_refresh_*_for_widget siblings of the existing calendar/weather/tasks/whiteboard cache helpers, read/ writing the new per-widget config tables -- the Frame-scoped originals are untouched and still what routers/device.py's actual dispatch calls until the Phase 2 cutover. 26 new tests (95 total): render_panel size/placement/orientation coverage, and per-widget-type render/action tests (unconfigured -> placeholder, a fetch failure -> placeholder not a crash, actions mutate the right state). Full suite passes; diff-reviewed to confirm device.py's actual RENDERERS dispatch and the old Frame-scoped get_or_refresh_* bodies are unchanged, so this is safe to deploy on its own despite being step 1 of a two-step cutover (see the project plan on why the *next* step, not this one, has to ship atomically). |
||
|
tfaour
|
aa194be09a |
Calendar mode polish batch + Today & Tomorrow view
Responds to post-launch feedback on calendar mode: configurable week-start day for week/month views, crisper non-antialiased text (threshold-masked instead of drawn straight, so Floyd-Steinberg dithering doesn't speckle glyph edges), a color-coded/proportionally filled battery icon on the manage overlay, word-wrapped placeholder text so "Calendar isn't set up yet" no longer clips in portrait, photo inlay support extended from agenda-only to every view, and a fix so manage-overlay face labels reposition correctly when a photo inlay is active (they previously assumed the photo filled the whole canvas). Also adds a fourth calendar view, "Today & Tomorrow" -- a two-day agenda that reuses the same per-day row-layout helper the single-day agenda view already has. |
||
|
tfaour
|
c007acde75 |
Add calendar frame mode + server-side manage overlay (server)
Build and push server image / build-and-push (push) Successful in 42s
calendar_feed.py/calendar_render.py: fetch/merge per-user ICS feeds, render agenda/week/month views. manage_overlay.py: composites the manage-button overlay server-side (QR, battery, location/date, share-QR, face labels), reused by every render mode. device.py/common.py wire both together: mode dispatch for /frame/image+advance+back, and the &manage=1 flag. Plus UI (frame_config.html Calendar card, settings calendar URL field) and the icalendar/recurring-ical-events deps. |
||
|
tfaour
|
f24c3b9c8e |
Gate firmware auto-check behind control+CSRF; skip faces with a null bounding box
Build and push server image / build-and-push (push) Successful in 39s
/api/frames/{id}/firmware/check could silently stage new firmware as a
side effect (the auto-apply path, when firmware_auto_update is on and
a newer release exists) but was gated by require_frame_view instead of
require_frame_control like its sibling firmware routes, and being a
GET, was exempt from the app's CSRF check (which only applies to
non-GET/HEAD/OPTIONS). A linked viewer without control -- or a
cross-site page riding a control-holding victim's session via a plain
GET -- could trigger an unreviewed firmware install. Now POST +
require_frame_control, matching /firmware/apply-latest; the frontend's
two callers (passive poll on page load, "Check now" button) both
already handle a 409 from a non-controller gracefully via the existing
apiError()/control-banner pattern, so this doesn't change UX for a
frame's actual controller.
Separately: Immich has been observed to return a face detection entry
with a null bounding-box field (a still-pending or otherwise
incomplete detection). Both places that do arithmetic on those fields
-- image_pipeline._face_aware_crop_box (crop_faces display mode) and
face_labels.compute_face_labels (manage-menu name labels) -- crashed
with an unhandled TypeError on such an entry, taking down that frame's
whole photo instead of the intended graceful fallback. Both now skip
any face missing a bounding-box field via a shared _has_bounding_box()
check; a face list with zero valid entries already degrades cleanly to
the plain center crop (the existing inf/-inf sentinel math already
handled "no faces" correctly, it just couldn't tell "none passed
Immich" apart from "one broken entry" before).
|
||
|
tfaour
|
9f9ad34a40 |
Fix stray tab whitespace in DEFAULT_PALETTE_RGB
Build and push server image / build-and-push (push) Successful in 41s
|
||
|
tfaour
|
e48ac50ea1 |
Color/contrast/dithering sliders + before/after render preview
Advanced configuration gains three sliders (PIL ImageEnhance factors for color/contrast, 0-2, 1=unchanged; a 0-1 dithering strength) applied to every photo this frame renders. Confirmed the parameter conventions against a similar project (jwchen119/EPF: ImageEnhance.Color/Contrast, 1.0 baseline) before implementing; dithering strength isn't natively exposed by PIL's quantize(), so it's implemented by blending the source toward its own flat/undithered quantization before running Floyd- Steinberg on the blend -- at 0 there's no quantization error left to diffuse (exactly the flat result), at 1 it's the original unmodified behavior, with a smooth continuum between rather than dithering being an on/off toggle. image_pipeline.py split into composition (_compose), enhancement (_enhance), quantization (_quantize), and transpose+pack stages so render_frame (device bytes) and the new render_preview_png (a normal viewable PNG, upright logical orientation) share the same pipeline instead of duplicating it. Named-face overlay label math (face_labels.py) was already routed through the shared _placement_transform, so it needed no changes for the new params. Also added the requested before/after comparison: the Configuration tab's new Preview card shows the current photo's untouched Immich preview next to that same photo run through the frame's actual saved rendering pipeline (two new GET endpoints, /preview/original and /preview/rendered) -- immediate visual feedback for tuning the palette and these new sliders. "Refresh preview" re-fetches after saving. Schema migration v6 adds color_boost/contrast_boost/dither_strength, defaulting to 1.0/1.0/1.0 -- reproduces the exact previous rendering until a frame's Configuration tab changes one. Verified against the live-shaped test database: the migration, sliders persisting and clamping out-of-range input, both preview endpoints (real JPEG passthrough / real PNG at correct logical size+orientation), confirmed dither_strength=0 actually changes the rendered bytes vs. default, and the standing legacy-device curl suite. |
||
|
tfaour
|
83c59af1dd |
Update server/app/image_pipeline.py
Build and push server image / build-and-push (push) Successful in 46s
Fix pallette defaults |
||
|
tfaour
|
49bc9f9ec9 |
Display mode: crop to fill, crop to faces, stretch to fill, shrink to fit
Build and push server image / build-and-push (push) Successful in 40s
Replaces the smart_crop_faces boolean with a 4-way display_mode select on each frame's Configuration tab (image_pipeline.DISPLAY_MODES): - Crop to fill / Crop to faces: the previous False/True behavior, unchanged (center-crop trimming excess, optionally shifted to keep faces on screen). - Stretch to fill (new): fills the panel exactly, aspect ratio not preserved -- a plain resize, no crop. - Shrink to fit (new): the whole photo visible, letterboxed with white where it doesn't fill the panel. Named-face overlay label positioning (face_labels.py, the manage menu's "who's in this photo") now goes through a shared _placement_transform() in image_pipeline.py instead of duplicating crop-box math, so label placement stays correct (and in-bounds) under all four modes, not just the two crop ones -- letterbox/stretch never crop a face out, so labels just use straight scale+offset math there. Schema migration v5 adds display_mode, backfills it from the old boolean (True/False -> crop_faces/crop_fill), and drops the boolean. Verified against the live-shaped test database: the migration (existing frames correctly preserved as crop_faces), the config page's new 4-option select, actual renders under letterbox (confirmed real white letterbox padding in the packed panel-code bytes) and stretch_fill, invalid-input fallback, and the standing legacy-device curl suite. |
||
|
tfaour
|
5b11f2accb |
Per-frame palette calibration + sidebar battery indicator
Build and push server image / build-and-push (push) Successful in 40s
Advanced configuration (Configuration tab, collapsed <details> section):
a color picker per ink color (black/white/yellow/red/blue/green),
overriding image_pipeline.DEFAULT_PALETTE_RGB for that frame's actual
panel -- different units can vary enough from the documented
approximations to be worth calibrating once you can compare a rendered
photo against the real hardware. Stored as Frame.palette_rgb (NULL =
default, schema migration v4), threaded through render_frame/
render_placeholder/_quantize_and_pack (which now builds the PIL palette
image per call instead of once at import) so both photos and the
unclaimed/unconfigured placeholder screen respect it. "Reset to
defaults" clears back to NULL. Config-save validates exactly 6 #rrggbb
values, rejecting anything else with a 400.
Also: each frame's sidebar entry now shows its last-reported battery
percent (🔋NN%) next to the name, using the frame_dot's existing
recently-seen indicator conventions -- silent when never reported
(mains-only frames, or before the first report), matching how battery
is hidden everywhere else it's not applicable.
Verified against the same live-shaped database as the SMTP work: the
v3->v4 migration, save/reload/reset round trip through the real HTTP
route, an actual rendered image using a custom palette (confirmed via
its packed panel-code bytes), input validation, and the sidebar badge
against real battery data -- plus the standing legacy-device curl suite.
|
||
|
tfaour
|
683e3881b1 |
Redesign phase C: claim flow, limited manage page, device protocol
The frame-claiming pipeline, end to end. Firmware: every request now carries ?id=<12-hex STA MAC> via build_url (mirrored in build_ota_url), and the captive portal's success page became a redirect that hands the user's browser to <server>/claim?device_id=... after ~7s -- enough time for the phone to drop the provisioning AP while the device reboots. The server pushes a per-frame device token through /frame/config during a one-time handshake; the firmware persists it to NVS (a dedicated single-key write that deliberately doesn't reset the connected-once flag or WiFi cache) and prefers it over the provisioned shared token from the next request on. Config response buffer grows 256->512. Both board variants compile clean; new firmware also works against an old server (which ignores ?id=) and old firmware against this server (the phase A legacy mapping), so either deploy order survives. Server: /claim lands the captive-portal redirect -- claim-gated signup (a valid unclaimed/unregistered device id IS the enrollment invitation), pending claims for the user-beats-the-frame race (auto-attached at self-registration, 24h expiry), and a waiting page that refreshes until the frame checks in. Unclaimed/unconfigured frames get a rendered instruction placeholder with a QR from /frame/image (200, never an error loop) -- new qrcode dep, placeholder shares the exact quantize/pack path photos use. The on-frame manage QR now resolves to a limited no-login page: scans of / carrying device credentials (new ?id&token or the legacy shared token) 303 to /m/<manage_token>, which allows exactly view queue, show-next, advance, back, and scoped thumbnails -- no settings, no removal, no other frames. Full control means logging in. One real protocol hole found by simulating full wake cycles: after self-registration the device could never authenticate again (the wake cycle fetches the image BEFORE /frame/config delivers its token). require_device now treats the id itself as the credential until the first authenticated request flips device_token_ack -- the same trust level as open registration, closing permanently once the handshake completes. |
||
|
tfaour
|
4cf75b4b11 |
Update server/app/image_pipeline.py
Build and push server image / build-and-push (push) Successful in 38s
Tweaked Palette... we might need to reverse this! |
||
|
tfaour
|
015993af00 |
Add battery level reporting (Kconfig-gated) and display orientation
Build and push server image / build-and-push (push) Successful in 32s
Battery (firmware + server, disabled by default): new battery.c reads a 2x200k voltage divider via ADC oneshot with curve-fitting calibration (the ESP32-C6's scheme), maps through a piecewise LiPo discharge curve, and restores the pin to button duty after each read -- the settled XIAO ESP32-C6 design shares the back button's GPIO0/A0, time-shared per wake. Skipped entirely when on mains (a 2x100k VBUS divider into a spare digital pin -- the 5V pin is dead on battery power, so presence = mains, where the charging voltage would read misleadingly full) or when the reading is implausible. The manage overlay gains a battery region (static outline glyph + "NN%", below the manage QR, all menu levels), and the device POSTs to the new /frame/battery endpoint after a successful fetch; the server stores percent + as-of timestamp, exposed via /api/queue and shown in the web UI. FRAME_BATTERY_ADC_GPIO / FRAME_VBUS_SENSE_GPIO default to -1 (fully inert on the dev board); compile-verified both disabled and enabled, hardware bring-up deferred until the ordered XIAO + batteries arrive. Orientation (server-side only): new config setting + web UI dropdown (landscape / portrait / landscape_flipped / portrait_flipped). Photos are composed/cropped at the logical hanging shape (portrait crops at 480x800, so face-aware crops match how the frame actually hangs), then rotated losslessly into the panel's native 800x480 byte layout after dithering -- the device never knows. Face-label anchors are transformed through the same rotation (logical_to_native()) so they stay attached to faces on rotated frames. Known documented limitation: the on-device manage overlay still renders in native orientation, so it appears sideways on a portrait-hung frame (QRs scan at any rotation; text reads sideways). |
||
|
tfaour
|
e870898490 |
Refine manage overlay: US/CAN state abbreviations, share-QR caption, and an escalating second menu with named-face labels
Build and push server image / build-and-push (push) Successful in 32s
Two rounds of follow-up work on the manage-button overlay:
1. Location formatting: US/Canada now show abbreviated state/province
("CA", "ON") instead of the full name, other countries show the full
country name, and each is its own line (was one line, now wraps to
two) so longer international place names have more room without
threatening to overlap the top-right QR box. The bottom-left share QR
also gets a "SCAN TO DOWNLOAD" caption.
2. Escalating menu: pressing the manage button again while its overlay
is already up adds a second level -- each Immich-identified person's
name labeled next to their face in the photo (using Immich's own
face recognition/People data, no detection/recognition added to this
project). A third press exits immediately instead of waiting out the
30s auto-revert timer. No new Immich API needed -- GET /api/faces
already embeds a nullable person.name per face; new
server/app/face_labels.py maps a named face's box into the final
800x480 frame's pixel space (reusing crop-box math extracted from
image_pipeline.py's face-aware cropping). Capped at 4 named faces,
sized to a real firmware RAM budget: each label is its own malloc'd
overlay region on the device, alongside the 4 fixed corner regions
already in use. New GET /frame/face-labels returns a flattened
fixed-slot JSON shape (not a real array) so firmware's existing
flat-scalar parser can read it without needing an actual array
parser. No persistent state needed for the escalation itself -- it's
all local control flow within one continuous awake session
(frame_client.c's run_management_menu()).
|
||
|
tfaour
|
f2a374b363 |
Make face-aware crop minimal-shift instead of full re-centering
Build and push server image / build-and-push (push) Failing after 10s
_face_aware_crop_box() previously always centered the crop on the union of all detected faces' centroid, even when the plain center-crop already kept every face fully on screen -- unnecessarily moving a composition that didn't need fixing. Now starts from the plain center-crop and only shifts it the minimum amount needed to bring an otherwise-cropped-out face back into frame; already-fine framing is left untouched (falls back to centering on the faces' midpoint only if they're spread too wide for any single shift to contain them all, which is unchanged from before). Verified: a face safely inside the plain center-crop now produces byte- identical output to the no-shift case (previously it still would have been re-centered); an edge face gets a 100px shift instead of the 1050px a full re-center would have applied. Re-ran against the real 4-face test photo from earlier -- all four were already fully visible, so the refined box now exactly matches the plain center-crop instead of shifting unnecessarily. |
||
|
tfaour
|
6e353c2271 |
Add server-configurable refresh interval + face-aware cropping
Build and push server image / build-and-push (push) Failing after 10s
Two features, both toggleable/settable from the web config UI:
Refresh interval: new GET /frame/config returns
{"refresh_interval_s": ...} as plain JSON. Reuses the endpoint the frame
already needs to hit for a reachability check each wake cycle (previously
/health) rather than adding a third round trip, and always returns 200
with current settings regardless of Immich-configured state so it stays
valid as a pure reachability signal. Clamped to [60, 86400] seconds in
POST /api/config.
Face-aware cropping: GET /api/faces?id={assetId} on Immich already
returns real per-photo face bounding boxes from its own People-feature
ML -- confirmed against a live instance, boxes scaled to the asset's
native resolution. No face detection built or bundled here at all, just
an API call plus rectangle math. image_pipeline.render_frame() gains an
optional `faces` param: when present, computes the largest crop window
matching the panel's aspect ratio that fits in the source image, centered
on the union of all face boxes' centroid (scaled into the downloaded
preview's actual resolution) instead of the image's geometric center,
clamped to stay within bounds. No faces (or the smart_crop_faces config
toggle off) falls straight back to the existing ImageOps.fit() center-crop
-- zero behavior change in that case. A faces-lookup failure logs and
degrades to center-crop rather than failing the whole request.
Verified: unit tests for the crop-box math (horizontal shift toward an
off-center face, edge clamping), a full mock-Immich end-to-end pass
(extended to serve /faces) confirming the toggle changes output and the
response is still exactly 192,000 bytes, and a live comparison against a
real 4-face photo on the user's Immich instance (crop top shifted from
528px to 246px toward the detected faces).
|
||
|
tfaour
|
5e7c47e2eb |
Add FastAPI server: pulls from Immich, pre-processes for the panel
Implements the server side of the architecture decided on: the ESP32-C6 has no PSRAM and a tight RAM budget, so all the heavy lifting (JPEG decode, resize, Floyd-Steinberg dithering, 6-color quantization, 4bpp packing) happens here instead of on-device. The frame just does a single GET and streams the response straight to SPI. - GET /frame/image: looks up the current cursor's asset in the configured Immich album, downloads its preview thumbnail, and returns it packed into the panel's exact 800x480/4bpp/2px-per-byte format (application/octet-stream, always exactly 192,000 bytes). - GET / + POST /api/config + GET /api/albums: a small web UI for entering the Immich URL/API key and picking an album, rather than cramming that into the ESP32's captive portal form. - Config (Immich creds, selected album, cursor) persists to a JSON file via a docker-compose volume mount. Verified locally with a venv (Docker isn't available in this environment): unit-tested image_pipeline against a synthetic image (exact byte count, valid panel color codes only), and ran a full end-to-end pass against a mock Immich HTTP server exercising the real /frame/image path. Pinned dependency versions in requirements.txt after hitting a real bug with unpinned floors: the latest starlette (1.3.1) resolved by `pip install fastapi` breaks Jinja2Templates outright. Not yet wired to the ESP32 side (task 6) or authenticated -- /frame/image is unauthenticated for now, fine on a trusted LAN but worth revisiting once the firmware sends a shared device token. |