espresso_frame/docs/widgets.md

Widget system

A frame's panel isn't one fixed "mode" anymore -- it holds N independently placed/sized widgets (photos/calendar/whiteboard), like arranging icons on an Android home screen. A frame can hold several widgets of the same type (e.g. two photo widgets pointed at different Immich albums side by side). This replaced an earlier design where Frame.mode picked exactly one full-panel renderer; that column (and the other now-dead per-mode Frame columns it left behind -- album_id, calendar_*, whiteboard_*, etc.) is still physically present but unused, pending a final cleanup migration (see "Known gaps" below).

The device-facing contract is unchanged by any of this: GET /frame/image, POST /frame/advance, POST /frame/back are the same frozen paths firmware has always called (see docs/architecture.md) -- what changed is entirely server-side, in how those endpoints decide what to render and what a button press does.

Data model

• Widget (server/app/models.py): id, frame_id, widget_type ("photos" | "calendar" | "whiteboard"), x/y/w/h (grid cells), sort_order. Widgets never overlap (enforced server-side in routers/api_widgets.py, re-validated regardless of what the client already checked) -- that's what keeps compositing simple: no z-order, no blending, just N independent regions pasted onto one shared canvas.
• Per-type 1:1 extension tables -- PhotoWidgetConfig, CalendarWidgetConfig, WhiteboardWidgetConfig, each keyed by widget_id with ondelete="CASCADE" -- rather than one wide table with every type's mostly-irrelevant columns. PhotoWidgetConfig mirrors app/photo_queue.py's attribute names exactly, so that module's advance/back/queue logic ports across widget instances unchanged.
• FrameCalendar is keyed by widget_id (not frame_id) since a frame can now have more than one independent calendar widget, each with its own set of included calendars.
• FrameButtonAction (id, frame_id, button ["next"|"back"], widget_id, action, sort_order) -- see "Button actions" below.

Placement: a grid, not freeform pixels

app/grid.py is pure grid math, no I/O. The grid is GRID_LONG=8 x GRID_SHORT=5 cells, defined relative to the panel's long/short axis (not "landscape" specifically) so it stays valid across image_pipeline.logical_render_size(orientation)'s genuine width/height swap for portrait -- landscape orientations are 8 cols x 5 rows, portrait are 5 cols x 8 rows, same cell size either way. Changing a frame's orientation invalidates its existing layout (an 8x5 arrangement isn't valid on a 5x8 grid) -- the server resets to one full-panel widget on an orientation change rather than trying to remap coordinates.

Each widget type has a minimum grid footprint (grid.MIN_FOOTPRINT): photos 1x1, calendar 3x2 (a crammed calendar is illegible regardless of size-tier scaling), whiteboard 2x2. Enforced both client-side (UX, in the Layout tab's drag/resize canvas -- static/frame_layout.js) and server-side (routers/api_widgets.py) -- the client is never trusted alone.

Rendering: one shared compositor

app/widgets/ is the render/action registry -- one module per widget_type (photos.py, calendar.py, whiteboard.py), each exposing:

• render(db, frame, widget, target_w, target_h, is_normal_wake) -> Image: an unquantized RGB image exactly target_w x target_h, the widget's content composed into its own region. Never raises for a foreseeable failure (an Immich hiccup, an unconfigured widget) -- falls back to a small placeholder within its own region instead, so one widget having a bad moment doesn't blank the whole panel.
• ACTIONS: dict[str, Callable] -- named button actions this type supports ("advance"/"back" for photos and calendar, "check_now" for whiteboard).
• ACTION_LABELS: dict[str, str] -- human labels for the button- assignment UI.

routers/device.py's _render_widgets loads every Widget row for the frame, maps each one's grid rect to pixels (grid.cell_to_pixels), calls its module's render(), and hands the whole list of (rect, image) regions to image_pipeline.render_panel -- which pastes every region onto one shared canvas, then runs enhance/manage-overlay/quantize/dither/pack once over the composited result. Quantizing the whole canvas together (not each region separately before pasting) is what keeps the 6-color e-ink dithering pattern consistent across a widget boundary instead of a visible seam at the edge.

Calendar widgets pick from discrete size tiers (calendar_render.py's _SIZE_TIERS) for font size/margins/row heights based on their actual grid footprint, rather than continuously scaling constants tuned for a full ~800x480 canvas -- falls back to agenda view if a widget is too small for month view to stay legible.

Button actions

Each physical button (NEXT/BACK) maps to an ordered list of (widget, action) bindings, not a fixed meaning -- e.g. NEXT can be "photo widget A: advance" and "calendar widget B: advance" together, or even a mismatched combination on purpose. On a press, routers/device.py's _run_button_actions runs every assigned action for that button in order (each in its own widget_locked span -- never nested, since the underlying per-frame lock isn't reentrant), catching and logging any single action's failure without blocking the rest, then re-renders and returns the whole composed panel once at the end regardless of which actions succeeded.

The web UI for this is the "Button assignments" card on a frame's Configuration tab (static/frame_config.js, GET/PUT /api/frames/{id}/buttons) -- add/remove/reorder, autosaved. Two widgets of the same type would otherwise both just say "Photos" in the assignment dropdowns; the UI disambiguates using each widget's grid position (e.g. "Photos 1 (left)" / "Photos 2 (right)"), the same way you'd tell them apart by eye on the Layout canvas.

A newly-created widget (including the one auto-migrated from a frame's old mode on upgrade) gets a sensible default binding reproducing its old button behavior -- see migration.py's _default_button_actions.

Per-widget config UI

Each widget has a gear-icon button on the Layout canvas that opens a <dialog> with that widget's own settings (album, calendar inclusion, whiteboard source, etc.) -- not a per-frame tab, since a frame can now have several widgets of the same type with independent settings. The dialog HTML is injected server-rendered (routers/frame_pages.py's widget_dialog, dispatching on widget.widget_type); its JS is a top-level, always-loaded file (static/widget_dialog_*.js) exposing init<Type>Dialog()/close<Type>Dialog(), since dynamically-injected HTML can't carry executable <script> tags. While a dialog is open, window.FRAME_API is repointed at that widget's own API base (/api/frames/{id}/widgets/{widget_id}) and restored on close; window.FRAME_BASE_API stays pointed at the frame-level base throughout for the always-present header/status-bar JS.

Known gaps (Phase 6, not yet done)

The original 8-phase rollout plan's last phase is still open:

• Legacy per-mode Frame columns (mode, album_id, current_asset_id, all calendar_*, all whiteboard_*, etc.) are still physically present in the schema but no longer read or written anywhere -- they need a dedicated final migration to drop them. Left in place deliberately through the widget-system rollout (a much larger blast radius cutover than this project's usual same-migration-drop convention) but there's no reason to keep carrying them now that every phase has shipped.
• server/README.md still describes photos/calendar/whiteboard as per-frame "modes" in several places rather than widgets -- needs a pass once the column drop above is safely deployed.
• Whiteboard rendering is tagged (alpha) in the UI -- not fully reliable yet, treat it as experimental if extending it.