espresso_frame/docs/widgets.md

# Widget system

A frame's panel isn't one fixed "mode" anymore -- it holds N independently
placed/sized widgets (photos/calendar/whiteboard), like arranging icons on
an Android home screen. A frame can hold several widgets of the same type
(e.g. two photo widgets pointed at different Immich albums side by side).
This replaced an earlier design where `Frame.mode` picked exactly one
full-panel renderer; that column (and the other now-dead per-mode `Frame`
columns it left behind -- `album_id`, `calendar_*`, `whiteboard_*`, etc.)
is still physically present but unused, pending a final cleanup migration
(see "Known gaps" below).

The device-facing contract is unchanged by any of this: `GET /frame/image`,
`POST /frame/advance`, `POST /frame/back` are the same frozen paths
firmware has always called (see `docs/architecture.md`) -- what changed is
entirely server-side, in how those endpoints decide what to render and what
a button press does.

## Data model

- `Widget` (`server/app/models.py`): `id`, `frame_id`, `widget_type`
  (`"photos"` | `"calendar"` | `"whiteboard"`), `x`/`y`/`w`/`h` (grid
  cells), `sort_order`. Widgets never overlap (enforced server-side in
  `routers/api_widgets.py`, re-validated regardless of what the client
  already checked) -- that's what keeps compositing simple: no z-order,
  no blending, just N independent regions pasted onto one shared canvas.
- Per-type 1:1 extension tables -- `PhotoWidgetConfig`,
  `CalendarWidgetConfig`, `WhiteboardWidgetConfig`, each keyed by
  `widget_id` with `ondelete="CASCADE"` -- rather than one wide table with
  every type's mostly-irrelevant columns. `PhotoWidgetConfig` mirrors
  `app/photo_queue.py`'s attribute names exactly, so that module's
  advance/back/queue logic ports across widget instances unchanged.
- `FrameCalendar` is keyed by `widget_id` (not `frame_id`) since a frame
  can now have more than one independent calendar widget, each with its
  own set of included calendars.
- `FrameButtonAction` (`id`, `frame_id`, `button` [`"next"`|`"back"`],
  `widget_id`, `action`, `sort_order`) -- see "Button actions" below.

## Placement: a grid, not freeform pixels

`app/grid.py` is pure grid math, no I/O. The grid is `GRID_LONG=8` x
`GRID_SHORT=5` cells, defined relative to the panel's long/short axis
(not "landscape" specifically) so it stays valid across
`image_pipeline.logical_render_size(orientation)`'s genuine width/height
swap for portrait -- landscape orientations are 8 cols x 5 rows, portrait
are 5 cols x 8 rows, same cell size either way. **Changing a frame's
orientation invalidates its existing layout** (an 8x5 arrangement isn't
valid on a 5x8 grid) -- the server resets to one full-panel widget on an
orientation change rather than trying to remap coordinates.

Each widget type has a minimum grid footprint (`grid.MIN_FOOTPRINT`):
photos 1x1, calendar 3x2 (a crammed calendar is illegible regardless of
size-tier scaling), whiteboard 2x2. Enforced both client-side (UX, in the
Layout tab's drag/resize canvas -- `static/frame_layout.js`) and
server-side (`routers/api_widgets.py`) -- the client is never trusted
alone.

## Rendering: one shared compositor

`app/widgets/` is the render/action registry -- one module per
`widget_type` (`photos.py`, `calendar.py`, `whiteboard.py`), each
exposing:

- `render(db, frame, widget, target_w, target_h, is_normal_wake) -> Image`:
  an unquantized RGB image exactly `target_w x target_h`, the widget's
  content composed into its own region. Never raises for a foreseeable
  failure (an Immich hiccup, an unconfigured widget) -- falls back to a
  small placeholder within its own region instead, so one widget having a
  bad moment doesn't blank the whole panel.
- `ACTIONS: dict[str, Callable]` -- named button actions this type
  supports (`"advance"`/`"back"` for photos and calendar, `"check_now"`
  for whiteboard).
- `ACTION_LABELS: dict[str, str]` -- human labels for the button-
  assignment UI.

`routers/device.py`'s `_render_widgets` loads every `Widget` row for the
frame, maps each one's grid rect to pixels (`grid.cell_to_pixels`), calls
its module's `render()`, and hands the whole list of `(rect, image)`
regions to `image_pipeline.render_panel` -- which pastes every region onto
one shared canvas, then runs enhance/manage-overlay/quantize/dither/pack
**once** over the composited result. Quantizing the whole canvas together
(not each region separately before pasting) is what keeps the 6-color
e-ink dithering pattern consistent across a widget boundary instead of a
visible seam at the edge.

Calendar widgets pick from discrete size tiers (`calendar_render.py`'s
`_SIZE_TIERS`) for font size/margins/row heights based on their actual
grid footprint, rather than continuously scaling constants tuned for a
full ~800x480 canvas -- falls back to agenda view if a widget is too small
for month view to stay legible.

## Button actions

Each physical button (NEXT/BACK) maps to an **ordered list** of
`(widget, action)` bindings, not a fixed meaning -- e.g. NEXT can be
"photo widget A: advance" *and* "calendar widget B: advance" together, or
even a mismatched combination on purpose. On a press,
`routers/device.py`'s `_run_button_actions` runs every assigned action for
that button in order (each in its own `widget_locked` span -- never nested,
since the underlying per-frame lock isn't reentrant), catching and
logging any single action's failure without blocking the rest, then
re-renders and returns the whole composed panel once at the end regardless
of which actions succeeded.

The web UI for this is the "Button assignments" card on a frame's
Configuration tab (`static/frame_config.js`, `GET`/`PUT
/api/frames/{id}/buttons`) -- add/remove/reorder, autosaved. Two widgets of
the same type would otherwise both just say "Photos" in the assignment
dropdowns; the UI disambiguates using each widget's grid position (e.g.
"Photos 1 (left)" / "Photos 2 (right)"), the same way you'd tell them
apart by eye on the Layout canvas.

A newly-created widget (including the one auto-migrated from a frame's old
`mode` on upgrade) gets a sensible default binding reproducing its old
button behavior -- see `migration.py`'s `_default_button_actions`.

## Per-widget config UI

Each widget has a gear-icon button on the Layout canvas that opens a
`<dialog>` with that widget's own settings (album, calendar inclusion,
whiteboard source, etc.) -- not a per-frame tab, since a frame can now
have several widgets of the same type with independent settings. The
dialog HTML is injected server-rendered (`routers/frame_pages.py`'s
`widget_dialog`, dispatching on `widget.widget_type`); its JS is a
top-level, always-loaded file (`static/widget_dialog_*.js`) exposing
`init<Type>Dialog()`/`close<Type>Dialog()`, since dynamically-injected
HTML can't carry executable `<script>` tags. While a dialog is open,
`window.FRAME_API` is repointed at that widget's own API base
(`/api/frames/{id}/widgets/{widget_id}`) and restored on close;
`window.FRAME_BASE_API` stays pointed at the frame-level base throughout
for the always-present header/status-bar JS.

## Known gaps (Phase 6, not yet done)

The original 8-phase rollout plan's last phase is still open:

- Legacy per-mode `Frame` columns (`mode`, `album_id`,
  `current_asset_id`, all `calendar_*`, all `whiteboard_*`, etc.) are
  still physically present in the schema but no longer read or written
  anywhere -- they need a dedicated final migration to drop them. Left in
  place deliberately through the widget-system rollout (a much larger
  blast radius cutover than this project's usual same-migration-drop
  convention) but there's no reason to keep carrying them now that every
  phase has shipped.
- `server/README.md` still describes photos/calendar/whiteboard as
  per-frame "modes" in several places rather than widgets -- needs a pass
  once the column drop above is safely deployed.
- Whiteboard rendering is tagged **(alpha)** in the UI -- not fully
  reliable yet, treat it as experimental if extending it.