tfaour
52ebafab78
New widget type with four display modes -- current conditions, an hourly forecast strip, a multi-day forecast, and several cities' current day side by side -- backed by a pluggable provider registry (app/weather/, mirroring the app/widgets/ dispatch pattern): Open-Meteo (worldwide) and NWS (US-only) both wired up now, Environment Canada documented as the next one to add given its more involved station/grid-lookup API. The calendar widget's existing embedded weather strip is untouched and still Open-Meteo-only; this lifts the same underlying icon-drawing primitives (now shared via app/weather_render.py, calendar_render.py still imports draw_weather_row unchanged) into a widget that can be placed and sized on its own. Icons are redrawn in the panel's actual ink colors (yellow sun/bolt, blue rain/snow) instead of flat black, and build_multi_city's icon/font sizing now scales with how many cities need to fit rather than the box's height alone -- both fixed after catching them via live browser verification, along with a mode-switch cache-shape crash and a mobile-width dialog overflow. New WeatherWidgetConfig table (migration 24), grid footprint, widget module, common.py fetch/cache helper, router endpoints (location set/ clear, city add/remove, preview), dialog template + JS, and full test coverage (providers, widget render, HTTP endpoints, migration replay). docs/widgets.md and CLAUDE.md's TODO updated accordingly.
271 lines
15 KiB
Markdown
271 lines
15 KiB
Markdown
# Widget system
|
|
|
|
A frame's panel isn't one fixed "mode" anymore -- it holds N independently
|
|
placed/sized widgets (photos/calendar/whiteboard/tasks/static image/text/
|
|
weather), like arranging icons on an Android home screen. A frame can hold
|
|
several widgets of the same type (e.g. two photo widgets pointed at
|
|
different Immich albums side by side).
|
|
This replaced an earlier design where `Frame.mode` picked exactly one
|
|
full-panel renderer; that column (and the other now-dead per-mode `Frame`
|
|
columns it left behind -- `album_id`, `calendar_*`, `whiteboard_*`, etc.)
|
|
is still physically present but unused, pending a final cleanup migration
|
|
(see "Known gaps" below).
|
|
|
|
The device-facing contract is unchanged by any of this: `GET /frame/image`,
|
|
`POST /frame/advance`, `POST /frame/back` are the same frozen paths
|
|
firmware has always called (see `docs/architecture.md`) -- what changed is
|
|
entirely server-side, in how those endpoints decide what to render and what
|
|
a button press does.
|
|
|
|
## Data model
|
|
|
|
- `Widget` (`server/app/models.py`): `id`, `frame_id`, `widget_type`
|
|
(`"photos"` | `"calendar"` | `"whiteboard"` | `"tasks"` | `"static"` |
|
|
`"text"` | `"weather"`), `x`/`y`/`w`/`h`
|
|
(grid cells), `sort_order`. Widgets never overlap (enforced server-side in
|
|
`routers/api_widgets.py`, re-validated regardless of what the client
|
|
already checked) -- that's what keeps compositing simple: no z-order,
|
|
no blending, just N independent regions pasted onto one shared canvas.
|
|
- Per-type 1:1 extension tables -- `PhotoWidgetConfig`,
|
|
`CalendarWidgetConfig`, `WhiteboardWidgetConfig`, `TaskWidgetConfig`,
|
|
`StaticWidgetConfig`, `TextWidgetConfig`, `WeatherWidgetConfig`, each
|
|
keyed by `widget_id` with
|
|
`ondelete="CASCADE"` -- rather than one wide table with every type's
|
|
mostly-irrelevant columns. `TextWidgetConfig.content` is parsed rich
|
|
text (paragraphs of styled runs), never raw HTML -- see
|
|
`server/app/text_content.py`'s module docstring for why that parse
|
|
step is the widget's actual stored-XSS sanitization boundary.
|
|
`PhotoWidgetConfig`
|
|
mirrors `app/photo_queue.py`'s attribute names exactly, so that module's
|
|
advance/back/queue logic ports across widget instances unchanged.
|
|
`TaskWidgetConfig` used to be a handful of `tasks_*` columns bolted onto
|
|
`CalendarWidgetConfig` (a week-view-only, single-list task list); split
|
|
into its own widget type (migration 17) so a task list can be placed
|
|
and sized independent of any calendar's view/footprint, then (migration
|
|
18) given the same multi-source shape a calendar widget already has.
|
|
`WeatherWidgetConfig` similarly lifts `CalendarWidgetConfig`'s embedded
|
|
weather strip (still present and unchanged, `weather_*` columns) out
|
|
into its own placeable widget type (migration 24) -- see "Weather
|
|
widget" below.
|
|
- `FrameCalendar`/`FrameTaskList` are keyed by `widget_id` (not
|
|
`frame_id`) since a frame can now have more than one independent
|
|
calendar/tasks widget, each with its own included set. Identical
|
|
shape and permission model (owner-added, anyone-linked-can-mute, see
|
|
"Per-widget config UI" below) -- `FrameTaskList` just has no `"ics"`
|
|
calendar_key variant, since a plain ICS subscription has no VTODO
|
|
(task) collection.
|
|
- `FrameButtonAction` (`id`, `frame_id`, `button` [`"next"`|`"back"`],
|
|
`widget_id`, `action`, `sort_order`) -- see "Button actions" below.
|
|
|
|
## Placement: a grid, not freeform pixels
|
|
|
|
`app/grid.py` is pure grid math, no I/O. The grid is `GRID_LONG=8` x
|
|
`GRID_SHORT=5` cells, defined relative to the panel's long/short axis
|
|
(not "landscape" specifically) so it stays valid across
|
|
`image_pipeline.logical_render_size(orientation)`'s genuine width/height
|
|
swap for portrait -- landscape orientations are 8 cols x 5 rows, portrait
|
|
are 5 cols x 8 rows, same cell size either way. **Changing a frame's
|
|
orientation invalidates its existing layout** (an 8x5 arrangement isn't
|
|
valid on a 5x8 grid) -- the server resets to one full-panel widget on an
|
|
orientation change rather than trying to remap coordinates.
|
|
|
|
Each widget type has a minimum grid footprint (`grid.MIN_FOOTPRINT`):
|
|
photos 1x1, calendar 3x2 (a crammed calendar is illegible regardless of
|
|
size-tier scaling), whiteboard 2x2, tasks 2x2, static image 1x1, text 2x1,
|
|
weather 2x2 (its hourly/daily strips need the room; current/multi_city
|
|
modes would tolerate smaller, but every mode shares one footprint value).
|
|
Enforced both client-side
|
|
(UX, in the Layout tab's drag/resize canvas -- `static/frame_layout.js`)
|
|
and server-side (`routers/api_widgets.py`) -- the client is never trusted
|
|
alone.
|
|
|
|
## Rendering: one shared compositor
|
|
|
|
`app/widgets/` is the render/action registry -- one module per
|
|
`widget_type` (`photos.py`, `calendar.py`, `whiteboard.py`, `tasks.py`,
|
|
`static_image.py`, `text.py`, `weather.py`), each exposing:
|
|
|
|
- `render(db, frame, widget, target_w, target_h, is_normal_wake) -> Image`:
|
|
an unquantized RGB image exactly `target_w x target_h`, the widget's
|
|
content composed into its own region. Never raises for a foreseeable
|
|
failure (an Immich hiccup, an unconfigured widget) -- falls back to a
|
|
small placeholder within its own region instead, so one widget having a
|
|
bad moment doesn't blank the whole panel.
|
|
- `ACTIONS: dict[str, Callable]` -- named button actions this type
|
|
supports (`"advance"`/`"back"` for photos and calendar, `"check_now"`
|
|
for whiteboard and weather -- both throttled external fetches with a
|
|
forced-refetch action). Empty for tasks, static image, and text --
|
|
nothing to advance/back/force for a passive checklist, a fixed
|
|
uploaded image, or a fixed block of authored text.
|
|
- `ACTION_LABELS: dict[str, str]` -- human labels for the button-
|
|
assignment UI.
|
|
|
|
`routers/device.py`'s `_render_widgets` loads every `Widget` row for the
|
|
frame, maps each one's grid rect to pixels (`grid.cell_to_pixels`), calls
|
|
its module's `render()`, and hands the whole list of `(rect, image)`
|
|
regions to `image_pipeline.render_panel` -- which pastes every region onto
|
|
one shared canvas, then runs enhance/manage-overlay/quantize/dither/pack
|
|
**once** over the composited result. Quantizing the whole canvas together
|
|
(not each region separately before pasting) is what keeps the 6-color
|
|
e-ink dithering pattern consistent across a widget boundary instead of a
|
|
visible seam at the edge.
|
|
|
|
Calendar widgets pick from discrete size tiers (`calendar_render.py`'s
|
|
`_SIZE_TIERS`) for font size/margins/row heights based on their actual
|
|
grid footprint, rather than continuously scaling constants tuned for a
|
|
full ~800x480 canvas -- falls back to agenda view if a widget is too small
|
|
for month view to stay legible.
|
|
|
|
## Button actions
|
|
|
|
Each physical button (NEXT/BACK) maps to an **ordered list** of
|
|
`(widget, action)` bindings, not a fixed meaning -- e.g. NEXT can be
|
|
"photo widget A: advance" *and* "calendar widget B: advance" together, or
|
|
even a mismatched combination on purpose. On a press,
|
|
`routers/device.py`'s `_run_button_actions` runs every assigned action for
|
|
that button in order (each in its own `widget_locked` span -- never nested,
|
|
since the underlying per-frame lock isn't reentrant), catching and
|
|
logging any single action's failure without blocking the rest, then
|
|
re-renders and returns the whole composed panel once at the end regardless
|
|
of which actions succeeded.
|
|
|
|
The web UI for this is the "Button assignments" card on a frame's
|
|
Configuration tab (`static/frame_config.js`, `GET`/`PUT
|
|
/api/frames/{id}/buttons`) -- add/remove/reorder, autosaved. Two widgets of
|
|
the same type would otherwise both just say "Photos" in the assignment
|
|
dropdowns; the UI disambiguates using each widget's grid position (e.g.
|
|
"Photos 1 (left)" / "Photos 2 (right)"), the same way you'd tell them
|
|
apart by eye on the Layout canvas.
|
|
|
|
A newly-created widget (including the one auto-migrated from a frame's old
|
|
`mode` on upgrade) gets a sensible default binding reproducing its old
|
|
button behavior -- see `migration.py`'s `_default_button_actions`.
|
|
|
|
## Per-widget config UI
|
|
|
|
Each widget has a gear-icon button on the Layout canvas that opens a
|
|
`<dialog>` with that widget's own settings (album, calendar/task-list
|
|
inclusion, whiteboard source, etc.) -- not a per-frame tab, since a
|
|
frame can now have several widgets of the same type with independent
|
|
settings. The
|
|
dialog HTML is injected server-rendered (`routers/frame_pages.py`'s
|
|
`widget_dialog`, dispatching on `widget.widget_type`); its JS is a
|
|
top-level, always-loaded file (`static/widget_dialog_*.js`) exposing
|
|
`init<Type>Dialog()`/`close<Type>Dialog()`, since dynamically-injected
|
|
HTML can't carry executable `<script>` tags. While a dialog is open,
|
|
`window.FRAME_API` is repointed at that widget's own API base
|
|
(`/api/frames/{id}/widgets/{widget_id}`) and restored on close;
|
|
`window.FRAME_BASE_API` stays pointed at the frame-level base throughout
|
|
for the always-present header/status-bar JS.
|
|
|
|
## Saved layouts
|
|
|
|
A user can snapshot a frame's whole widget arrangement -- every widget's
|
|
type/placement/settings, calendar/task sources, and button-action
|
|
bindings -- under a name (`SavedLayout` + `SavedLayoutWidget` +
|
|
`SavedLayoutSource` + `SavedLayoutButtonAction`, `server/app/models.py`),
|
|
then switch back to it later, or apply it to a *different* frame. Saved
|
|
layouts are owned by the **user**, not any one frame -- the same set
|
|
shows up (with a per-frame `compatible` flag) on every frame that user
|
|
controls whose grid matches (`grid.grid_dims(orientation)`'s cols/rows,
|
|
landscape-class 8x5 vs. portrait-class 5x8), not just the frame it was
|
|
captured from.
|
|
|
|
Saving only captures an authored *setting*, never runtime/cache state --
|
|
a photo widget's current queue position, a calendar's fetch cache, a
|
|
whiteboard's rendered-image cache, etc. are deliberately left out (see
|
|
`routers/api_layouts.py`'s `LAYOUT_CONFIG_FIELDS` allowlist per
|
|
`widget_type`), so applying a layout feels like a fresh widget of that
|
|
type with its settings pre-filled, not a resurrection of stale state
|
|
from whenever it was saved. A static-image widget's uploaded bytes are
|
|
the one exception carried through verbatim (`SavedLayoutWidget.image`).
|
|
Saving again under a name the user already has overwrites that layout's
|
|
snapshot in place rather than erroring or creating a duplicate --
|
|
`SavedLayout`'s own docstring.
|
|
|
|
Applying a layout to a frame (`api_layout_apply`, `require_frame_control`)
|
|
deletes every widget currently on that frame and recreates the saved
|
|
arrangement from scratch, remapping calendar/task sources and button
|
|
bindings onto the newly-created widget ids -- same "act unconditionally
|
|
on the server, confirm on the client" posture as the Layout tab's own
|
|
"Clear all". A source whose owning user account no longer exists is
|
|
silently dropped rather than left dangling (config is JSON, not
|
|
FK-checked, so nothing else would catch that).
|
|
|
|
The web UI lives in the Layout tab's "Saved layouts" card
|
|
(`static/saved_layouts.js`, `GET`/`POST /api/frames/{id}/layouts`,
|
|
`PATCH`/`DELETE /api/layouts/{id}`, `POST
|
|
/api/frames/{id}/layouts/{id}/apply`) -- name + Save, then a list of
|
|
saved layouts each with Apply/rename/delete, incompatible ones shown
|
|
greyed-out with a "different orientation" badge rather than hidden.
|
|
|
|
## Weather widget
|
|
|
|
A standalone widget type (`models.WeatherWidgetConfig`, `app/widgets/
|
|
weather.py`) -- distinct from, and unrelated in code to,
|
|
`CalendarWidgetConfig`'s own embedded weather strip (still present,
|
|
still Open-Meteo-only, still working exactly as before). Four display
|
|
modes (`WeatherWidgetConfig.mode`, switchable in the widget's dialog like
|
|
`calendar_view`):
|
|
|
|
- `current` -- one city's current temp + a condition icon.
|
|
- `hourly` -- one city, a row of ticks across the day at a configurable
|
|
interval (`hourly_interval_hours`: 3/4/6/12).
|
|
- `daily` -- one city, a multi-day strip (`daily_days`, 1-14).
|
|
- `multi_city` -- several cities' current-day high/low/icon side by
|
|
side -- the calendar widget's embedded strip, as a standalone
|
|
widget's whole content instead of a strip above an agenda day.
|
|
|
|
`current`/`hourly`/`daily` share one configured location
|
|
(`city_label`/`city_latitude`/`city_longitude`, set via `POST .../
|
|
weather-location`, geocoded through `weather.geocode_city`); `multi_city`
|
|
has its own list (`cities`, add/remove via `POST .../weather-widget-
|
|
cities/add`|`remove` -- named to avoid colliding with the calendar
|
|
widget's own, differently-scoped `weather-cities/add`|`remove` routes,
|
|
which share the same `{widget_id}`-parameterized path shape).
|
|
|
|
**Providers** (`app/weather/`, a dispatch registry over pluggable
|
|
implementations mirroring `app/widgets/` itself): `WeatherWidgetConfig.
|
|
provider` selects which of `app/weather.PROVIDERS` actually fetches --
|
|
`"open_meteo"` (worldwide, no API key) or `"nws"` (api.weather.gov, US
|
|
only, no API key, approximates "current" with the first hourly forecast
|
|
period rather than a real station observation). Every provider function
|
|
returns already-normalized `{"category": ...}` entries (one of `clear`/
|
|
`partly_cloudy`/`cloudy`/`fog`/`rain`/`snow`/`thunderstorm`) so
|
|
`app/weather_render.py`'s drawing code never needs to know which
|
|
provider supplied an entry. `geocode_city` (name -> lat/lon) always goes
|
|
through Open-Meteo's free geocoder regardless of which provider is
|
|
chosen to fetch with the result.
|
|
|
|
**Environment Canada is a deliberate gap, not an oversight** -- its free
|
|
API (the MSC GeoMet OGC service) is built around station/grid lookups,
|
|
not simple lat/lon REST like the two providers above, and would have
|
|
meaningfully expanded the initial pass. Next provider to add if this
|
|
gets revisited.
|
|
|
|
`app/weather_render.py` holds every weather-related drawing primitive:
|
|
`draw_cloud`/`draw_weather_icon`/`draw_weather_row` (extracted out of
|
|
`calendar_render.py`, which still imports `draw_weather_row` for its own
|
|
embedded strip, unchanged) plus this widget's own `build_current`/
|
|
`build_hourly`/`build_daily`/`build_multi_city`, dispatched by `build()`
|
|
-- the weather analogue of `calendar_render.py`'s own `_build_tasks`/
|
|
`render_tasks_preview_png` relationship.
|
|
|
|
## Known gaps (Phase 6, not yet done)
|
|
|
|
The original 8-phase rollout plan's last phase is still open:
|
|
|
|
- Legacy per-mode `Frame` columns (`mode`, `album_id`,
|
|
`current_asset_id`, all `calendar_*`, all `whiteboard_*`, etc.) are
|
|
still physically present in the schema but no longer read or written
|
|
anywhere -- they need a dedicated final migration to drop them. Left in
|
|
place deliberately through the widget-system rollout (a much larger
|
|
blast radius cutover than this project's usual same-migration-drop
|
|
convention) but there's no reason to keep carrying them now that every
|
|
phase has shipped.
|
|
- `server/README.md` still describes photos/calendar/whiteboard as
|
|
per-frame "modes" in several places rather than widgets -- needs a pass
|
|
once the column drop above is safely deployed.
|
|
- Whiteboard rendering is tagged **(alpha)** in the UI -- not fully
|
|
reliable yet, treat it as experimental if extending it.
|