Add CLAUDE.md and docs/widgets.md; fix stale single-mode architecture docs
Repo-tracked context so a fresh Claude Code session (this machine or a remote/cloud one) gets accurate project context without relying on this session's local, machine-specific memory: repo conventions (no co-author trailers, flag copyleft deps explicitly, scope security gates broadly -- each backed by a real past incident), testing/deploy workflow, and pointers into the existing docs. docs/architecture.md's sequence diagram and boot-flow text still described the pre-widget-system single-photo-queue model (e.g. "force- advance to next queued photo") even though that was fully replaced by the widget system across this branch's recent history -- fixed, and added docs/widgets.md distilling the widget system's actual design (data model, grid placement, compositor, button-action dispatch) as current-state documentation, including the still-open Phase 6 cleanup (legacy Frame columns not yet dropped) as a known gap.
This commit is contained in:
@@ -0,0 +1,71 @@
|
||||
# espresso_frame
|
||||
|
||||
A DIY e-ink photo frame: an ESP32-C6 (`firmware/`, ESP-IDF) driving a
|
||||
Waveshare 7.3" E Ink Spectra 6 panel (800x480, 6-color, SPI), paired with a
|
||||
self-hosted FastAPI server (`server/`) that pulls from Immich, does all
|
||||
image processing (crop/dither/quantize/pack), and serves a placeable
|
||||
photos/calendar/whiteboard widget system to the device.
|
||||
|
||||
Start here, don't re-derive from scratch:
|
||||
- [`docs/architecture.md`](docs/architecture.md) -- how firmware and
|
||||
server talk (sequence diagram, boot flow).
|
||||
- [`docs/widgets.md`](docs/widgets.md) -- the server-side widget system
|
||||
(data model, grid placement, compositor, button-action dispatch). Notes
|
||||
a known gap at the bottom (legacy `Frame` columns not yet dropped).
|
||||
- [`docs/hardware.md`](docs/hardware.md) -- wiring.
|
||||
- [`server/README.md`](server/README.md), [`firmware/README.md`](firmware/README.md)
|
||||
-- per-component setup, config, and a lot of accumulated gotchas
|
||||
(Immich API shape, TLS trust-anchor details, button GPIO wakeup
|
||||
quirks, etc.) -- check these before assuming something is a new bug.
|
||||
|
||||
## Conventions specific to this repo
|
||||
|
||||
- **No `Co-Authored-By: Claude` trailers in commits.** Attribution lives
|
||||
in the root [`README.md`](README.md) instead (see its last line) --
|
||||
the maintainer's explicit preference, not the default.
|
||||
- **Copyleft dependencies need an explicit flag, not a silent decision.**
|
||||
Before adding anything LGPL/GPL/AGPL (or unclear), verify the actual
|
||||
license via `pip show`/package metadata -- including transitive deps,
|
||||
not just the top-level package -- and present the finding and tradeoff
|
||||
in plain text rather than picking an approach unilaterally (hand-rolling
|
||||
an alternative, swapping packages, silently accepting it). This project
|
||||
has knowingly accepted AGPL-3.0-or-later exposure once already
|
||||
(`icalendar-searcher`, a transitive dep of `caldav`) as a deliberate,
|
||||
explicit call -- not a precedent for skipping the check next time.
|
||||
- **Scope new auth/access-control broadly, not just to the literal
|
||||
endpoint named.** When a request changes the trust model (e.g. adding
|
||||
public-internet exposure), apply the new gate to every endpoint serving
|
||||
real data or performing a real action, and call out anything you're
|
||||
tempted to exclude and why. This repo shipped a token gate once that
|
||||
covered `/api/*` but left `/frame/image` -- the actual photo bytes --
|
||||
open; caught immediately in production.
|
||||
|
||||
## Working in this repo
|
||||
|
||||
- **Server tests**: `cd server && pytest` (SQLite, fixtures wipe/reseed
|
||||
between tests -- see `tests/conftest.py`). Migration changes need a
|
||||
matching test in `tests/test_migrations.py`; anything touching
|
||||
`require_frame_view`/`require_frame_control` boundaries needs a
|
||||
same-shape permission test (see `tests/test_permission_boundaries.py`
|
||||
and `tests/test_button_actions.py` for the pattern: owner, linked user,
|
||||
unrelated user, logged out).
|
||||
- **UI changes**: verify in a real browser (Playwright), not just by
|
||||
reading the JS -- this project has hit multiple bugs that only showed up
|
||||
live (mobile viewport CSS collapse, a dialog's status message landing
|
||||
behind its own backdrop, a JSON/form-urlencoded body mismatch). Spin up
|
||||
`uvicorn app.main:app` against a scratch `DATABASE_URL`/`CONFIG_PATH`
|
||||
sqlite file, don't touch the real deployment's data.
|
||||
- **Deploy**: Gitea Actions at `git.thumeit.com/tfaour/espresso_frame`
|
||||
(`.gitea/workflows/server-docker-build.yml`: `test` -> `build-and-push`
|
||||
-> `deploy` on any push to `main` touching `server/**`; `deploy` SSHes
|
||||
into the host as `espressoframe_deployer` and runs `docker compose pull
|
||||
&& docker compose up -d`). A separate workflow
|
||||
(`firmware-release-build.yml`) builds+publishes firmware binaries as
|
||||
Gitea release assets when `firmware/version.txt` changes. Poll CI status
|
||||
with `curl https://git.thumeit.com/api/v1/repos/tfaour/espresso_frame/actions/tasks`
|
||||
rather than asking the user to check.
|
||||
- **Device-facing paths are frozen.** `/frame/image`, `/frame/advance`,
|
||||
`/frame/back`, `/frame/config`, `/frame/battery`, `/frame/firmware` and
|
||||
their exact JSON key names (`refresh_interval_s`, `firmware_version`,
|
||||
etc.) are baked into deployed firmware -- never rename or restructure
|
||||
these without a firmware-side migration story to match.
|
||||
@@ -24,9 +24,10 @@ time in deep sleep.
|
||||
- ESP32-C6 dev board (8MB flash)
|
||||
- [Waveshare 7.3" E Ink Spectra 6 (E6)](https://www.waveshare.com/7.3inch-e-paper-hat-e.htm) panel -- 800x480, 6-color, SPI
|
||||
|
||||
See [`docs/hardware.md`](docs/hardware.md) for wiring and
|
||||
See [`docs/hardware.md`](docs/hardware.md) for wiring,
|
||||
[`docs/architecture.md`](docs/architecture.md) for how the two halves talk
|
||||
to each other.
|
||||
to each other, and [`docs/widgets.md`](docs/widgets.md) for the server's
|
||||
placeable photos/calendar/whiteboard widget system.
|
||||
|
||||
## Getting started
|
||||
|
||||
|
||||
+16
-5
@@ -22,17 +22,17 @@ sequenceDiagram
|
||||
Frame->>Frame: Connect to home WiFi
|
||||
alt next-photo button pressed
|
||||
Frame->>Server: POST /frame/advance
|
||||
Server->>Server: Force-advance to next queued photo, reset interval clock
|
||||
Server->>Server: Run every action assigned to NEXT, in order<br/>(may span several widgets -- see docs/widgets.md)
|
||||
else back-photo button pressed
|
||||
Frame->>Server: POST /frame/back
|
||||
Server->>Server: Return to previously-current photo (bounded history),<br/>reset interval clock
|
||||
Server->>Server: Run every action assigned to BACK, in order
|
||||
else normal wake
|
||||
Frame->>Server: GET /frame/image
|
||||
Server->>Server: Advance only if refresh_interval_s has elapsed<br/>since the current photo was set -- otherwise a no-op
|
||||
Server->>Server: Render every widget on the panel into its own region<br/>(each independently idempotent -- a photo widget only<br/>actually advances once its own refresh_interval_s has elapsed)
|
||||
end
|
||||
Server->>Immich: List album assets / download preview / faces
|
||||
Server->>Immich: List album assets / download preview / faces<br/>(once per photo widget on the panel)
|
||||
Immich-->>Server: JPEG + face bounding boxes
|
||||
Server->>Server: Crop (face-aware) + quantize (dither) + pack 4bpp
|
||||
Server->>Server: Composite every widget's region onto one canvas,<br/>then enhance/overlay/quantize (dither)/pack 4bpp once
|
||||
Server-->>Frame: 192,000 raw bytes, streamed
|
||||
Frame->>Frame: Write to panel SPI buffer, compute CRC32
|
||||
alt CRC unchanged since last physical refresh
|
||||
@@ -45,6 +45,17 @@ sequenceDiagram
|
||||
Frame->>Frame: Deep sleep (server-configured interval, or a short<br/>retry interval on any failure)
|
||||
```
|
||||
|
||||
The device-facing endpoints above (`/frame/image`, `/frame/advance`,
|
||||
`/frame/back`, `/frame/config`) are frozen -- baked into deployed firmware
|
||||
-- and unchanged by any of this. What *does* change server-side: a frame's
|
||||
panel isn't a single fixed "mode" anymore, it holds an arbitrary
|
||||
arrangement of independently placed/sized widgets (photos/calendar/
|
||||
whiteboard, including several of the same type), each rendered into its
|
||||
own region and composited together, with NEXT/BACK each mapped to their
|
||||
own ordered list of per-widget actions rather than one fixed meaning. See
|
||||
[`docs/widgets.md`](widgets.md) for the widget system's data model,
|
||||
placement grid, and button-action dispatch.
|
||||
|
||||
## Firmware boot flow
|
||||
|
||||
1. **No stored config** (first boot, or NVS erased): bring up the display,
|
||||
|
||||
+149
@@ -0,0 +1,149 @@
|
||||
# Widget system
|
||||
|
||||
A frame's panel isn't one fixed "mode" anymore -- it holds N independently
|
||||
placed/sized widgets (photos/calendar/whiteboard), like arranging icons on
|
||||
an Android home screen. A frame can hold several widgets of the same type
|
||||
(e.g. two photo widgets pointed at different Immich albums side by side).
|
||||
This replaced an earlier design where `Frame.mode` picked exactly one
|
||||
full-panel renderer; that column (and the other now-dead per-mode `Frame`
|
||||
columns it left behind -- `album_id`, `calendar_*`, `whiteboard_*`, etc.)
|
||||
is still physically present but unused, pending a final cleanup migration
|
||||
(see "Known gaps" below).
|
||||
|
||||
The device-facing contract is unchanged by any of this: `GET /frame/image`,
|
||||
`POST /frame/advance`, `POST /frame/back` are the same frozen paths
|
||||
firmware has always called (see `docs/architecture.md`) -- what changed is
|
||||
entirely server-side, in how those endpoints decide what to render and what
|
||||
a button press does.
|
||||
|
||||
## Data model
|
||||
|
||||
- `Widget` (`server/app/models.py`): `id`, `frame_id`, `widget_type`
|
||||
(`"photos"` | `"calendar"` | `"whiteboard"`), `x`/`y`/`w`/`h` (grid
|
||||
cells), `sort_order`. Widgets never overlap (enforced server-side in
|
||||
`routers/api_widgets.py`, re-validated regardless of what the client
|
||||
already checked) -- that's what keeps compositing simple: no z-order,
|
||||
no blending, just N independent regions pasted onto one shared canvas.
|
||||
- Per-type 1:1 extension tables -- `PhotoWidgetConfig`,
|
||||
`CalendarWidgetConfig`, `WhiteboardWidgetConfig`, each keyed by
|
||||
`widget_id` with `ondelete="CASCADE"` -- rather than one wide table with
|
||||
every type's mostly-irrelevant columns. `PhotoWidgetConfig` mirrors
|
||||
`app/photo_queue.py`'s attribute names exactly, so that module's
|
||||
advance/back/queue logic ports across widget instances unchanged.
|
||||
- `FrameCalendar` is keyed by `widget_id` (not `frame_id`) since a frame
|
||||
can now have more than one independent calendar widget, each with its
|
||||
own set of included calendars.
|
||||
- `FrameButtonAction` (`id`, `frame_id`, `button` [`"next"`|`"back"`],
|
||||
`widget_id`, `action`, `sort_order`) -- see "Button actions" below.
|
||||
|
||||
## Placement: a grid, not freeform pixels
|
||||
|
||||
`app/grid.py` is pure grid math, no I/O. The grid is `GRID_LONG=8` x
|
||||
`GRID_SHORT=5` cells, defined relative to the panel's long/short axis
|
||||
(not "landscape" specifically) so it stays valid across
|
||||
`image_pipeline.logical_render_size(orientation)`'s genuine width/height
|
||||
swap for portrait -- landscape orientations are 8 cols x 5 rows, portrait
|
||||
are 5 cols x 8 rows, same cell size either way. **Changing a frame's
|
||||
orientation invalidates its existing layout** (an 8x5 arrangement isn't
|
||||
valid on a 5x8 grid) -- the server resets to one full-panel widget on an
|
||||
orientation change rather than trying to remap coordinates.
|
||||
|
||||
Each widget type has a minimum grid footprint (`grid.MIN_FOOTPRINT`):
|
||||
photos 1x1, calendar 3x2 (a crammed calendar is illegible regardless of
|
||||
size-tier scaling), whiteboard 2x2. Enforced both client-side (UX, in the
|
||||
Layout tab's drag/resize canvas -- `static/frame_layout.js`) and
|
||||
server-side (`routers/api_widgets.py`) -- the client is never trusted
|
||||
alone.
|
||||
|
||||
## Rendering: one shared compositor
|
||||
|
||||
`app/widgets/` is the render/action registry -- one module per
|
||||
`widget_type` (`photos.py`, `calendar.py`, `whiteboard.py`), each
|
||||
exposing:
|
||||
|
||||
- `render(db, frame, widget, target_w, target_h, is_normal_wake) -> Image`:
|
||||
an unquantized RGB image exactly `target_w x target_h`, the widget's
|
||||
content composed into its own region. Never raises for a foreseeable
|
||||
failure (an Immich hiccup, an unconfigured widget) -- falls back to a
|
||||
small placeholder within its own region instead, so one widget having a
|
||||
bad moment doesn't blank the whole panel.
|
||||
- `ACTIONS: dict[str, Callable]` -- named button actions this type
|
||||
supports (`"advance"`/`"back"` for photos and calendar, `"check_now"`
|
||||
for whiteboard).
|
||||
- `ACTION_LABELS: dict[str, str]` -- human labels for the button-
|
||||
assignment UI.
|
||||
|
||||
`routers/device.py`'s `_render_widgets` loads every `Widget` row for the
|
||||
frame, maps each one's grid rect to pixels (`grid.cell_to_pixels`), calls
|
||||
its module's `render()`, and hands the whole list of `(rect, image)`
|
||||
regions to `image_pipeline.render_panel` -- which pastes every region onto
|
||||
one shared canvas, then runs enhance/manage-overlay/quantize/dither/pack
|
||||
**once** over the composited result. Quantizing the whole canvas together
|
||||
(not each region separately before pasting) is what keeps the 6-color
|
||||
e-ink dithering pattern consistent across a widget boundary instead of a
|
||||
visible seam at the edge.
|
||||
|
||||
Calendar widgets pick from discrete size tiers (`calendar_render.py`'s
|
||||
`_SIZE_TIERS`) for font size/margins/row heights based on their actual
|
||||
grid footprint, rather than continuously scaling constants tuned for a
|
||||
full ~800x480 canvas -- falls back to agenda view if a widget is too small
|
||||
for month view to stay legible.
|
||||
|
||||
## Button actions
|
||||
|
||||
Each physical button (NEXT/BACK) maps to an **ordered list** of
|
||||
`(widget, action)` bindings, not a fixed meaning -- e.g. NEXT can be
|
||||
"photo widget A: advance" *and* "calendar widget B: advance" together, or
|
||||
even a mismatched combination on purpose. On a press,
|
||||
`routers/device.py`'s `_run_button_actions` runs every assigned action for
|
||||
that button in order (each in its own `widget_locked` span -- never nested,
|
||||
since the underlying per-frame lock isn't reentrant), catching and
|
||||
logging any single action's failure without blocking the rest, then
|
||||
re-renders and returns the whole composed panel once at the end regardless
|
||||
of which actions succeeded.
|
||||
|
||||
The web UI for this is the "Button assignments" card on a frame's
|
||||
Configuration tab (`static/frame_config.js`, `GET`/`PUT
|
||||
/api/frames/{id}/buttons`) -- add/remove/reorder, autosaved. Two widgets of
|
||||
the same type would otherwise both just say "Photos" in the assignment
|
||||
dropdowns; the UI disambiguates using each widget's grid position (e.g.
|
||||
"Photos 1 (left)" / "Photos 2 (right)"), the same way you'd tell them
|
||||
apart by eye on the Layout canvas.
|
||||
|
||||
A newly-created widget (including the one auto-migrated from a frame's old
|
||||
`mode` on upgrade) gets a sensible default binding reproducing its old
|
||||
button behavior -- see `migration.py`'s `_default_button_actions`.
|
||||
|
||||
## Per-widget config UI
|
||||
|
||||
Each widget has a gear-icon button on the Layout canvas that opens a
|
||||
`<dialog>` with that widget's own settings (album, calendar inclusion,
|
||||
whiteboard source, etc.) -- not a per-frame tab, since a frame can now
|
||||
have several widgets of the same type with independent settings. The
|
||||
dialog HTML is injected server-rendered (`routers/frame_pages.py`'s
|
||||
`widget_dialog`, dispatching on `widget.widget_type`); its JS is a
|
||||
top-level, always-loaded file (`static/widget_dialog_*.js`) exposing
|
||||
`init<Type>Dialog()`/`close<Type>Dialog()`, since dynamically-injected
|
||||
HTML can't carry executable `<script>` tags. While a dialog is open,
|
||||
`window.FRAME_API` is repointed at that widget's own API base
|
||||
(`/api/frames/{id}/widgets/{widget_id}`) and restored on close;
|
||||
`window.FRAME_BASE_API` stays pointed at the frame-level base throughout
|
||||
for the always-present header/status-bar JS.
|
||||
|
||||
## Known gaps (Phase 6, not yet done)
|
||||
|
||||
The original 8-phase rollout plan's last phase is still open:
|
||||
|
||||
- Legacy per-mode `Frame` columns (`mode`, `album_id`,
|
||||
`current_asset_id`, all `calendar_*`, all `whiteboard_*`, etc.) are
|
||||
still physically present in the schema but no longer read or written
|
||||
anywhere -- they need a dedicated final migration to drop them. Left in
|
||||
place deliberately through the widget-system rollout (a much larger
|
||||
blast radius cutover than this project's usual same-migration-drop
|
||||
convention) but there's no reason to keep carrying them now that every
|
||||
phase has shipped.
|
||||
- `server/README.md` still describes photos/calendar/whiteboard as
|
||||
per-frame "modes" in several places rather than widgets -- needs a pass
|
||||
once the column drop above is safely deployed.
|
||||
- Whiteboard rendering is tagged **(alpha)** in the UI -- not fully
|
||||
reliable yet, treat it as experimental if extending it.
|
||||
Reference in New Issue
Block a user