tfaour
b9649c35ec
The panel driver now splits writing a frame into its SPI buffer (epd_write_frame(), which also computes a CRC32 as it streams) from actually triggering the physical refresh (epd_turn_on_display()). frame_client.c compares the new CRC against the last one that was actually refreshed (persisted in NVS) and skips the refresh entirely when they match -- e.g. a reboot redisplaying the same photo before the server's refresh interval elapsed no longer causes a visible flash for no visual change. Also reorders the per-wake fetch cycle: the image fetch (15s timeout) now goes before the config check (3s timeout), instead of after. The config check's tighter timeout was intermittently tripping on connection-setup latency that's common on the first request after waking from a long deep sleep (e.g. stale ARP); putting the more tolerant request first absorbs that latency, and the config check then rides the connection it already warmed up.
104 lines
5.2 KiB
Markdown
104 lines
5.2 KiB
Markdown
# Architecture
|
|
|
|
Two independent pieces talk over plain HTTP on the local network: the
|
|
ESP32-C6 firmware, and a small FastAPI server that sits between it and
|
|
Immich.
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Immich
|
|
participant Server as ESPresso Frame Server
|
|
participant Frame as ESP32-C6 Frame
|
|
|
|
Note over Frame: First boot / never provisioned
|
|
Frame->>Frame: Generate AP SSID/password, draw QR + config QR on panel
|
|
Frame->>Frame: Bring up ESPRESSO_XXXXXX softAP + captive portal
|
|
Note over Frame: User scans WiFi QR, then config QR -> fills in<br/>home WiFi + "Tools Server" host:port
|
|
Frame->>Frame: Save config to NVS, reboot
|
|
|
|
Note over Frame: Every wake (deep sleep timer, next-photo button,<br/>or any other reboot)
|
|
Frame->>Frame: Connect to home WiFi
|
|
alt next-photo button pressed
|
|
Frame->>Server: POST /frame/advance
|
|
Server->>Server: Force-advance to next queued photo, reset interval clock
|
|
else normal wake
|
|
Frame->>Server: GET /frame/image
|
|
Server->>Server: Advance only if refresh_interval_s has elapsed<br/>since the current photo was set -- otherwise a no-op
|
|
end
|
|
Server->>Immich: List album assets / download preview / faces
|
|
Immich-->>Server: JPEG + face bounding boxes
|
|
Server->>Server: Crop (face-aware) + quantize (dither) + pack 4bpp
|
|
Server-->>Frame: 192,000 raw bytes, streamed
|
|
Frame->>Frame: Write to panel SPI buffer, compute CRC32
|
|
alt CRC unchanged since last physical refresh
|
|
Frame->>Frame: Skip refresh (nothing visually changed)
|
|
else CRC changed
|
|
Frame->>Frame: Trigger physical refresh, store new CRC
|
|
end
|
|
Frame->>Server: GET /frame/config
|
|
Server-->>Frame: {"refresh_interval_s": ...}
|
|
Frame->>Frame: Deep sleep (server-configured interval, or a short<br/>retry interval on any failure)
|
|
```
|
|
|
|
## Firmware boot flow
|
|
|
|
1. **No stored config** (first boot, or NVS erased): bring up the display,
|
|
render a WiFi-join QR code + plaintext password (left) and a
|
|
captive-portal config QR code (right), *then* start the `ESPRESSO_XXXXXX`
|
|
softAP + DNS redirect + HTTP server. The display goes up before the AP
|
|
so the join instructions are visible before the network is joinable.
|
|
The captive portal form saves SSID/password/toolsserver to NVS and
|
|
reboots.
|
|
2. **Stored config exists**: connect to the saved WiFi network (a few
|
|
retries before falling back to provisioning if it fails), then run the
|
|
fetch cycle in `frame_client.c`:
|
|
- Check the next-photo button (`next_button_check()`) -- if it was
|
|
what woke the device (checked via the latched
|
|
`esp_sleep_get_gpio_wakeup_status()`, not a live pin read, since a
|
|
quick tap can release before boot gets around to polling it) or is
|
|
currently held, the fetch below hits `POST /frame/advance` instead
|
|
of `GET /frame/image`, forcing the server to skip ahead immediately.
|
|
- Fetch the frame and write it into the panel's SPI buffer
|
|
(`epd_write_frame()`), computing a CRC32 as it streams -- never
|
|
buffering the full ~192KB frame in RAM. The panel driver refuses to
|
|
write a short/wrong-size response into the buffer at all, so a
|
|
truncated fetch can't corrupt what's already there.
|
|
- Compare the new CRC32 against the last one that was actually
|
|
refreshed onto the panel (persisted in NVS). If it matches -- the
|
|
same photo is already visibly on screen, e.g. the device rebooted
|
|
before the server's refresh interval elapsed -- skip the physical
|
|
refresh entirely (`epd_turn_on_display()`), avoiding its visible
|
|
flash and 15-30s duration for no visual change. Otherwise trigger
|
|
the refresh and store the new CRC.
|
|
- `GET /frame/config` for the refresh interval, used to set the deep
|
|
sleep duration -- deliberately fetched *after* the image, not
|
|
before: its timeout is much tighter (3s vs. the image fetch's 15s),
|
|
and fetching second lets it ride the connection the image fetch just
|
|
warmed up rather than eating the latency spike common on the first
|
|
request after waking from a long sleep.
|
|
- Deep sleep for the server-configured interval on success, or a
|
|
shorter retry interval on any failure.
|
|
|
|
The factory-reset button (hold 10s) is checked earlier, before any of
|
|
this -- see [`firmware/README.md`](../firmware/README.md#resetting-to-provisioning-mode).
|
|
|
|
See [`docs/hardware.md`](hardware.md) for wiring and
|
|
[`server/README.md`](../server/README.md) for the server side.
|
|
|
|
## Why image processing happens server-side
|
|
|
|
The ESP32-C6 has no PSRAM and a tight SRAM budget (already tight enough
|
|
that a single 4KB stack buffer caused a crash during development -- see
|
|
git history). Decoding a JPEG, then resizing/dithering/quantizing it to
|
|
the panel's 6-color palette, would be expensive on-device in both memory
|
|
and battery. Instead, the server does all of that with Pillow and hands
|
|
the frame a pre-packed, ready-to-stream buffer -- the device never
|
|
decodes an image at all.
|
|
|
|
## Why face detection isn't run on-device (or even on the server)
|
|
|
|
Immich already runs face detection for its own "People" feature. The
|
|
server just asks Immich for the bounding boxes it already computed
|
|
(`GET /api/faces?id=...`) and biases the crop to keep them on screen,
|
|
rather than bundling a detector (OpenCV/dlib) anywhere in this project.
|