diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..5856ec1 --- /dev/null +++ b/LICENSE @@ -0,0 +1,39 @@ +MIT License + +Copyright (c) 2026 Thomas Faour + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. + +--- + +This repository vendors a small amount of third-party code under their own +permissive licenses, unmodified, each in its own directory with attribution: + +- `firmware/components/qrcode/` -- Nayuki's QR Code generator library, MIT. +- `firmware/components/epaper_fonts/` -- an STMicroelectronics bitmap font + table redistributed by Waveshare, BSD-3-Clause. +- `firmware/components/dns_server/` -- from Espressif's ESP-IDF examples, + Unlicense OR CC0-1.0. + +See each directory's source comments / README for full license text. +`firmware/components/epd7in3e/` is original code written for this project, +but its register/command sequence is a transcription of values from +Waveshare's MIT-licensed `EPD_7in3e.c` reference driver (this panel has no +public datasheet) -- see the attribution comment at the top of +`epd7in3e.c`. diff --git a/README.md b/README.md new file mode 100644 index 0000000..64bd937 --- /dev/null +++ b/README.md @@ -0,0 +1,56 @@ +# ESPresso Frame + +A DIY e-ink photo frame: an ESP32-C6 pulls photos from your +[Immich](https://immich.app) library and displays them on a 7.3" full-color +e-paper panel, waking on a timer to refresh and spending the rest of its +time in deep sleep. + +- **No cables to a computer, no SD card shuffling.** Provisioning is a + captive portal with a QR code drawn on the panel itself -- scan, join, + fill in your WiFi and server address, done. +- **The frame never decodes an image.** A small self-hosted server does + all the work (pulling from Immich, cropping, dithering, packing into + the panel's exact pixel format) and hands the device a stream it can + write straight to SPI. The ESP32-C6 has no PSRAM and not much SRAM to + spare -- keeping it a dumb display client is what makes that workable. +- **Crops toward faces, not just the center**, using face bounding boxes + Immich already computed for its own People feature -- no bundled face + detector. +- **Refresh interval and album are configurable from a web UI**, no + reflashing needed to change them. + +## Hardware + +- ESP32-C6 dev board (8MB flash) +- [Waveshare 7.3" E Ink Spectra 6 (E6)](https://www.waveshare.com/7.3inch-e-paper-hat-e.htm) panel -- 800x480, 6-color, SPI + +See [`docs/hardware.md`](docs/hardware.md) for wiring and +[`docs/architecture.md`](docs/architecture.md) for how the two halves talk +to each other. + +## Getting started + +1. **[`server/`](server/)** -- run the FastAPI server first (Docker + Compose, points at your Immich instance). See + [`server/README.md`](server/README.md). +2. **[`firmware/`](firmware/)** -- build and flash the ESP32-C6, then + scan the QR codes it draws on first boot to provision it. See + [`firmware/README.md`](firmware/README.md). + +## Repo layout + +``` +firmware/ ESP-IDF project for the ESP32-C6 +server/ FastAPI server: Immich -> crop/dither/pack -> the frame +docs/ Wiring and architecture notes +``` + +## License + +MIT -- see [`LICENSE`](LICENSE). A few small pieces of vendored +third-party code (a QR code generator, a bitmap font table) keep their +own permissive licenses; see `LICENSE` for details. + +--- + +Built with substantial assistance from [Claude Code](https://claude.com/claude-code). diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..ed8e7c3 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,80 @@ +# Architecture + +Two independent pieces talk over plain HTTP on the local network: the +ESP32-C6 firmware, and a small FastAPI server that sits between it and +Immich. + +```mermaid +sequenceDiagram + participant Immich + participant Server as ESPresso Frame Server + participant Frame as ESP32-C6 Frame + + Note over Frame: First boot / never provisioned + Frame->>Frame: Generate AP SSID/password, draw QR + config QR on panel + Frame->>Frame: Bring up ESPRESSO_XXXXXX softAP + captive portal + Note over Frame: User scans WiFi QR, then config QR -> fills in
home WiFi + "Tools Server" host:port + Frame->>Frame: Save config to NVS, reboot + + Note over Frame: Every wake (deep sleep timer) + Frame->>Frame: Connect to home WiFi + Frame->>Server: GET /frame/config + Server-->>Frame: {"refresh_interval_s": ...} + alt server unreachable + Frame->>Frame: Show "SERVER: FAILED" status screen + Frame->>Frame: Deep sleep (short retry interval) + else server reachable + Frame->>Server: GET /frame/image + Server->>Immich: List album assets / download preview / faces + Immich-->>Server: JPEG + face bounding boxes + Server->>Server: Crop (face-aware) + quantize (dither) + pack 4bpp + Server-->>Frame: 192,000 raw bytes, streamed + Frame->>Frame: Stream straight to panel SPI, refresh + Frame->>Frame: Deep sleep (server-configured interval) + end +``` + +## Firmware boot flow + +1. **No stored config** (first boot, or NVS erased): bring up the display, + render a WiFi-join QR code + plaintext password (left) and a + captive-portal config QR code (right), *then* start the `ESPRESSO_XXXXXX` + softAP + DNS redirect + HTTP server. The display goes up before the AP + so the join instructions are visible before the network is joinable. + The captive portal form saves SSID/password/toolsserver to NVS and + reboots. +2. **Stored config exists**: connect to the saved WiFi network (a few + retries before falling back to provisioning if it fails), then run the + fetch cycle in `frame_client.c`: + - `GET /frame/config` on the configured tools server -- doubles as a + reachability check and the source of the refresh interval (a + Kconfig value is only used as a fallback). + - If reachable, `GET /frame/image` and stream the response directly + into the panel over SPI (`epd_display_stream()`), never buffering + the full ~192KB frame in RAM. + - The panel driver refuses to physically refresh unless the stream + supplied *exactly* the expected byte count -- a truncated or + wrong-size response leaves the previous image on screen instead of + painting garbage. + - Deep sleep for the server-configured interval on success, or a + shorter retry interval on any failure. + +See [`docs/hardware.md`](hardware.md) for wiring and +[`server/README.md`](../server/README.md) for the server side. + +## Why image processing happens server-side + +The ESP32-C6 has no PSRAM and a tight SRAM budget (already tight enough +that a single 4KB stack buffer caused a crash during development -- see +git history). Decoding a JPEG, then resizing/dithering/quantizing it to +the panel's 6-color palette, would be expensive on-device in both memory +and battery. Instead, the server does all of that with Pillow and hands +the frame a pre-packed, ready-to-stream buffer -- the device never +decodes an image at all. + +## Why face detection isn't run on-device (or even on the server) + +Immich already runs face detection for its own "People" feature. The +server just asks Immich for the bounding boxes it already computed +(`GET /api/faces?id=...`) and biases the crop to keep them on screen, +rather than bundling a detector (OpenCV/dlib) anywhere in this project. diff --git a/docs/hardware.md b/docs/hardware.md new file mode 100644 index 0000000..9e53d3e --- /dev/null +++ b/docs/hardware.md @@ -0,0 +1,52 @@ +# Hardware + +## Parts + +- An ESP32-C6 dev board (e.g. ESP32-C6-DevKitC-1). Needs 8MB flash -- + see [`firmware/sdkconfig.defaults`](../firmware/sdkconfig.defaults) and + [`firmware/partitions.csv`](../firmware/partitions.csv) if yours differs. +- [Waveshare 7.3" E Ink Spectra 6 (E6) panel](https://www.waveshare.com/7.3inch-e-paper-hat-e.htm) -- + 800x480, 6-color, SPI. +- A USB cable for flashing/power. + +## Wiring + +The panel connects over SPI plus three control lines (data/command, reset, +busy). Defaults below match the reference build and are set in +[`firmware/components/epd7in3e/Kconfig`](../firmware/components/epd7in3e/Kconfig) -- +override via `idf.py menuconfig` under **E-Paper Display (epd7in3e) +Configuration** if your wiring differs. + +| Panel pin | ESP32-C6 GPIO | Kconfig option | +| --------- | ------------- | ----------------- | +| CLK | 20 | `EPD_PIN_CLK` | +| DIN | 19 | `EPD_PIN_MOSI` | +| CS | 18 | `EPD_PIN_CS` | +| DC | 9 | `EPD_PIN_DC` | +| RST | 10 | `EPD_PIN_RST` | +| BUSY | 11 | `EPD_PIN_BUSY` | +| VCC | 3.3V | -- | +| GND | GND | -- | + +A couple of things worth knowing if you pick different pins: + +- **Avoid the ESP32-C6's strapping pins** (GPIO 4, 5, 8, 9, 15) and the + USB-JTAG pins (12, 13) where possible -- strapping pins are sampled at + reset to select boot mode. The reference wiring above already uses + GPIO9 for DC, which *is* a strapping pin; it's only sampled during + power-on/reset, so it's safe once the app is running, but if + flashing/boot ever misbehaves on your board, check whether the panel is + pulling that line low during reset. +- The panel's SPI interface is rated well above the firmware's default + 4MHz clock (`EPD_SPI_CLOCK_HZ`), but breadboard/dupont-wire connections + are often unreliable much past a few MHz. Raise it once your physical + wiring is confirmed solid. + +## Power + +The device spends nearly all its time in deep sleep, waking briefly once +an hour (configurable, see the server's web UI) to fetch and display a +photo. A full-color refresh on this panel takes 15-30+ seconds and draws +more current than deep sleep by a wide margin -- expect battery life (if +not running from USB power) to be dominated by refresh frequency, not +sleep current. diff --git a/firmware/README.md b/firmware/README.md index 7b13b55..b7eb388 100644 --- a/firmware/README.md +++ b/firmware/README.md @@ -1,129 +1,91 @@ -| Supported Targets | ESP32 | ESP32-C2 | ESP32-C3 | ESP32-C5 | ESP32-C6 | ESP32-C61 | ESP32-S2 | ESP32-S3 | -| ----------------- | ----- | -------- | -------- | -------- | -------- | --------- | -------- | -------- | +# ESPresso Frame Firmware -# Captive Portal Example +ESP-IDF firmware for the ESP32-C6. On first boot it provisions itself over +a WiFi captive portal; after that it wakes on a timer, fetches an +already-processed frame from the [server](../server/), streams it straight +to the panel over SPI, and goes back to deep sleep. -(See the README.md file in the upper level 'examples' directory for more information about examples.) +See [`docs/architecture.md`](../docs/architecture.md) for the full boot/fetch +cycle and [`docs/hardware.md`](../docs/hardware.md) for wiring. -This example demonstrates two methods of a captive portal, used to direct users to an authentication page or other necessary starting point before browsing. +## Build and flash -One approach response to all DNS queries with the address of the softAP, and redirects all HTTP requests to the captive portal root page. This "funnelling" of DNS and traffic triggers the captive portal (sign in) to appear on Android, iOS, and Windows. Note that the example will not redirect HTTPS requests. - -The other approach is a more modern method which includes a field in the DHCP offer (AKA DHCP Option 114), provided when the client is assigned an IP address, which specifies to the client where the captive portal is. This is advantageous because it doesn't require the overhead of DNS redirects and can work more reliably around HTTPS, HTST, and other security systems, as well as being more standards compliant. This feature is toggleable in the `Example Configuration`, but does not conflict with the DNS methodology -- these two methods work towards the same goal and can complement each other. - -## How to Use Example - -Before project configuration and build, be sure to set the correct chip target using `idf.py set-target `. - -### Hardware Required - -* A development board with ESP32/ESP32-S2/ESP32-C3 SoC (e.g., ESP32-DevKitC, ESP-WROVER-KIT, etc.) -* A USB cable for power supply and programming -* WiFi interface - -### Configure the project - -Open the project configuration menu (`idf.py menuconfig`). - -In the `Example Configuration` menu: - -* Set the Wi-Fi configuration. - * Set `SoftAP SSID` - * Set `SoftAP Password` - * Set `Maximal STA connections` - * Set `DHCP Captive portal` to enable or disable DHCP Option 114 - -### Build and Flash - -Build the project and flash it to the board, then run monitor tool to view serial output: +Requires [ESP-IDF](https://docs.espressif.com/projects/esp-idf/en/stable/esp32c6/get-started/) +(developed against v5.x/v6.x) with the environment sourced (`. $IDF_PATH/export.sh` +or your distro's equivalent). ``` +idf.py set-target esp32c6 +idf.py build idf.py -p PORT flash monitor ``` -(To exit the serial monitor, type ``Ctrl-]``.) +(`Ctrl-]` exits the monitor.) -See the Getting Started Guide for full steps to configure and use ESP-IDF to build projects. +The committed [`sdkconfig.defaults`](sdkconfig.defaults) pins an 8MB flash +size and a custom [`partitions.csv`](partitions.csv) (2MB app partition -- +the default "single app" ~1MB partition runs out of room once the HTTP +client, TLS, and vendored fonts/QR library are linked in). If your board +has less flash, you'll need to shrink the app partition and drop features +to fit. -## Example Output +## Configuration (`idf.py menuconfig`) +Under **ESPresso Frame Configuration**: -### ESP32 Output for DNS Redirect +| Option | Default | What it does | +| --- | --- | --- | +| `ESP_AP_SSID` | `ESPRESSO` | Provisioning softAP SSID prefix (device appends `_XXXXXX` from its MAC) | +| `ESP_MAX_STA_CONN` | 4 | Max clients on the provisioning softAP | +| `ESP_ENABLE_DHCP_CAPTIVEPORTAL` | on | DHCP Option 114 captive portal detection | +| `FRAME_STA_CONNECT_MAX_RETRIES` | 3 | Home WiFi connect attempts before falling back to provisioning | +| `FRAME_STA_CONNECT_TIMEOUT_MS` | 15000 | Per-attempt WiFi connect timeout | +| `FRAME_SERVER_CHECK_TIMEOUT_MS` | 3000 | Timeout for `GET /frame/config` (reachability check + refresh interval) | +| `FRAME_FETCH_TIMEOUT_MS` | 15000 | Timeout for `GET /frame/image` | +| `FRAME_SLEEP_INTERVAL_S` | 3600 | **Fallback only** -- the refresh interval is normally set server-side; see below | +| `FRAME_RETRY_INTERVAL_S` | 300 | Sleep duration after a failed cycle, before retrying | + +Under **E-Paper Display (epd7in3e) Configuration**: SPI/GPIO pin +assignments and SPI clock speed -- see +[`docs/hardware.md`](../docs/hardware.md) for the wiring these correspond to. + +### Why `FRAME_SLEEP_INTERVAL_S` says "fallback" + +The actual refresh interval is set from the server's web UI (see +[`server/README.md`](../server/README.md)) and delivered to the device on +every wake via `GET /frame/config`, so it can be changed without +reflashing. The Kconfig value only applies before the device has ever +successfully reached a configured server, or if the response doesn't +include a valid interval. + +## First boot + +With no stored WiFi config (a fresh device, or after erasing NVS -- see +below), the device brings up the display before anything else and shows a +two-step setup screen: + +1. **Connect to WiFi** -- a QR code encoding `WIFI:T:WPA;S:...;P:...;;` + for the device's own `ESPRESSO_XXXXXX` softAP, with the SSID and + password also printed underneath for anyone provisioning from a + desktop/laptop that can't scan a QR code. +2. **Configure device** -- a QR code linking straight to the captive + portal's config page (`http://192.168.4.1/` by default), for a + one-scan shortcut once you've joined the AP. + +The config page asks for your home WiFi SSID/password and the "Tools +Server" address (`host:port` of the [server](../server/) -- **not** your +Immich server). Saving reboots the device, which then connects to your +home network and starts its normal fetch/sleep cycle. + +## Resetting to provisioning mode + +There's currently no in-field way to force the device back into +provisioning (a future addition) -- reconfiguring means erasing its NVS +partition over USB: ``` -I (733) example: Set up softAP with IP: 192.168.4.1 -I (743) example: wifi_init_softap finished. SSID:'esp32_ssid' password:'esp32_pwd' -I (753) example: Starting server on port: '80' -I (753) example: Registering URI handlers -I (763) example_dns_redirect_server: Socket created -I (763) example_dns_redirect_server: Socket bound, port 53 -I (773) example_dns_redirect_server: Waiting for data -I (1873) wifi:new:<1,1>, old:<1,1>, ap:<1,1>, sta:<255,255>, prof:1 -I (1873) wifi:station: e8:84:a5:18:8f:80 join, AID=1, bgn, 40U -I (2203) example: station e8:84:a5:18:8f:80 join, AID=1 -I (2833) example_dns_redirect_server: Received 50 bytes from 192.168.4.2 | DNS reply with len: 66 -I (2843) example_dns_redirect_server: Waiting for data -I (3043) example_dns_redirect_server: Received 39 bytes from 192.168.4.2 | DNS reply with len: 55 -I (3043) example_dns_redirect_server: Waiting for data -I (3043) example_dns_redirect_server: Received 42 bytes from 192.168.4.2 | DNS reply with len: 58 -I (3053) example_dns_redirect_server: Waiting for data -W (3203) wifi:idx:4 (ifx:1, e8:84:a5:18:8f:80), tid:0, ssn:9, winSize:64 -I (3533) example: Redirecting to root -I (5693) example_dns_redirect_server: Received 37 bytes from 192.168.4.2 | DNS reply with len: 53 -I (5693) example_dns_redirect_server: Waiting for data -I (5783) example_dns_redirect_server: Received 46 bytes from 192.168.4.2 | DNS reply with len: 62 -I (5783) example_dns_redirect_server: Waiting for data -I (6303) example_dns_redirect_server: Received 41 bytes from 192.168.4.2 | DNS reply with len: 57 -I (6303) example_dns_redirect_server: Waiting for data -I (6303) example_dns_redirect_server: Received 41 bytes from 192.168.4.2 | DNS reply with len: 57 -I (6313) example_dns_redirect_server: Waiting for data -I (6593) example: Redirecting to root -I (9623) example: Redirecting to root -I (12913) example: Redirecting to root -I (13263) example_dns_redirect_server: Received 34 bytes from 192.168.4.2 | DNS reply with len: 50 -I (13273) example_dns_redirect_server: Waiting for data -I (13273) example_dns_redirect_server: Received 34 bytes from 192.168.4.2 | DNS reply with len: 50 -I (13283) example_dns_redirect_server: Waiting for data -I (16303) example_dns_redirect_server: Received 32 bytes from 192.168.4.2 | DNS reply with len: 48 -I (16303) example_dns_redirect_server: Waiting for data -I (18073) example: Redirecting to root -I (18273) example_dns_redirect_server: Received 34 bytes from 192.168.4.2 | DNS reply with len: 50 -I (18273) example_dns_redirect_server: Waiting for data -I (18273) example_dns_redirect_server: Received 34 bytes from 192.168.4.2 | DNS reply with len: 50 -I (18283) example_dns_redirect_server: Waiting for data -I (20683) example_dns_redirect_server: Received 42 bytes from 192.168.4.2 | DNS reply with len: 58 -I (20683) example_dns_redirect_server: Waiting for data -I (20753) example: Redirecting to root -I (21323) example: Redirecting to root -I (22683) example_dns_redirect_server: Received 48 bytes from 192.168.4.2 | DNS reply with len: 64 -I (22693) example_dns_redirect_server: Waiting for data -I (23443) example_dns_redirect_server: Received 48 bytes from 192.168.4.2 | DNS reply with len: 64 -I (23453) example_dns_redirect_server: Waiting for data -I (23473) example: Serve root -I (23503) example_dns_redirect_server: Received 48 bytes from 192.168.4.2 | DNS reply with len: 64 -I (23513) example_dns_redirect_server: Waiting for data +python -m esptool --chip esp32c6 -p PORT erase-region 0x9000 0x6000 ``` -### `tcpdump` Output for DHCP Option 114 - -Note `URL (114)` with the AP address. - -``` -19:14:20.522698 c8:yy:yy:yy:yy:yy > 74:xx:xx:xx:xx:xx, ethertype IPv4 (0x0800), length 590: (tos 0x0, ttl 64, id 243, offset 0, flags [none], proto UDP (17), length 576) - 192.168.4.1.67 > 192.168.4.2.68: [udp sum ok] BOOTP/DHCP, Reply, length 548, xid 0x76a26648, Flags [none] (0x0000) - Your-IP 192.168.4.2 - Client-Ethernet-Address 74:xx:xx:xx:xx:xx - Vendor-rfc1048 Extensions - Magic Cookie 0x63825363 - DHCP-Message (53), length 1: Offer - Subnet-Mask (1), length 4: 255.255.255.0 - Lease-Time (51), length 4: 7200 - Server-ID (54), length 4: 192.168.4.1 - Default-Gateway (3), length 4: 192.168.4.1 - Domain-Name-Server (6), length 4: 192.168.4.1 - BR (28), length 4: 192.168.4.255 - MTU (26), length 2: 1500 - URL (114), length 18: "http://192.168.4.1" - Router-Discovery (31), length 1: N - Vendor-Option (43), length 6: 1.4.0.0.0.2 -``` +(Offset/size match the `nvs` entry in [`partitions.csv`](partitions.csv); +this only wipes the config, not the app itself.) diff --git a/server/README.md b/server/README.md index 2e7f4b8..e946f6a 100644 --- a/server/README.md +++ b/server/README.md @@ -30,12 +30,15 @@ algorithm itself -- it just streams the response straight to the panel. ## Endpoints -- `GET /` -- config UI +- `GET /` -- config UI (album, order, refresh interval, face-aware crop + toggle -- not Immich URL/API key, see Setup above) - `GET /api/albums` -- lists Immich albums (used by the config UI) -- `POST /api/config` -- saves Immich URL/API key/album/order +- `POST /api/config` -- saves album/order/refresh_interval_s/smart_crop_faces - `GET /frame/image` -- returns the current photo pre-processed into the panel's raw 800x480, 4-bit-per-pixel, 2-pixels-per-byte format (`application/octet-stream`, exactly 192,000 bytes) +- `GET /frame/config` -- `{"refresh_interval_s": ...}`, polled by the frame + each wake alongside its reachability check - `GET /health` -- liveness check ## Notes