Refine manage overlay: US/CAN state abbreviations, share-QR caption, and an escalating second menu with named-face labels

Two rounds of follow-up work on the manage-button overlay:

1. Location formatting: US/Canada now show abbreviated state/province
   ("CA", "ON") instead of the full name, other countries show the full
   country name, and each is its own line (was one line, now wraps to
   two) so longer international place names have more room without
   threatening to overlap the top-right QR box. The bottom-left share QR
   also gets a "SCAN TO DOWNLOAD" caption.

2. Escalating menu: pressing the manage button again while its overlay
   is already up adds a second level -- each Immich-identified person's
   name labeled next to their face in the photo (using Immich's own
   face recognition/People data, no detection/recognition added to this
   project). A third press exits immediately instead of waiting out the
   30s auto-revert timer. No new Immich API needed -- GET /api/faces
   already embeds a nullable person.name per face; new
   server/app/face_labels.py maps a named face's box into the final
   800x480 frame's pixel space (reusing crop-box math extracted from
   image_pipeline.py's face-aware cropping). Capped at 4 named faces,
   sized to a real firmware RAM budget: each label is its own malloc'd
   overlay region on the device, alongside the 4 fixed corner regions
   already in use. New GET /frame/face-labels returns a flattened
   fixed-slot JSON shape (not a real array) so firmware's existing
   flat-scalar parser can read it without needing an actual array
   parser. No persistent state needed for the escalation itself -- it's
   all local control flow within one continuous awake session
   (frame_client.c's run_management_menu()).

server/README.md

@@ -48,13 +48,16 @@ algorithm itself -- it just streams the response straight to the panel.
   button (see `firmware/README.md`).
 - `GET /frame/config` -- `{"refresh_interval_s": ...}`, polled by the frame
   each wake alongside its reachability check
-- `GET /frame/photo-info` -- `{"asset_id": ..., "location": ... | null,
-  "taken_at": ... | null}` for the current photo (same idempotent
-  current-photo semantics as `/frame/image`). `location` is `city, state`
-  (or `city, country`, or just `city`) if Immich reverse-geocoded the
-  photo's GPS EXIF, else `null`; `taken_at` is `MM/DD/YY` from the
-  photo's EXIF capture date, else `null`. Used by the device's manage
-  button to build its overlay text
+- `GET /frame/photo-info` -- `{"asset_id": ..., "location_line1": ... |
+  null, "location_line2": ... | null, "taken_at": ... | null}` for the
+  current photo (same idempotent current-photo semantics as
+  `/frame/image`). `location_line1`/`location_line2` are `city` /
+  `state-or-country` if Immich reverse-geocoded the photo's GPS EXIF
+  (both `null` if not) -- for US/Canada, the region line is the
+  abbreviated state/province (`"CA"`, `"ON"`); elsewhere it's the full
+  country name. `taken_at` is `MM/DD/YY` from the photo's EXIF capture
+  date, else `null`. Used by the device's manage button to build its
+  overlay text
 - `GET /frame/share/{asset_id}` -- creates a 30-minute public, view-only
   Immich share link for `asset_id` and redirects (302) to it. Only works
   for the photo currently showing or in the upcoming queue on this frame
@@ -62,6 +65,13 @@ algorithm itself -- it just streams the response straight to the panel.
   (i.e. when someone actually scans the manage overlay's share QR), not
   when the button's pressed, so the 30-minute window starts when it's
   actually used
+- `GET /frame/face-labels` -- `{"count": N, "name_0": ..., "x_0": ...,
+  "y_0": ..., ...}` (up to 4 slots) -- named people from Immich's face
+  recognition, positioned in final 800x480 frame pixel space. Only faces
+  Immich already has an identified name for are included (no face
+  detection/recognition happens in this project, see
+  `app/face_labels.py`); `count: 0` if none are named. Used by the
+  device manage button's escalated second menu level
 - `GET /api/queue` -- `{"current": {...} | null, "upcoming": [...]}`, each
   entry an asset id + thumbnail URL; used by the config UI
 - `POST /api/queue/reorder` -- reorders the upcoming queue; body is

server/app/face_labels.py

@@ -0,0 +1,72 @@
+"""Maps named faces (from Immich's own face recognition/People feature)
+onto their position in the final rendered 800x480 frame, for the
+manage-button overlay's escalated "who's in this photo" menu level.
+
+No face detection or recognition happens here or anywhere else in this
+project -- Immich's GET /api/faces?id={assetId} already returns each
+detected face's bounding box plus a nullable `person` object (with a
+`name`, if the user has identified them in Immich); this module only
+does the coordinate math to place a label next to a *named* one.
+"""
+
+from __future__ import annotations
+
+import io
+
+from PIL import Image, ImageOps
+
+from .image_pipeline import EPD_HEIGHT, EPD_WIDTH, _face_aware_crop_box, _plain_center_crop_box
+
+# Small caps, not arbitrary: each label is its own malloc'd overlay
+# buffer on the device (see firmware/main/manage_qr_overlay.c), and the
+# four existing fixed corner regions already use a meaningful chunk of
+# the ESP32-C6's limited RAM. Capping at 4 short names keeps the total
+# overlay memory budget well clear of the WiFi/HTTP stack's own needs.
+MAX_LABELED_FACES = 4
+NAME_MAX_LEN = 10
+
+
+def compute_face_labels(preview_bytes: bytes, faces: list[dict], smart_crop_faces: bool) -> list[dict]:
+    """Returns up to MAX_LABELED_FACES [{"name", "x", "y"}], x/y in final
+    800x480 frame pixel space at each named face's bottom-center point.
+    Faces without an Immich-identified person name are skipped entirely.
+    preview_bytes must be the same preview image render_frame() used for
+    the currently-displayed frame, and smart_crop_faces must match the
+    setting that was active then -- otherwise the crop box computed here
+    won't match what's actually on screen.
+    """
+    named = [face for face in faces if (face.get("person") or {}).get("name")]
+    if not named:
+        return []
+
+    fitted = ImageOps.exif_transpose(Image.open(io.BytesIO(preview_bytes)).convert("RGB"))
+
+    if smart_crop_faces and faces:
+        left, top, right, bottom = _face_aware_crop_box(fitted.width, fitted.height, EPD_WIDTH, EPD_HEIGHT, faces)
+        crop_w, crop_h = right - left, bottom - top
+    else:
+        left, top, crop_w, crop_h = _plain_center_crop_box(fitted.width, fitted.height, EPD_WIDTH, EPD_HEIGHT)
+
+    labels = []
+    for face in named[:MAX_LABELED_FACES]:
+        face_w = face.get("imageWidth") or fitted.width
+        face_h = face.get("imageHeight") or fitted.height
+        scale_x = fitted.width / face_w
+        scale_y = fitted.height / face_h
+
+        center_x = (face["boundingBoxX1"] + face["boundingBoxX2"]) / 2 * scale_x
+        bottom_y = face["boundingBoxY2"] * scale_y
+
+        frame_x = (center_x - left) * (EPD_WIDTH / crop_w)
+        frame_y = (bottom_y - top) * (EPD_HEIGHT / crop_h)
+
+        if not (0 <= frame_x <= EPD_WIDTH and 0 <= frame_y <= EPD_HEIGHT):
+            continue  # this face got cropped out of the final frame entirely
+
+        name = face["person"]["name"]
+        if len(name) > NAME_MAX_LEN:
+            name = name[: NAME_MAX_LEN - 3] + "..."
+
+        labels.append({"name": name, "x": int(frame_x), "y": int(frame_y)})
+
+    return labels

server/app/image_pipeline.py

@@ -34,6 +34,27 @@ def _build_palette_image() -> Image.Image:
 _PALETTE_IMAGE = _build_palette_image()
 
 
+def _plain_center_crop_box(
+    img_width: int, img_height: int, target_width: int, target_height: int
+) -> tuple[float, float, int, int]:
+    """The largest target_width:target_height window centered in the
+    source image -- the same box ImageOps.fit() computes internally when
+    there's no face-aware shift to apply. Returns (left, top, crop_w,
+    crop_h); left/top are floats (not yet rounded) since callers that go
+    on to face-shift this box need the unrounded center point."""
+    target_ratio = target_width / target_height
+    if img_width / img_height > target_ratio:
+        crop_h = img_height
+        crop_w = int(crop_h * target_ratio)
+    else:
+        crop_w = img_width
+        crop_h = int(crop_w / target_ratio)
+
+    left = (img_width - crop_w) / 2
+    top = (img_height - crop_h) / 2
+    return left, top, crop_w, crop_h
+
+
 def _face_aware_crop_box(
     img_width: int, img_height: int, target_width: int, target_height: int, faces: list[dict]
 ) -> tuple[int, int, int, int]:
@@ -63,16 +84,7 @@ def _face_aware_crop_box(
         min_y = min(min_y, face["boundingBoxY1"] * scale_y)
         max_y = max(max_y, face["boundingBoxY2"] * scale_y)
 
-    target_ratio = target_width / target_height
-    if img_width / img_height > target_ratio:
-        crop_h = img_height
-        crop_w = int(crop_h * target_ratio)
-    else:
-        crop_w = img_width
-        crop_h = int(crop_w / target_ratio)
-
-    left = (img_width - crop_w) / 2
-    top = (img_height - crop_h) / 2
+    left, top, crop_w, crop_h = _plain_center_crop_box(img_width, img_height, target_width, target_height)
 
     if max_x - min_x <= crop_w:
         if min_x < left:

server/app/main.py

@@ -15,6 +15,7 @@ from PIL import Image
 from pydantic import BaseModel
 
 from . import config, photo_queue
+from .face_labels import compute_face_labels
 from .image_pipeline import render_frame
 from .immich_client import ImmichClient
 
@@ -163,24 +164,64 @@ def frame_advance():
     return Response(content=_render_asset(client, cfg, cfg.current_asset_id), media_type="application/octet-stream")
 
 
-LOCATION_MAX_LEN = 14
+LOCATION_LINE_MAX_LEN = 14
+
+US_STATE_ABBR = {
+    "alabama": "AL", "alaska": "AK", "arizona": "AZ", "arkansas": "AR", "california": "CA",
+    "colorado": "CO", "connecticut": "CT", "delaware": "DE", "florida": "FL", "georgia": "GA",
+    "hawaii": "HI", "idaho": "ID", "illinois": "IL", "indiana": "IN", "iowa": "IA",
+    "kansas": "KS", "kentucky": "KY", "louisiana": "LA", "maine": "ME", "maryland": "MD",
+    "massachusetts": "MA", "michigan": "MI", "minnesota": "MN", "mississippi": "MS", "missouri": "MO",
+    "montana": "MT", "nebraska": "NE", "nevada": "NV", "new hampshire": "NH", "new jersey": "NJ",
+    "new mexico": "NM", "new york": "NY", "north carolina": "NC", "north dakota": "ND", "ohio": "OH",
+    "oklahoma": "OK", "oregon": "OR", "pennsylvania": "PA", "rhode island": "RI", "south carolina": "SC",
+    "south dakota": "SD", "tennessee": "TN", "texas": "TX", "utah": "UT", "vermont": "VT",
+    "virginia": "VA", "washington": "WA", "west virginia": "WV", "wisconsin": "WI", "wyoming": "WY",
+    "district of columbia": "DC",
+}
+
+CA_PROVINCE_ABBR = {
+    "alberta": "AB", "british columbia": "BC", "manitoba": "MB", "new brunswick": "NB",
+    "newfoundland and labrador": "NL", "northwest territories": "NT", "nova scotia": "NS",
+    "nunavut": "NU", "ontario": "ON", "prince edward island": "PE", "quebec": "QC",
+    "saskatchewan": "SK", "yukon": "YT",
+}
+
+US_COUNTRY_NAMES = {"united states", "united states of america", "usa", "us"}
+CA_COUNTRY_NAMES = {"canada"}
 
 
-def _format_location(exif: dict) -> str | None:
+def _truncate(text: str, max_len: int) -> str:
+    if len(text) <= max_len:
+        return text
+    return text[: max_len - 3] + "..."
+
+
+def _format_location(exif: dict) -> tuple[str, str] | None:
+    """Returns (city_line, region_line), each independently truncated to
+    fit its own corner-overlay line, or None if Immich hasn't geocoded
+    this photo. region_line is the abbreviated state/province for US/CAN
+    locations (e.g. "CA", "ON"), else the full country name."""
     city = exif.get("city")
     if not city:
         return None
+
     state = exif.get("state")
     country = exif.get("country")
-    if state:
-        location = f"{city}, {state}"
+    country_key = (country or "").strip().lower()
+
+    if state and country_key in US_COUNTRY_NAMES:
+        region = US_STATE_ABBR.get(state.strip().lower(), state)
+    elif state and country_key in CA_COUNTRY_NAMES:
+        region = CA_PROVINCE_ABBR.get(state.strip().lower(), state)
     elif country:
-        location = f"{city}, {country}"
+        region = country
+    elif state:
+        region = state
     else:
-        location = city
-    if len(location) > LOCATION_MAX_LEN:
-        location = location[: LOCATION_MAX_LEN - 3] + "..."
-    return location
+        region = ""
+
+    return _truncate(city, LOCATION_LINE_MAX_LEN), _truncate(region, LOCATION_LINE_MAX_LEN)
 
 
 def _format_taken_at(exif: dict) -> str | None:
@@ -217,9 +258,11 @@ def frame_photo_info():
         raise HTTPException(502, f"Could not reach Immich: {e}") from e
 
     exif = asset.get("exifInfo") or {}
+    location = _format_location(exif)
     return {
         "asset_id": cfg.current_asset_id,
-        "location": _format_location(exif),
+        "location_line1": location[0] if location else None,
+        "location_line2": location[1] if location and location[1] else None,
         "taken_at": _format_taken_at(exif),
     }
 
@@ -249,6 +292,55 @@ def frame_share(asset_id: str):
     return RedirectResponse(share_url)
 
 
+@app.get("/frame/face-labels")
+def frame_face_labels():
+    """Named-face positions for the manage button's escalated "level 2"
+    menu -- who's in the current photo, per Immich's own face
+    recognition (no detection/recognition happens here, see
+    app/face_labels.py). Response is a flattened, fixed-slot shape
+    (name_0/x_0/y_0, name_1/x_1/y_1, ...) rather than a JSON array, so
+    the device's hand-rolled parser can read it with the same flat-
+    scalar helpers it already has, instead of needing a real array
+    parser. Empty (count: 0) if no faces are named, or if anything about
+    fetching them fails -- this is a "nice to have" addition to the
+    overlay, not worth failing the whole menu over."""
+    cfg = config.load()
+    _require_configured(cfg)
+
+    client = ImmichClient(cfg.immich_url, cfg.immich_api_key)
+    assets = _list_assets(client, cfg)
+
+    if photo_queue.get_current(cfg, assets):
+        config.save(cfg)
+
+    if not cfg.current_asset_id:
+        return {"count": 0}
+
+    try:
+        faces = client.get_asset_faces(cfg.current_asset_id)
+    except httpx.HTTPError as e:
+        logger.warning("Could not fetch faces for asset %s: %s", cfg.current_asset_id, e)
+        return {"count": 0}
+
+    if not any((face.get("person") or {}).get("name") for face in faces):
+        return {"count": 0}  # skip the extra preview download in the common no-named-faces case
+
+    try:
+        preview_bytes = client.download_asset_preview(cfg.current_asset_id)
+    except httpx.HTTPError as e:
+        logger.warning("Could not download asset %s for face-label mapping: %s", cfg.current_asset_id, e)
+        return {"count": 0}
+
+    labels = compute_face_labels(preview_bytes, faces, cfg.smart_crop_faces)
+
+    result: dict[str, object] = {"count": len(labels)}
+    for i, label in enumerate(labels):
+        result[f"name_{i}"] = label["name"]
+        result[f"x_{i}"] = label["x"]
+        result[f"y_{i}"] = label["y"]
+    return result
+
+
 @app.get("/api/queue")
 def api_queue():
     cfg = config.load()