Add build-firmware skill: native ESP-IDF build, no Docker needed
CI builds firmware inside the espressif/idf Docker image, but this sandbox can't run containers at all -- it strips cap_sys_admin (and blocks unshare) from the capability set even for root, which container image-layer extraction and namespace setup both need. Confirmed by hand: docker.io installs and dockerd starts fine, but even a bare `docker run hello-world` fails to extract its own layer. Works around it by installing ESP-IDF natively instead (git clone + its own install.sh, scoped to just this project's esp32c6 target) -- the same way a developer would set it up on their own machine, needing nothing this sandbox disallows. Verified end-to-end: both board variants (devkit, xiao) build clean from a fresh checkout via the packaged setup.sh/build.sh.
This commit is contained in:
@@ -0,0 +1,128 @@
|
||||
---
|
||||
name: build-firmware
|
||||
description: Compile the espresso_frame ESP32-C6 firmware (firmware/) for both board variants without Docker -- a native, non-container ESP-IDF v6.0 install. Use when asked to build the firmware, verify a firmware/main/*.c change actually compiles, or check both the devkit and xiao board targets.
|
||||
---
|
||||
|
||||
Compiles `firmware/` (ESP-IDF, targeting ESP32-C6) locally, without
|
||||
Docker -- CI's `firmware-build-check.yml`/`firmware-release-build.yml`
|
||||
build inside the `espressif/idf:release-v6.0` container image, but
|
||||
**this sandbox cannot run containers at all**: `docker.io` installs and
|
||||
`dockerd` starts fine even as root, but the sandbox strips
|
||||
`cap_sys_admin` (and blocks the bare `unshare` syscall) from the
|
||||
capability set regardless of uid, which container image-layer
|
||||
extraction and namespace setup both require. Confirmed by hand:
|
||||
`docker run hello-world` fails to extract even the tiny hello-world
|
||||
layer ("failed to extract layer... operation not permitted" with the
|
||||
overlayfs snapshotter; "unshare: operation not permitted" even with
|
||||
the vfs storage driver instead). This is a hard restriction of the
|
||||
sandbox itself, not a permissions/setup problem -- don't spend time
|
||||
re-trying `--privileged`-equivalent flags or alternate storage drivers,
|
||||
none of it routes around a missing `cap_sys_admin`.
|
||||
|
||||
The workaround: skip containers entirely and install ESP-IDF the same
|
||||
way a developer would set it up on their own machine (`git clone` +
|
||||
ESP-IDF's own `install.sh`) -- that path needs nothing this sandbox
|
||||
disallows, just normal file/process operations.
|
||||
|
||||
## Setup (once per fresh container)
|
||||
|
||||
```bash
|
||||
bash .claude/skills/build-firmware/setup.sh
|
||||
```
|
||||
|
||||
Installs (via real `apt-get` -- this container actually has root and a
|
||||
working package manager, unlike run-server's Chromium bootstrap which
|
||||
had neither):
|
||||
- OS build deps: `python3`/`venv`/`pip`, `cmake`, `ninja-build`,
|
||||
`flex`/`bison`/`gperf`, `build-essential`, `libusb-1.0-0`.
|
||||
- ESP-IDF itself: a shallow, single-branch, recursive-submodule clone
|
||||
of `release/v6.0` (~700MB) into `~/.espressif-idf/esp-idf` -- matches
|
||||
the IDF version CI's Docker image pins. Only clones once; re-running
|
||||
`setup.sh` never touches an existing checkout.
|
||||
- The esp32c6 toolchain + Python venv, via ESP-IDF's own
|
||||
`./install.sh esp32c6` -- scoped to just this project's one target
|
||||
(see `firmware/README.md`'s board table), not every chip ESP-IDF
|
||||
supports, to keep the download/disk footprint down. `install.sh` is
|
||||
already idempotent on its own, so `setup.sh` always calls it rather
|
||||
than duplicating that check -- a re-run costs a few seconds once
|
||||
everything's cached.
|
||||
|
||||
Takes a few minutes on a cold run (mostly `install.sh`'s own pip/tool
|
||||
downloads), well under a minute on a re-run. Needs real root (`apt-get
|
||||
install`) -- if this container ever runs as non-root, this setup
|
||||
doesn't apply as-is (would need the same non-root apt-download +
|
||||
`dpkg-deb -x` extraction dance `run-server`'s `setup.sh` uses for
|
||||
Chromium).
|
||||
|
||||
Disk: budget ~4GB free before starting (esp-idf checkout + toolchain +
|
||||
Python env land around 3.4GB in `~/.espressif`, plus the ~700MB
|
||||
checkout itself). Confirmed working with as little as ~7GB free.
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
bash .claude/skills/build-firmware/build.sh # devkit (default)
|
||||
bash .claude/skills/build-firmware/build.sh xiao
|
||||
bash .claude/skills/build-firmware/build.sh both # both variants
|
||||
```
|
||||
|
||||
Each board gets its own build directory and generated sdkconfig (see
|
||||
`firmware/build_for_board.sh`'s own comment) -- building one never
|
||||
disturbs the other. `build.sh` auto-runs `set-target esp32c6` the very
|
||||
first time a board is built (no generated sdkconfig yet); later builds
|
||||
skip straight to `idf.py build`. Extra arguments pass straight through
|
||||
to `idf.py`, e.g.:
|
||||
|
||||
```bash
|
||||
bash .claude/skills/build-firmware/build.sh xiao flash -p /dev/ttyUSB0
|
||||
```
|
||||
|
||||
`flash`/`monitor` need an actual attached device and serial port --
|
||||
this sandbox has neither, so those only work when this skill runs
|
||||
somewhere hardware is actually plugged in (a real dev machine, or a
|
||||
differently-configured environment with device passthrough).
|
||||
|
||||
A clean build of one board takes ~30s once the target's already been
|
||||
configured (~1,000 build steps total split across both boards, most of
|
||||
it ESP-IDF's own components -- this project's own `firmware/main/*.c`
|
||||
and `firmware/components/*` sources are a small fraction of that and
|
||||
compile in a few seconds). Output lands at
|
||||
`firmware/build/espresso_frame.bin` (devkit) or
|
||||
`firmware/build_xiao/espresso_frame.bin` (xiao) -- both paths are
|
||||
gitignored (`firmware/.gitignore`... actually the repo root
|
||||
`.gitignore`'s "ESP-IDF firmware build output" section), so nothing
|
||||
here needs cleaning up before a commit.
|
||||
|
||||
## Verified
|
||||
|
||||
Both board variants (`devkit` set-target esp32c6 + build, `xiao`
|
||||
set-target esp32c6 + build) built successfully end-to-end using this
|
||||
exact setup.sh/build.sh pair, producing real
|
||||
`espresso_frame.bin` images with normal free-space margins (41%/36%
|
||||
of their respective app partitions) and no errors -- only one
|
||||
pre-existing, unrelated warning (`battery.c`'s unused `TAG` when that
|
||||
file's logging is compiled out). This is a real compile check, not
|
||||
just a syntax read -- if a future change breaks the build, this skill
|
||||
will actually catch it.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **`docker: ... unshare: operation not permitted` / `failed to
|
||||
extract layer ... operation not permitted`**: expected in this
|
||||
sandbox, see the top of this file. Don't debug it further -- use this
|
||||
skill's native install instead.
|
||||
- **`ESP-IDF not found at ... -- run setup.sh first`**: `build.sh`'s
|
||||
own check for a missing `$IDF_DIR/export.sh` -- run `setup.sh` (see
|
||||
above) before the first build.
|
||||
- **`idf.py: command not found` if you try to run it directly**: same
|
||||
gotcha `firmware/build_for_board.sh` already documents -- `idf.py` is
|
||||
normally a shell *function* from ESP-IDF's `export.sh`, not on PATH
|
||||
as a real executable, so it isn't inherited into a script's own
|
||||
subshell even after sourcing `export.sh` in your interactive shell
|
||||
first. Use `build.sh` (or `build_for_board.sh`, which calls
|
||||
`python "$IDF_PATH/tools/idf.py"` directly) instead of typing
|
||||
`idf.py` in a fresh script/subshell.
|
||||
- **Disk pressure during `install.sh`**: this environment runs close to
|
||||
full (single-digit GB free is normal, not a sign of a leak) --
|
||||
`df -h /` before running `setup.sh` if a build mysteriously fails
|
||||
partway with a "no space left on device"-shaped error.
|
||||
Executable
+63
@@ -0,0 +1,63 @@
|
||||
#!/usr/bin/env bash
|
||||
# Builds (or flashes/monitors, if a serial port is actually attached)
|
||||
# the espresso_frame firmware for one board variant, via the project's
|
||||
# own firmware/build_for_board.sh -- this script just sources the
|
||||
# ESP-IDF environment first and auto-runs `set-target esp32c6` on a
|
||||
# board's very first build (a fresh clone has no generated sdkconfig
|
||||
# yet, same reasoning as CI's own build steps -- see firmware/README.md's
|
||||
# "Building for the Seeed XIAO ESP32-C6" section).
|
||||
#
|
||||
# Usage:
|
||||
# build.sh # build devkit (default)
|
||||
# build.sh devkit
|
||||
# build.sh xiao
|
||||
# build.sh both # build both board variants
|
||||
# build.sh xiao flash -p /dev/ttyUSB0 # only meaningful with real hardware attached
|
||||
set -euo pipefail
|
||||
|
||||
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
firmware_dir="$(git -C "$script_dir" rev-parse --show-toplevel)/firmware"
|
||||
|
||||
IDF_DIR="$HOME/.espressif-idf/esp-idf"
|
||||
if [ ! -f "$IDF_DIR/export.sh" ]; then
|
||||
echo "ESP-IDF not found at $IDF_DIR -- run setup.sh first" >&2
|
||||
exit 1
|
||||
fi
|
||||
# export.sh is chatty and assumes an interactive shell prompt in spots;
|
||||
# redirect its own stdout, not ours, so build.sh's actual output (and a
|
||||
# real failure's stderr) stays visible.
|
||||
source "$IDF_DIR/export.sh" > /dev/null
|
||||
|
||||
cd "$firmware_dir"
|
||||
|
||||
build_one() {
|
||||
local board="$1"
|
||||
shift
|
||||
local sdkconfig
|
||||
case "$board" in
|
||||
devkit) sdkconfig="sdkconfig" ;;
|
||||
xiao) sdkconfig="sdkconfig.xiao_local" ;;
|
||||
*) echo "Unknown board '$board' -- expected 'devkit' or 'xiao'" >&2; exit 1 ;;
|
||||
esac
|
||||
|
||||
if [ ! -f "$sdkconfig" ]; then
|
||||
echo "==> $board: no generated sdkconfig yet, setting target esp32c6"
|
||||
./build_for_board.sh "$board" set-target esp32c6
|
||||
fi
|
||||
|
||||
local args=("$@")
|
||||
if [ ${#args[@]} -eq 0 ]; then
|
||||
args=(build)
|
||||
fi
|
||||
./build_for_board.sh "$board" "${args[@]}"
|
||||
}
|
||||
|
||||
board="${1:-devkit}"
|
||||
shift || true
|
||||
|
||||
if [ "$board" = "both" ]; then
|
||||
build_one devkit "$@"
|
||||
build_one xiao "$@"
|
||||
else
|
||||
build_one "$board" "$@"
|
||||
fi
|
||||
Executable
+58
@@ -0,0 +1,58 @@
|
||||
#!/usr/bin/env bash
|
||||
# One-time (idempotent) environment bootstrap for compiling the
|
||||
# espresso_frame firmware (firmware/) without Docker -- see this
|
||||
# skill's SKILL.md for why not Docker, even though that's what CI uses.
|
||||
# Re-run any time; every step is safe/fast to repeat once already done.
|
||||
set -euo pipefail
|
||||
|
||||
IDF_ROOT="$HOME/.espressif-idf"
|
||||
IDF_DIR="$IDF_ROOT/esp-idf"
|
||||
# Matches the espressif/idf:release-v6.0 image CI's
|
||||
# firmware-build-check.yml/firmware-release-build.yml use -- keep this
|
||||
# in sync with those workflow files if the project's pinned IDF version
|
||||
# ever changes.
|
||||
IDF_BRANCH="release/v6.0"
|
||||
|
||||
# 1. OS packages ESP-IDF's own install.sh needs (python3 + venv/pip,
|
||||
# cmake, ninja, a C toolchain for the odd host-side code generator, git
|
||||
# for the clone below, flex/bison/gperf for mbedtls/etc.'s generated
|
||||
# parsers, libusb for esptool's USB/JTAG bits even though this skill
|
||||
# doesn't flash real hardware). Installed via apt with real root --
|
||||
# unlike run-server's Chromium bootstrap, this container actually has
|
||||
# root and a working apt, so no non-root extraction dance is needed
|
||||
# here.
|
||||
PKGS="git python3 python3-venv python3-pip cmake ninja-build ccache libusb-1.0-0 wget flex bison gperf build-essential"
|
||||
missing=()
|
||||
for pkg in $PKGS; do
|
||||
dpkg -s "$pkg" >/dev/null 2>&1 || missing+=("$pkg")
|
||||
done
|
||||
if [ ${#missing[@]} -gt 0 ]; then
|
||||
echo "installing OS packages: ${missing[*]}"
|
||||
apt-get update
|
||||
DEBIAN_FRONTEND=noninteractive apt-get install -y "${missing[@]}"
|
||||
fi
|
||||
|
||||
# 2. ESP-IDF checkout -- shallow, single branch, recursive submodules
|
||||
# also shallow (~700MB total, vs. several GB for a full clone). Only
|
||||
# clones once; re-running this script never re-clones or resets it, so
|
||||
# any local changes you made for debugging survive a re-run.
|
||||
if [ ! -d "$IDF_DIR/.git" ]; then
|
||||
echo "cloning esp-idf $IDF_BRANCH into $IDF_DIR ..."
|
||||
mkdir -p "$IDF_ROOT"
|
||||
git clone --branch "$IDF_BRANCH" --depth 1 --shallow-submodules --recursive \
|
||||
https://github.com/espressif/esp-idf.git "$IDF_DIR"
|
||||
else
|
||||
echo "esp-idf already cloned at $IDF_DIR"
|
||||
fi
|
||||
|
||||
# 3. Toolchain + Python virtualenv, scoped to esp32c6 only -- this
|
||||
# project's one target (see firmware/README.md's board table). Scoping
|
||||
# avoids downloading toolchains for every chip ESP-IDF supports, which
|
||||
# matters given this container's disk headroom. install.sh is already
|
||||
# idempotent on its own (checks what's present and skips it), so this
|
||||
# always calls it rather than trying to duplicate that check here --
|
||||
# a re-run only costs a few seconds once everything's cached.
|
||||
echo "running esp-idf install.sh esp32c6 (fast if already installed) ..."
|
||||
(cd "$IDF_DIR" && ./install.sh esp32c6)
|
||||
|
||||
echo "setup complete -> $IDF_DIR/export.sh (build.sh sources this for you)"
|
||||
Reference in New Issue
Block a user