# ADR 0012: Unofficial aarch64 Android toolchain on the mesh runner for on-device lab tests

**Status:** Accepted
**Date:** 2026-05-20

---

## Context

peat-mesh's `actions-runner-mesh` self-hosted runner lives on the GB10 host
(`peat-arm64-linux-gb10-mesh`, runner id 21), an NVIDIA Grace **aarch64**
Linux box. The runner is the staging point for on-device Android
verification: building APKs, pushing them to lab devices over ADB, and
running instrumented tests against real hardware (per `CLAUDE.md` device
test inventory).

The PR-gating job intentionally **does not** run on this host. It runs
on `ubuntu-latest` with the official Google NDK
(`ndk;27.0.12077973` via `android-actions/setup-android@v3`) and the
KVM x86_64 emulator — see PR #139 and
`docs/plans/android-functional-test.md`. That path stays as-is.

What the mesh runner needs that `ubuntu-latest` cannot provide:

1. A native aarch64 toolchain — `sdkmanager` installs `linux-x86_64`
   binaries (clang/lld for NDK, aapt2 and friends for build-tools)
   that fail on the GB10 with `cannot execute binary file: Exec format
   error`. The host has no x86_64 emulation layer, and adding
   `qemu-user-static` or `box64` would put an emulator in the build
   path for marginal benefit versus running on the right architecture.
2. Physical-device reach — the lab Samsung tablet (`SM-X210`) is
   connected to the mesh runner via USB; ADB from a GitHub-hosted
   runner is not viable.

Google **does not ship a native Linux aarch64 build of any Android
host tool** — NDK, SDK build-tools (which contain AAPT2), or
platform-tools. Every official Linux release contains only
`linux-x86_64` prebuilts. NDK GitHub issue #1752 has tracked this
request as open since August 2022; the maintainer's most recent
position (June 2024) is "no plans, the infrastructure we need to make
it happen just doesn't exist." The same gap applies to AAPT2 — the
Android Asset Packaging Tool that processes resources during APK
packaging.

This contrasts with Apple Silicon (`darwin-arm64`), for which Google
ships native builds of NDK and AAPT2 since NDK r24 (2022). The
asymmetry reflects investment priorities, not technical capability.

## Decision

The mesh runner uses two community-maintained aarch64-linux-musl
rebuilds from the same maintainer (`HomuHomu833`):

| Component | Source | Pinned version |
|---|---|---|
| NDK | [HomuHomu833/android-ndk-custom](https://github.com/HomuHomu833/android-ndk-custom) | `r27d` (release tag `r27`) |
| SDK (build-tools, cmdline-tools, platform-tools) | [HomuHomu833/android-sdk-custom](https://github.com/HomuHomu833/android-sdk-custom) | `36.0.2` |

**Pinned digests** (SHA256 as returned by the GitHub releases API):

- `android-ndk-r27d-aarch64-linux-musl.tar.xz` — SHA256
  `457bbc5da8959a90fec3ceabf82bd922293231ec9a15a083df992e107587a157`
- `android-sdk-aarch64-linux-musl.tar.xz` — SHA256
  `5a70f38173ae4e465c661e600e7b91aef2bb2b8f3355054e401631c668ecd28f`

Both projects rebuild only the **host** toolchain binaries (clang,
lld, llvm for the NDK; aapt2, adb, sdkmanager, etc. for the SDK) with
[Zig's LLVM backend](https://github.com/HomuHomu833/llvm-custom)
against musl libc so they run on any aarch64 Linux distribution. The
**target sysroot** (Android bionic) and **resource formats** (AAPT2
proto/binary) are unchanged from Google's official releases, so
produced artifacts are byte-compatible with official-toolchain builds.

Both rebuilds from the same maintainer means **one community trust
boundary, not two**.

### AAPT2 plumbing

AGP downloads its own aapt2 jar from Google's Maven repo
(`aapt2-<v>-linux.jar`) and unpacks it into Gradle's transform cache.
That binary is `linux-x86_64` and cannot run on this host. The fix
is the AGP-supported Gradle property:

```
android.aapt2FromMavenOverride=/home/kit/Android/Sdk/build-tools/36.1.0/aapt2
```

Set at **user level** in `~/.gradle/gradle.properties` so every
Gradle invocation on this host picks it up automatically without
per-project plumbing. **Not** committed to any project's
`gradle.properties` — CI on `ubuntu-latest` must continue to use
Google's bundled AAPT2.

Verified against AGP 8.13.0. Bumping AGP requires re-running the
verified invocation; if the new AGP expects a newer AAPT2 protocol,
refresh the HomuHomu SDK pin and update this ADR.

## Scope boundary

This toolchain is for **lab use on the mesh runner only**.

- **PR-gating CI** (`android-test` job in `.github/workflows/ci.yml`)
  stays on `ubuntu-latest` with the official Google NDK and Google
  AAPT2. It is the only Android job that runs per-PR and is the
  regression gate for bug classes like peat-mesh#134 and #136.
- **Release publishing** (any future Maven Central or
  user-distributable artifact) **must** continue to build with the
  official Google toolchain on `ubuntu-latest` — mirroring
  `peat-btle/.github/workflows/publish-maven.yml`. Shipped artifacts
  that touch security primitives (Ed25519, X25519, ChaCha20-Poly1305,
  HMAC-SHA256, per `CLAUDE.md`) should be produced by a toolchain
  whose provenance the user community can independently verify
  against Google.
- **Lab work on the mesh runner** — manual cross-compile, JNI
  builds, on-device instrumented tests pushed to lab devices via
  ADB — uses the unofficial toolchain because no official path
  exists.

The unofficial toolchain produces artifacts that exist only inside
the lab.

## Consequences

**Positive**

- End-to-end build + deploy + instrumented test against the lab
  Samsung tablet (`SM-X210`) verified on 2026-05-20: `./gradlew
  connectedCheck -PandroidTestAbis=arm64-v8a` on the mesh runner
  compiles the JNI shim for `aarch64-linux-android`, packages the
  test APK, installs it on the tablet over ADB, and runs the
  instrumented sync-protocol test. Total wall clock 21s after the
  toolchain swap.
- The HomuHomu NDK ships `toolchains/llvm/prebuilt/linux-x86_64` as
  a **symlink to `linux-arm64`**, so Gradle projects that hardcode
  the `linux-x86_64/bin` path (including
  `android-tests/build.gradle.kts`) resolve transparently to the
  aarch64 binaries with no patches required.
- ADB is native aarch64 from two independent sources (HomuHomu SDK
  `platform-tools/adb` and Ubuntu's
  `android-sdk-platform-tools-common`). Either works.
- Test execution path on this host is **on-device only**, by
  design. The Android emulator is not viable — Google ships
  `emulator` for `linux-x86_64` only. Instrumented tests on the
  mesh runner reach hardware via ADB (USB or WiFi), not via
  emulation.

**Negative**

- Toolchain has no Google attestation. We rely on GitHub's release
  immutability and the asset SHA256 digests for integrity, not
  signed builds.
- Single-maintainer organization (HomuHomu833) provides both
  rebuilds. Two distinct risks:
  - **Maintainer stalls** — fallback is to build the toolchains
    from source ourselves (the Zig+musl build scripts are open).
  - **Pipeline compromise** — both rebuilds flow through one
    GitHub Actions release pipeline; the asset SHA pin guards
    against tampering *after* we adopt a release, not against
    adopting a malicious release in the first place. Mitigation
    (independent provenance attestation, second-source NDK such
    as `SnowNF/ndk-aarch64-linux`, in-house build verification
    against an independently-built copy before pin bumps) is an
    open design item; this ADR does not resolve it.
- One patch-level skew from `peat-btle`'s release pin (NDK r27d on
  mesh runner vs r27.0 in peat-btle's release workflow).
  Acceptable because they serve different purposes (lab vs
  release).

**Operational state captured in** `Peat/RUNNERS.md` (workspace-level — sibling to this repo under the `Peat/` super-directory; not part of this repo's tree).
The runner's `.env` carries `ANDROID_NDK_HOME` and the linux-arm64
toolchain on `PATH`; the AAPT2 override lives in
`~/.gradle/gradle.properties`. No CI workflow targets this runner
for Android today; future on-device jobs that do should use
`runs-on: [self-hosted, android]` and remain explicitly scoped per
the boundary above.
