TL;DR

If you’ve read that Rockchip’s rknn-toolkit2 model converter “only runs on x86,” that was true once and isn’t anymore. On version 2.3.2 the official aarch64 wheel installs, imports, and converts an ONNX model to a .rknn end-to-end — I did it on a plain arm64 Ubuntu 24.04 host (Python 3.12), no x86 anywhere, no cross-machine dance.

The conversion itself is boring, which is the point. What isn’t boring is the two things that stop you before you get there — both undocumented, both a fast dead end if you don’t know them:

1. pip install 'setuptools<81'. rknn-toolkit2 2.3.2 imports pkg_resources at load; modern setuptools deleted it, so from rknn.api import RKNN blows up before you write a line of your own code.
2. Your output .rknn won’t match anyone else’s md5 — and that’s normal. rknn files aren’t byte-reproducible. If you “verify” a fresh conversion by diffing checksums against a reference, you’ll scare yourself for no reason. The real check is elsewhere.

The rest is the walk-through and the reasoning.

TL;DR

Goal: run MobileNet on the RK3576’s NPU through the open rocket driver + Mesa Teflon — the same stack is byte-perfect on the RK3588, so the bug is RK3576-specific. A month of all-zero output later:

• The int8 convolution is byte-correct now. The “all-grey” wall was a fixed-point bug: the rescale multiplier went out at Q14 where the chip wants Q4 — 2¹⁰ too hot, so every pixel saturated. Fix that (plus a pad value and a bias term) and a single conv matches the CPU reference byte-for-byte.
• MobileNet end-to-end still returns zero, behind what looked like two walls but turned out to be one, living below the registers: the command stream I send is byte-identical to the vendor’s and the chip still behaves differently. The wall is multi-task dispatch — the compute units won’t re-arm themselves for each task the way the vendor’s do — and the depthwise, which I’d taken for a second separate wall, is just the first layer wide enough to be forced through it (I confirmed that by instrumenting the vendor’s own driver and watching its tiled depthwise run aligned and correct). Ordinary single-task convolutions compute fine on the same path.
• So the open driver is exonerated — every byte I hand the chip matches the vendor’s; the gap is in silicon state, below what software can observe on either side. I even booted the driver on top of a mainline OP-TEE to rule out the firmware — same failure. The way around (not through): the wall only bites multi-task jobs, so send each row-tile as its own single-task job, which the hardware runs. That’s the next build.

The rest is the long version — mostly me being wrong, in order.