On-device intelligence and the future of private AI
Privacy is a decision about where the model runs, made before the first prompt is sent.
5 min read
Once a photo, a workout, or a health note leaves the phone, the privacy policy is a story about a decision you already made. We prefer to make that decision in the architecture.
On-device models are smaller, sometimes less capable, and much easier to explain to the person holding the phone. For a training log or a skin photo, that trade is usually the right one. The model that runs in your pocket cannot be breached from a bucket you do not control.
What stays, and what may leave
We start every health-adjacent brief with a list: values that must never leave the device, values that can be aggregated, and values we should not collect. The list is short on purpose. If a feature needs the third category, the feature has to justify itself in writing.
Cloud inference still has a place. A large model, a shared index, a job that needs data the device does not have. When we use it, the request is specific, logged, and deletable. “We sent the context to the model” is not a design.
Capability is not the only requirement
Teams adopt the largest model they can afford because the demo looks better. The product then inherits a latency, a bill, and a data flow nobody drew. We size the model to the decision. If a smaller on-device model answers the decision, it is the better system.