On-Device AI

AI models that run on the user's own device, in an app or in the browser, so the input stays there and no server does the work.

Topic Created Also in German: KI auf dem eigenen Gerät

Reviewed by adrian on

On-device AI runs a model where the data already is: on a phone, a laptop or in a browser tab, instead of sending the input to a server and waiting for the answer. The input, such as a photo, a recording or a text, never leaves the device. The price is size: the model and its engine have to be small enough to download and fast enough on ordinary hardware, which usually means compact, specialised models instead of large general ones. Runtimes such as WebAssembly make this possible in the browser without installing anything. This topic collects projects and tools that run models locally.

Based on

More on this

Covered in

  • Image Voice Memos Native macOS app that pairs photos with voice memos and transcribes them on the device with Apple's speech recognition.
  • What the Machine Sees A small app from my old website: choose a photo, and the vision model Mirage found objects, text and the scene in it, entirely in the browser.

Part of

  • AI / Language Models What I use language models for: agents, local models, RAG and small programs in which a model handles one task.

Show in the graph view