On-Device & Edge AI
Mobile NPUs, memory-bound inference on consumer silicon, compilation targets and privacy-driven local models.
5concepts
60flashcards
37minutes of reading
- 01 Federated Learning on Edge Devices Training a shared model without collecting the data, why non-IID client distributions break the averaging assumption, and the systems constraints that decide what is actually trainable.
- 02 On-Device LLM Inference Constraints The arithmetic that decides whether a language model can run on a phone, why bandwidth rather than compute is the binding constraint, and what the KV cache does to the memory budget.