Now you can run a huge model with 35 billion parameters directly on an iPhone — the open-source Edge0 engine has been…

Now you can run a huge model with 35 billion parameters directly on an iPhone — the open-source Edge0 engine has been introduced. It enables working with local neural networks based on the MoE architecture without creating a large load on RAM — at peak moments, only 1 to 2.5 GB is used. A trick is used for optimization: instead of RAM, the device's flash memory is utilized. Models with 8 billion parameters (https://huggingface.co/Edge0/Edge0-8B-A1B-preview) and 35 billion (https://huggingface.co/Edge0/Edge0-35B-A3B-preview) are available; the second version requires an iPhone 15 Pro or a newer model. The source code can be downloaded from GitHub (https://github.com/Edge0-AI/edge0/).