content.raw: There are 2.2 billion active 
content.type: text
content.value: There are 2.2 billion active 
content.raw: @Apple
content.type: mention
content.user: Apple
content.raw:  devices 🍏 and all of them just got smarter thanks to Apple Intelligence (AI) 🧠
content.type: text
content.value:  devices 🍏 and all of them just got smarter thanks to Apple Intelligence (AI) 🧠
content.type: new_line
content.type: new_line
content.raw: Well, almost all devices... 🤔
content.type: text
content.value: Well, almost all devices... 🤔
content.type: new_line
content.type: new_line
content.raw: Your device needs:
content.type: text
content.value: Your device needs:
content.type: new_line
content.raw: - A17 Pro chip or later if it's an iPhone 📱,
content.type: text
content.value: - A17 Pro chip or later if it's an iPhone 📱,
content.type: new_line
content.raw: - M1 chip or later if iPad 📱,
content.type: text
content.value: - M1 chip or later if iPad 📱,
content.type: new_line
content.raw: - M1 chip or later if Mac 💻.
content.type: text
content.value: - M1 chip or later if Mac 💻.
content.type: new_line
content.type: new_line
content.raw: All this aside, this is probably the largest deployment of on-device LLMs 🌍.
content.type: text
content.value: All this aside, this is probably the largest deployment of on-device LLMs 🌍.
content.type: new_line
content.type: new_line
content.raw: Here is the technical goodness:
content.type: text
content.value: Here is the technical goodness:
content.type: new_line
content.raw: - AI will run ~3B LLM on device (Mac, iPhone, iPad) with grouped-query-attention, activation, and embedding quantization (Talaria bit rate selection) running on the neural engine 🚀.
content.type: text
content.value: - AI will run ~3B LLM on device (Mac, iPhone, iPad) with grouped-query-attention, activation, and embedding quantization (Talaria bit rate selection) running on the neural engine 🚀.
content.type: new_line
content.raw: - Will be using fine-tuned LoRA Adapters for different tasks, claiming to outperform other 7B and 3B LLMs! 🥇
content.type: text
content.value: - Will be using fine-tuned LoRA Adapters for different tasks, claiming to outperform other 7B and 3B LLMs! 🥇
content.type: new_line
content.raw: - iPhone 15 Pro 0.6 ms time-to-first-token with 30 tokens/second latency ⏱.
content.type: text
content.value: - iPhone 15 Pro 0.6 ms time-to-first-token with 30 tokens/second latency ⏱.
content.type: new_line
content.raw: - No server model size or details 🤐.
content.type: text
content.value: - No server model size or details 🤐.
content.type: new_line
content.raw: - Will be dynamically loading, caching, and swapping LoRA adapters (think LoRA Land) 🔄.
content.type: text
content.value: - Will be dynamically loading, caching, and swapping LoRA adapters (think LoRA Land) 🔄.
content.type: new_line
content.raw: - On-device model has 49K vocab size, while the server model goes 100K 📚.
content.type: text
content.value: - On-device model has 49K vocab size, while the server model goes 100K 📚.
content.type: new_line
content.raw: - Using rejection sampling fine-tuning and RLHF in post-processing 📈.
content.type: text
content.value: - Using rejection sampling fine-tuning and RLHF in post-processing 📈.
content.type: new_line
content.raw: - A rejection sampling fine-tuning algorithm with teacher committee 🎓.
content.type: text
content.value: - A rejection sampling fine-tuning algorithm with teacher committee 🎓.
content.type: new_line
content.raw: - And reinforcement learning from human feedback (RLHF) algorithm with mirror descent policy optimization and a leave-one-out advantage estimator 🧮.
content.type: text
content.value: - And reinforcement learning from human feedback (RLHF) algorithm with mirror descent policy optimization and a leave-one-out advantage estimator 🧮.
content.type: new_line
content.raw: - Used synthetic data generation (from bigger models, does not mention which) for tasks like summaries 📝.
content.type: text
content.value: - Used synthetic data generation (from bigger models, does not mention which) for tasks like summaries 📝.
content.type: new_line
content.raw: - 750 evaluation samples for each production use case to evaluate summarization (dataset not released) 📊.
content.type: text
content.value: - 750 evaluation samples for each production use case to evaluate summarization (dataset not released) 📊.
content.type: new_line
content.raw: - No mention of multilingual support 🌐.
content.type: text
content.value: - No mention of multilingual support 🌐.
content.type: new_line
content.raw: - Used Apple's AXLearn framework (JAX) and FSP to train on TPUs and GPUs 💪.
content.type: text
content.value: - Used Apple's AXLearn framework (JAX) and FSP to train on TPUs and GPUs 💪.
content.type: new_line
content.raw: - 3B + Adapter outperforms Phi-3 mini, Gemma 7B, Mistral 7B on summarization 🏆.
content.type: text
content.value: - 3B + Adapter outperforms Phi-3 mini, Gemma 7B, Mistral 7B on summarization 🏆.
content.type: new_line
content.raw: - 3B + Adapter achieves 78.7% on IFEval beating Phi-3 mini, Gemma 7B, Mistral 7B; Server Model matches GPT-4-Turbo and beats Mixtral 8x22B and GPT-3.5-turbo ✨.
content.type: text
content.value: - 3B + Adapter achieves 78.7% on IFEval beating Phi-3 mini, Gemma 7B, Mistral 7B; Server Model matches GPT-4-Turbo and beats Mixtral 8x22B and GPT-3.5-turbo ✨.
content.type: new_line
content.type: new_line
content.raw: LoRA for the win! 🎉
content.type: text
content.value: LoRA for the win! 🎉
content.type: new_line
content.type: new_line
content.raw: Blog: 
content.type: text
content.value: Blog: 
content.href: https://machinelearning.apple.com/research/introducing-apple-foundation-models
content.raw: https://machinelearning.apple.com/research/introducing-apple-foundation-models
content.type: link
rawContent: There are 2.2 billion active @Apple devices 🍏 and all of them just got smarter thanks to Apple Intelligence (AI) 🧠
  
  Well, almost all devices... 🤔
  
  Your device needs:
  - A17 Pro chip or later if it's an iPhone 📱,
  - M1 chip or later if iPad 📱,
  - M1 chip or later if Mac 💻.
  
  All this aside, this is probably the largest deployment of on-device LLMs 🌍.
  
  Here is the technical goodness:
  - AI will run ~3B LLM on device (Mac, iPhone, iPad) with grouped-query-attention, activation, and embedding quantization (Talaria bit rate selection) running on the neural engine 🚀.
  - Will be using fine-tuned LoRA Adapters for different tasks, claiming to outperform other 7B and 3B LLMs! 🥇
  - iPhone 15 Pro 0.6 ms time-to-first-token with 30 tokens/second latency ⏱.
  - No server model size or details 🤐.
  - Will be dynamically loading, caching, and swapping LoRA adapters (think LoRA Land) 🔄.
  - On-device model has 49K vocab size, while the server model goes 100K 📚.
  - Using rejection sampling fine-tuning and RLHF in post-processing 📈.
  - A rejection sampling fine-tuning algorithm with teacher committee 🎓.
  - And reinforcement learning from human feedback (RLHF) algorithm with mirror descent policy optimization and a leave-one-out advantage estimator 🧮.
  - Used synthetic data generation (from bigger models, does not mention which) for tasks like summaries 📝.
  - 750 evaluation samples for each production use case to evaluate summarization (dataset not released) 📊.
  - No mention of multilingual support 🌐.
  - Used Apple's AXLearn framework (JAX) and FSP to train on TPUs and GPUs 💪.
  - 3B + Adapter outperforms Phi-3 mini, Gemma 7B, Mistral 7B on summarization 🏆.
  - 3B + Adapter achieves 78.7% on IFEval beating Phi-3 mini, Gemma 7B, Mistral 7B; Server Model matches GPT-4-Turbo and beats Mixtral 8x22B and GPT-3.5-turbo ✨.
  
  LoRA for the win! 🎉
  
  Blog: https://machinelearning.apple.com/research/introducing-apple-foundation-models
reactions.reaction: 🤝
reactions.users: louisbrulenaudet zzffss SamChaTICE thethinkmachine
commentators._id: 6646428e923866048f3e13e0
commentators.avatarUrl: /avatars/54483699273ac58a4a6fe1fa4aab65fe.svg
commentators.fullname: Robert Sinclair
commentators.name: ZeroWw
commentators.type: user