Apple, to me, is the winner in the end of this AI chapter.
Distillation works: we can get models leaner, have better performance with lesser and lesser parameters, more efficient, running locally. Apple gets this and we'll see the result with M7 Max/ultra chips and I reckon this will be one of the titular apps when M7 line of chips are introduced.
I guess it explains the haste Anthropic and OpenAI are in to IPO, cash out and leave the the retail investor holding the bag.
pen2l
"Harbor is currently in a private preview under Apple’s Claris brand"
There's a name I haven't heard in a long time! Claris (always an Apple subsidiary, since being founded in 1987) renamed themselves to FileMaker in 1998, but apparently they rebranded back to Claris in 2019.
What I need most right now is a model like TypeSafe's Jev that can be built directly into the computer.
Jev allows for numerous small judgments: Is this email important? Which program should handle this request? Is this operation risky? There's no need for large LLMs to slowly generate long paragraphs of text; all you need is a fast, structured result that returns the message.
Having a local Jev-like model in the operating system for quick decision-making, and then calling the larger model when deeper thinking is needed, would be very useful.
iantpe
I have been using llama.app, which is the desktop interface built on top of llama.cpp by the same people. It is an alternative to Ollama that might interest people who want to switch given Ollama's reputation and previous decisions. Llama.app has a really clean design, an elegant and intuitive interface. But it's good to get alternatives as well, if Apple also gets into the game.
As models improve and hardware becomes more capable Apple are definitely well positioned.
I am super interested to see what the Apple silicon team can deliver when their full attention is turned to local LLM acceleration.
gehsty
If you ask who would pay for Apple local models, I would. A Mac Studio high end 512GB costs ~$1000/month for 3 years versus my current run rate of ~$1000/month for five devs for Claude Ultra (or ~$10k/week for Claude Sonnet if I paid API rates rather than Ultra).
If/when Anthropic changes to pure API pricing, or for API-heavy projects, that's when I'd switch to more local models. Of course there are tradeoffs for speed, quality, upgradeability, etc.
I currently run a Mac with 128GB and qwen3.8:27b-mlx. It hits the heat limit of the Mac when working on a 200k LOC codebase for security auditing and hardening for many hours at a time. This isn't as strong/fast as Sonnet, to be sure, yet the cost is only $6K one-time plus the usual electricity, maintenance time, etc.
jph
I've been running local models under ollama, then lm-studio, and now mtplx (command-line, from supervisord). I'd welcome a vendor-supported solution, if it doesn't turn into bloatware (like lm-studio did).
jwr
Calculate the total cost of two new Mac Studios with max memory and cables. For that budget you could burn tokens on cloud APIs for a long time or build a dedicated server on open hardware with clear server buses
KolibriFly
That will be really cool. It is already possible though through Apple Intelligence. There is an open source app that already leverages that.
> Cluster for scale: “Connect two or more Thunderbolt 5-capable Mac computers and Harbor pools their unified memory into a single inference resource. Run models that exceed any single computer’s capacity, or serve more concurrent requests across your cluster.”
This seems like the main way it would differentiate itself over existing solutions like LMStudio etc.
comments (10)
Distillation works: we can get models leaner, have better performance with lesser and lesser parameters, more efficient, running locally. Apple gets this and we'll see the result with M7 Max/ultra chips and I reckon this will be one of the titular apps when M7 line of chips are introduced.
I guess it explains the haste Anthropic and OpenAI are in to IPO, cash out and leave the the retail investor holding the bag.
pen2l
There's a name I haven't heard in a long time! Claris (always an Apple subsidiary, since being founded in 1987) renamed themselves to FileMaker in 1998, but apparently they rebranded back to Claris in 2019.
Looks like they've mostly just released annual upgrades to FileMaker for the past decade, but they did add MCP to it last year: https://www.claris.com/blog/2025/why-mcp-matters-for-claris-...
simonw
Jev allows for numerous small judgments: Is this email important? Which program should handle this request? Is this operation risky? There's no need for large LLMs to slowly generate long paragraphs of text; all you need is a fast, structured result that returns the message.
Having a local Jev-like model in the operating system for quick decision-making, and then calling the larger model when deeper thinking is needed, would be very useful.
iantpe
https://llama.app/
thibsrn
I am super interested to see what the Apple silicon team can deliver when their full attention is turned to local LLM acceleration.
gehsty
If/when Anthropic changes to pure API pricing, or for API-heavy projects, that's when I'd switch to more local models. Of course there are tradeoffs for speed, quality, upgradeability, etc.
I currently run a Mac with 128GB and qwen3.8:27b-mlx. It hits the heat limit of the Mac when working on a 200k LOC codebase for security auditing and hardening for many hours at a time. This isn't as strong/fast as Sonnet, to be sure, yet the cost is only $6K one-time plus the usual electricity, maintenance time, etc.
jph
jwr
KolibriFly
https://github.com/pdparchitect/noodle
_pdp_
This seems like the main way it would differentiate itself over existing solutions like LMStudio etc.
jedbrooke