Core ML vs Foundation Models: Which Should You Use?

Source: Dev.to

iOS 26.3: Core ML vs. Foundation Models

With iOS 26.3 now in the wild, iOS developers have two powerful on‑device AI frameworks to choose from:

• Core ML – Apple’s veteran ML inference engine.
• Foundation Models – the new framework that exposes Apple’s ~3 B‑parameter LLM from iOS 26 onward.

They sound like they do similar things. They don’t.

This article cuts through the confusion, explains what each framework is actually designed for, and gives you a clear decision framework so you can pick the right tool for the job — or know when to combine both.

---

What Each Framework Actually Is

Before comparing them, let’s be precise about what they are.

Core ML has been available since iOS 11. It’s a general‑purpose inference engine — you bring a trained model (in .mlmodel or .mlpackage format), and Core ML runs it on‑device using the best available hardware (Neural Engine, GPU, or CPU) depending on the task. Core ML itself doesn’t contain any models. You either:

• Train your own (with Create ML),
• Convert from TensorFlow or PyTorch, or
• Download pre‑trained ones.

It supports image classification, object detection, NLP, audio analysis, tabular data prediction, and more.

Foundation Models was introduced in iOS 26. It’s an API that gives you direct access to Apple’s own pre‑trained ~3 B‑parameter large language model — the same one behind Apple Intelligence. You don’t bring a model; the model is already on the device (downloaded as part of enabling Apple Intelligence). The framework specializes in natural language: text generation, summarisation, structured data extraction, and tool calling.

Bottom line: Core ML is a runtime that runs your models. Foundation Models is an API for a specific pre‑built Apple model.

---

Device Availability: A Critical Difference

This is where the two frameworks diverge most sharply, and it matters enormously for your architecture decisions.

If your feature needs to work for the majority of your users today, Core ML is the safer choice. If you’re building a feature that’s an enhancement for users on newer devices, Foundation Models is a compelling option.

---

What Each Framework Is Good At

Think of the two frameworks as operating in completely different problem spaces.

Core ML: Structured ML Tasks

Core ML excels when your task is well‑defined, has a clear input/output structure, and can be solved with a trained model:

• Image classification – “What object is in this photo?”
• Object detection – “Where are the faces/products/items in this frame?”
• Pose estimation – “Where are this person’s joints?”
• Audio classification – “Is this the sound of a dog or a car?”
• Text classification / sentiment analysis – “Is this review positive or negative?”
• Tabular prediction – “Based on these health metrics, what category does this fall into?”
• Real‑time video analysis – Frame‑by‑frame inference at high frequency

A key characteristic of Core ML tasks: you can define the exact output schema up front, and the model reliably produces a label, a bounding box, a confidence score, or a numeric prediction. Deterministic, fast, and predictable.

Foundation Models: Language and Reasoning Tasks

Foundation Models is designed for tasks that involve language understanding and generation:

• Text summarisation – turning long content into concise summaries
• Structured extraction – pulling structured data from unstructured text (e.g., extracting a name, date, and location from a messy user note)
• Content classification using natural language – nuanced categorisation with explanation
• Contextual suggestions – generating personalised recommendations based on user context
• Tool‑augmented reasoning – letting the model call into your app’s data to answer user questions

Apple explicitly warns that the on‑device 3 B model is not designed for world‑knowledge Q&A, code generation, or complex maths. It’s optimised for task‑oriented, app‑integrated intelligence.

---

When to Use Core ML

Use Core ML when:

• You need vision or audio inference. Core ML is the only on‑device option for camera‑based features — real‑time object detection, face analysis, pose estimation, scene classification. Foundation Models cannot process images (no image input support as of iOS 26.3).
• You need it to work on older devices. If your feature must work on an iPhone 12, 13, or 14, Core ML is your only on‑device option. It runs on any device with iOS 11+.
• You have a specific, narrow ML task. A model trained to classify 10 types of skin lesions, or to detect a specific product in a frame, will outperform a general 3 B LLM on that narrow task — and at a fraction of the memory and compute cost.
• You need deterministic, repeatable outputs. Core ML models return the same output for the same input, every time. Foundation Models, being a generative LLM, produces varied responses.
• You’re doing real‑time inference. Core ML can process frames from a camera feed at 30+ fps. Foundation Models is not suited for frame‑by‑frame tasks.

---

When to Use Foundation Models

Use Foundation Models when:

• Your task is fundamentally a language task. Summarising a document, extracting key facts from a user’s note, generating a personalised caption — these are natural fits.
• You want structured output from unstructured text. The @Generable macro lets you define the exact JSON schema you expect, and the model will attempt to produce output that conforms to it.
• You need tool‑augmented reasoning. The model can call into your app’s APIs (via the tool‑calling interface) to fetch up‑to‑date data, perform calculations, or query a database, then incorporate the result into its response.
• You’re targeting the newest devices and can rely on Apple Intelligence being enabled. If your user base is primarily iPhone 15 Pro/Max or later, you can safely assume the model is present.
• You accept non‑deterministic, generative results. For creative text generation, nuanced classification, or contextual suggestions, variability can be a feature, not a bug.

---

TL;DR Decision Matrix

Both frameworks can be combined: use Core ML for heavy‑weight vision/audio preprocessing, then feed the results into Foundation Models for natural‑language reasoning or summarisation. The key is to match the problem space to the right tool.

Extracting Type‑Safe Swift Structs Directly from Free‑Form Input

No JSON parsing, no regex, no post‑processing.

Why Use Natural Language Reasoning with Tool Calling?

Foundation Models can decide when to call into your app’s data, fetch it, and incorporate it into a response. Core ML models can’t reason about when or whether to request more context.

Zero Model Maintenance

• Core ML – you own the model, retrain it, update it, and handle drift over time.
• Foundation Models – Apple maintains the base model; you get improvements automatically with OS updates.

Speed of Integration

Building a Core ML‑powered feature involves selecting an architecture, gathering data, training, converting, and integrating.
Foundation Models can be integrated in an afternoon.

---

The Combination Pattern

The two frameworks aren’t competitors; they’re complementary layers. The most powerful apps combine them.

Example: SwingVision (a tennis/pickleball coaching app) uses Core ML to analyze video frames and extract structured data about a user’s movement and technique. That structured output is then fed as context into a Foundation Model session to generate natural‑language coaching feedback.

How It Works

Use Cases

• Fitness app: Core ML for pose estimation → Foundation Models generate a personalized workout summary.
• Cooking app: Core ML identifies ingredients from a photo → Foundation Models suggest a recipe.

// Step 1: Core ML classifies the image
let classificationRequest = VNCoreMLRequest(model: mlModel) { request, _ in
    guard let result = request.results?.first as? VNClassificationObservation else { return }

    // Step 2: Feed the structured output into Foundation Models
    Task {
        let session = LanguageModelSession()
        let response = try await session.respond(
            to: "The user photographed: \(result.identifier) with \(Int(result.confidence * 100))% confidence. Suggest what they could make with this."
        )
        print(response.content)
    }
}

---

Decision Framework

---

Summary

• Core ML – mature, flexible inference engine for deterministic ML tasks across all users.
• Foundation Models – purpose‑built API for language tasks on Apple‑Intelligence‑compatible devices, with almost zero setup cost.

Requirements

• Core ML: iOS 11+ (any iPhone 11+ for iOS 26)
• Foundation Models: iOS 26+ on Apple‑Intelligence‑compatible devices (iPhone 15 Pro or newer, all iPhone 16/17 models)