AWS re:Invent 2025 - Streamline AI model development lifecycle with Amazon SageMaker AI (AIM364)
Source: Dev.to
Introduction
In this session, Khushboo Srivastava, Bruno Pistone, and Manikandan Paramasivan from KOHO demonstrate how Amazon SageMaker Studio streamlines the AI model development lifecycle—from data preparation to deployment. The presentation covers end‑to‑end workflows, including fine‑tuning large language models (LLMs) with SageMaker HyperPod and EKS orchestration, MLflow experiment tracking, and various deployment options. Key features highlighted are IDE flexibility (JupyterLab, Code Editor, VS Code remote access), Trusted Identity Propagation for security, Amazon Nova model customization, and the new SageMaker Spaces add‑on for running IDEs on HyperPod clusters with GPU sharing.
KOHO’s case study shows impressive results:
- 98 % cost reduction (from $1.5 M to $26 K annually)
- 15 ms latency for fraud‑detection processing over 1 M+ daily transactions
These outcomes illustrate how SageMaker Studio enables enterprise‑scale ML solutions with startup agility across both traditional ML and generative AI (GenAI) applications.
Speakers
- Khushboo Srivastava – Senior Product Manager, Technical, Amazon SageMaker Studio (6+ years at AWS)
- Bruno Pistone – Senior Worldwide Specialist Solutions Architect, AWS (5.5 years), focus on model training and LLM customization
- Manikandan Paramasivan – Senior Staff Architect, Data, ML & AI, KOHO; leads architecture, infrastructure, and operations for KOHO’s data and ML platforms
Generative AI Market Landscape
- IDC predicts global spending on generative AI will reach $202 B by 2028, representing ~32 % of total AI spending.
- With a CAGR of 29 %, Goldman Sachs forecasts generative AI could boost global GDP by up to 7 % (~$7 trillion) and lift productivity growth by 1.5 percentage points over the next decade.
Adoption metrics
- 89 % of enterprises are advancing generative AI initiatives.
- 92 % plan to increase investments by 2027.
- 78 % already use AI in at least one business function.
- 77 % prefer models ≤ 3 B parameters, indicating a demand for customizable, cost‑effective solutions.
Challenges in ML Development
- Fragmented toolsets – Disparate, disconnected ML tools increase time‑to‑market; teams spend more time managing tools than building solutions.
- Isolation between roles – Data scientists, AI developers, and business teams often work in silos, leading to duplicated effort and missed opportunities.
- Governance complexity – Scaling AI/ML projects makes security, compliance, and governance increasingly difficult without a unified framework.
- Infrastructure management – Training and fine‑tuning large models require flexible, high‑performance compute resources that are hard to provision and manage.
Traditional vs. Generative AI Workflow
| Stage | Traditional ML | Generative AI (GenAI) |
|---|---|---|
| Data preparation | Clean, transform, feature‑engineer data | Format data into prompt templates or LLM‑specific structures |
| Model selection | Choose algorithm (e.g., XGBoost, Random Forest) | Select a foundational LLM (e.g., Claude, Llama) |
| Compute | Run on CPUs/GPUs or managed services | Use specialized clusters (e.g., HyperPod) for large‑scale training |
| Evaluation & deployment | Validate metrics, deploy as API or batch job | Evaluate generation quality, fine‑tune, and deploy as an endpoint or integrated service |
Amazon SageMaker Studio Overview
SageMaker Studio is a purpose‑built, end‑to‑end ML development platform that addresses the challenges above by consolidating data preparation, model training, fine‑tuning, deployment, and monitoring into a single environment.
IDE Options
- SageMaker Studio IDE – Integrated visual interface.
- JupyterLab – Classic notebook experience.
- Code Editor – VS Code‑based editor with remote access.
Data Preparation
- Use SageMaker Studio notebooks for scripting data generation and preprocessing.
- Leverage built‑in EMR connections to run large‑scale Spark jobs directly from notebooks.
Model Selection & Fine‑tuning
- Access a hub of pre‑built foundation models.
- Choose fine‑tuning techniques (e.g., supervised fine‑tuning, reinforcement learning) or bring your own model to train from scratch.
Deployment & Monitoring
- Visual interface for creating and managing production endpoints.
- Single pane of glass to monitor models, endpoints, and associated resources.
Experiment Tracking & Pipelines
- MLflow integration for experiment tracking and reproducibility.
- SageMaker Pipelines for building, visualizing, and orchestrating workflows (drag‑and‑drop or code‑first).
Note: Tens of thousands of customers use Amazon SageMaker AI, including 3M, Coinbase, Intuit, Domino’s, and many others.
Demo Overview
The live demo walks through a complete generative AI project:
- Data preparation – Ingest and format data for LLM fine‑tuning.
- Model fine‑tuning – Use SageMaker HyperPod with EKS orchestration to train a large language model.
- Deployment – Deploy the fine‑tuned model via SageMaker endpoints, showcasing different deployment options.
The demo emphasizes how SageMaker AI services (Studio, HyperPod, MLflow, Pipelines, Spaces) work together to streamline each step.
Architecture & Personas
Two primary personas are illustrated:
- Platform Administrator – Sets up a private networking environment, provisions a HyperPod cluster with EKS orchestration, and configures the shared compute resources for both training and inference.
- Data Scientist / ML Engineer – Uses the provisioned environment to run notebooks, track experiments, and deploy models, all within SageMaker Studio’s unified interface.