Building RecallMe: An On-Device AI Companion for Dementia Care Using Flutter & Kiro

Source: Dev.to

Introduction

Dementia affects over 55 million people worldwide. One of the most heartbreaking challenges is when loved ones can no longer recognize family members or recall cherished memories. Traditional solutions often rely on cloud‑based AI, raising privacy concerns and requiring constant internet connectivity.

RecallMe is a privacy‑first, on‑device AI companion built with Flutter that helps dementia patients recognize faces and recall memories without ever leaving the device.

Core Features

• On‑device face recognition using Google ML Kit and a custom 256‑dimensional embedding algorithm.
• AI‑powered memory conversations with context‑aware chat (Azure OpenAI GPT‑4, replaceable with an on‑device model).
• Voice interaction for natural, accessible communication (speech‑to‑text and text‑to‑speech).
• Smart routine management with notifications, timezone awareness, and progress tracking.
• Fully offline – no cloud, no privacy concerns, all data stays locally.

Technology Stack

Custom Embedding Algorithm (256 D)

• Color histograms – 64 D
• Spatial grid features – 64 D
• Gradient features – 6 D
• LBP texture analysis – 16 D
• Quadrant averages – 4 D

The algorithm runs entirely in Kotlin, optimized for Arm architecture with NEON SIMD instructions.

Face Recognition Workflow

// 1. Face Detection (using ML Kit)
Future<List<FaceDetection>> detectFaces(Uint8List imageBytes) async {
  final result = await _channel.invokeMethod('detectFaces', {
    'imageBytes': imageBytes,
  });
  return result.map((face) => FaceDetection.fromMap(face)).toList();
}

// 2. Generate Embedding (custom algorithm in Kotlin)
// Combines multiple feature extraction techniques
// Returns 256‑dimensional vector

// 3. Match Against Stored Embeddings
double similarity = calculateCosineSimilarity(
  newEmbedding,
  storedEmbedding
);
if (similarity > 0.45) {
  // Match found!
}

All steps execute locally; no network request is made.

Development with Kiro

Kiro, an AI‑powered IDE, accelerated development through steering documents placed in .kiro/steering/:

• product.md – dementia‑friendly design principles, target users, core features.
• structure.md – layered architecture (Presentation → State → Business Logic → Data) and code organization patterns.
• tech.md – technology stack, dependencies, and build commands.

These documents gave Kiro deep context, enabling it to generate code that adhered to:

• Warm color palette (soft yellows, creams, oranges).
• File naming conventions (snake_case, _screen.dart suffix).
• Provider‑based state management.
• Hive database schema.

Example Conversation

Me: Build a routine management screen with add/edit/delete functionality, notification scheduling, and completion tracking.
Kiro: Generates a complete implementation following our architecture patterns, using Provider for state, Hive for persistence, and our warm color theme.

Result: 15+ screens built with consistent patterns, timezone‑aware notifications, and high code quality—all in days instead of weeks.

User Experience

• Face identification: Point the camera at someone and instantly see “This is Sarah (85 % confidence)”.
• Memory chat: Tap a photo to start a conversation; the AI remembers the last 5 messages, includes photo metadata (name, year, person, memory word), and replies in short, simple sentences.
• Voice‑first interaction: Users speak naturally; responses are spoken via TTS—no typing required.
• Routine scheduling: Daily routines with precise times, notifications, and progress tracking across Home, Schedule, and Daily Tasks screens.
• Weekly reports: Visual progress summaries for caregivers.

All data remains on‑device:

• Face embeddings stored in Hive.
• Photos saved in the app’s private directory.
• Sensitive data (API keys, PIN) encrypted locally.

Performance Optimizations for Arm Devices

• NEON SIMD for vectorized histogram calculations.
• Big.LITTLE‑aware task scheduling.
• Efficient memory patterns for image processing.
• GPU acceleration when available (via ML Kit).

The app runs smoothly on mid‑range Arm devices, making it accessible to a wide audience.

Lessons Learned

• On‑device ML: Balancing accuracy with performance constraints.
• Accessibility: Designing UI/UX for cognitive impairments.
• Privacy: Ensuring all AI processing stays local.
• Integration: Seamlessly combining multiple AI technologies.

Open Source & Community

RecallMe is open source to:

• Help developers learn on‑device ML techniques.
• Provide dementia‑friendly design patterns.
• Demonstrate Arm optimization best practices.
• Inspire more privacy‑first AI applications.

Repository:

git clone https://github.com/Gaurav-derry/Recall

The repo includes complete source code, setup instructions, architecture documentation, and an Arm device build guide.

Future Enhancements

• Replace Azure OpenAI with an on‑device LLM.
• Multi‑language support.
• Caregiver dashboard with analytics.
• Integration with health‑monitoring devices.

Conclusion

RecallMe shows that sophisticated AI can run entirely on mobile devices while respecting privacy and accessibility. By combining Flutter’s cross‑platform capabilities, on‑device ML, and Kiro’s AI‑powered development workflow, we can build applications that make a real difference in people’s lives.

Technologies: Flutter, Dart, Kotlin, Google ML Kit, Azure OpenAI, Hive, Provider
Development Tools: Kiro (AI‑powered IDE), Android Studio, VS Code