Automate Your Health: Building an AI Medical Agent with GPT-4 Function Calling and Google Calendar

Source: Dev.to

Introduction

We’ve all been there: you’re feeling under the weather, but the thought of navigating a clunky hospital portal or staying on hold for 20 minutes to book a doctor’s appointment feels like another symptom.

What if you could just say, “I have a sharp pain in my lower back, find me an orthopedic specialist for next Tuesday afternoon,” and have a smart AI agent handle the rest?

In this tutorial we’ll build a Healthcare Agent Controller. By leveraging OpenAI Function Calling, AI agents, and the Google Calendar API, we’ll create a system that:

1. Understands natural‑language requests.
2. Queries doctor availability.
3. Cross‑references your personal schedule.
4. Books the appointment – all via a clean FastAPI backend.

This is the future of healthcare automation and personalized AI assistants.

Architecture Overview

To build a reliable agent we need a clear separation between the “Brain” (LLM) and the “Hands” (APIs). Below is the data‑flow diagram.

sequenceDiagram
    participant User
    participant FastAPI as API_Gateway
    participant OpenAI as GPT-4o_(Brain)
    participant GCal as Google_Calendar_API_(Hands)

    User->>FastAPI: "Book a dentist for next Monday morning"
    FastAPI->>OpenAI: Prompt + Tools Definition
    OpenAI->>OpenAI: Analyze Intent & Extract Date/Type
    OpenAI-->>FastAPI: Call Function: search_doctors(specialty, date)
    FastAPI->>GCal: GET /events (Check slots)
    GCal-->>FastAPI: Returns Available Slots
    FastAPI->>OpenAI: "Found 10:00 AM and 11:30 AM"
    OpenAI-->>FastAPI: Call Function: create_appointment(time)
    FastAPI->>GCal: POST /events (Book it!)
    GCal-->>FastAPI: Confirmation
    FastAPI-->>User: "Success! You're booked for 10:00 AM."

Prerequisites

Note: Function calling lets GPT‑4 describe the arguments for functions you’ve defined in code. We’ll define two tools: one to fetch availability and another to create the event.

1️⃣ Define the Tools (tools.py)

# tools.py
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_calendar_availability",
            "description": "Check if a specific time slot is free on the doctor's calendar",
            "parameters": {
                "type": "object",
                "properties": {
                    "start_time": {
                        "type": "string",
                        "description": "ISO‑8601 formatted start time"
                    },
                    "end_time": {
                        "type": "string",
                        "description": "ISO‑8601 formatted end time"
                    },
                },
                "required": ["start_time", "end_time"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "book_appointment",
            "description": "Create a new calendar event for the medical appointment",
            "parameters": {
                "type": "object",
                "properties": {
                    "summary": {
                        "type": "string",
                        "description": "Reason for the visit (e.g., 'Orthopedic consult')"
                    },
                    "start_time": {"type": "string"},
                    "end_time": {"type": "string"},
                },
                "required": ["summary", "start_time", "end_time"],
            },
        },
    },
]

2️⃣ Helper Functions for Google Calendar

For brevity we assume you’ve already completed the OAuth2 flow and have a service object ready.

# gcal_helper.py
from googleapiclient.discovery import build

def create_appointment(summary: str, start_time: str, end_time: str):
    """
    Insert a new event into the authenticated user's primary calendar.
    """
    event = {
        "summary": summary,
        "start": {"dateTime": start_time, "timeZone": "UTC"},
        "end":   {"dateTime": end_time,   "timeZone": "UTC"},
    }
    # `service` should be a pre‑authenticated Google Calendar service instance.
    return service.events().insert(calendarId="primary", body=event).execute()

3️⃣ FastAPI Endpoint (main.py)

# main.py
import os
import json
from fastapi import FastAPI
from openai import OpenAI
from tools import tools
from gcal_helper import create_appointment

app = FastAPI()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

@app.post("/schedule")
async def handle_medical_request(user_input: str):
    """
    Receive a natural‑language request, let the LLM decide which tool to call,
    and execute the corresponding function.
    """
    # 1️⃣ Send request to OpenAI
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": user_input}],
        tools=tools,
        tool_choice="auto",
    )

    message = response.choices[0].message

    # 2️⃣ Did the model request a function call?
    if message.tool_calls:
        for tool_call in message.tool_calls:
            if tool_call.function.name == "book_appointment":
                args = json.loads(tool_call.function.arguments)
                result = create_appointment(
                    summary=args["summary"],
                    start_time=args["start_time"],
                    end_time=args["end_time"],
                )
                return {"status": "success", "data": result}

    # If no tool call, ask the user for clarification
    return {"status": "clarification_needed", "message": message.content}

4️⃣ Production Considerations

• HIPAA compliance – encrypt data at rest and in transit, restrict access, and audit logs.
• Retrieval‑Augmented Generation (RAG) – integrate a medical knowledge base for accurate advice.
• State management – handle multi‑turn conversations (e.g., confirming date/time).

For a deeper dive into production‑ready agent patterns, see the WellAlly Blog (specialized guides on prompt‑injection prevention, multi‑agent orchestration, etc.).

Next Steps

🎉 You’ve now built a minimal, functional healthcare‑agent controller!

By combining the reasoning power of GPT‑4 with the structural reliability of OpenAI Function Calling, we’ve turned a tedious administrative task into a seamless conversation. Keep iterating, secure the pipeline, and you’ll have a production‑grade AI assistant ready to help patients book appointments with just a few words.

Use a medical directory API or other data source to enrich the specialist lookup.

Are you building AI agents for healthcare or productivity? Drop a comment below or share your repo—I’d love to see how you’re implementing these patterns!