Connecting GPT-4o to Microcontrollers: A Complete Guide

Running a large language model on a microcontroller is not possible — GPT-4o needs far more memory than any ESP32 has. But calling GPT-4o from a microcontroller over Wi-Fi is not only possible, it is exactly how the ProjectOSAI Companion Watch works. This guide covers everything you need to make it happen.

The Architecture

Your ESP32 acts as a thin client. It captures user input (voice, text, or sensor data), sends it to OpenAI's API over HTTPS, receives the response, and acts on it (speaking it, displaying it, or triggering an action). The AI runs in the cloud; the ESP32 orchestrates the interaction.

This approach has trade-offs. You need a Wi-Fi connection for every AI interaction, and there is latency from the network round-trip. But you get access to the most powerful AI models available, which is impossible to match with on-device inference on a microcontroller.

Prerequisites

ESP32-S3 board with PSRAM (8MB recommended)
Arduino IDE with ESP32 board package
ArduinoJson library (version 7+)
OpenAI API key with GPT-4o access
Wi-Fi network credentials

Step 1: Setting Up the HTTPS Connection

OpenAI's API requires HTTPS. The ESP32's WiFiClientSecure handles TLS, but you need to either provide the root CA certificate or disable certificate verification for prototyping:

#include <WiFi.h>
#include <HTTPClient.h>
#include <ArduinoJson.h>

const char* ssid = "YOUR_WIFI";
const char* password = "YOUR_PASSWORD";
const char* apiKey = "sk-YOUR-API-KEY";
const char* endpoint = "https://api.openai.com/v1/chat/completions";

void setup() {
  Serial.begin(115200);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("Connected to Wi-Fi");
}

Step 2: Building the API Request

GPT-4o uses the chat completions endpoint. You need to construct a JSON body with the model name, messages array, and parameters:

String callGPT(String userMessage) {
  HTTPClient http;
  http.begin(endpoint);
  http.addHeader("Content-Type", "application/json");
  http.addHeader("Authorization", String("Bearer ") + apiKey);
  http.setTimeout(30000);  // 30 second timeout

  // Build JSON request
  JsonDocument doc;
  doc["model"] = "gpt-4o";
  doc["max_tokens"] = 150;
  doc["temperature"] = 0.7;

  JsonArray messages = doc["messages"].to<JsonArray>();

  JsonObject sysMsg = messages.add<JsonObject>();
  sysMsg["role"] = "system";
  sysMsg["content"] = "You are a helpful assistant on a "
    "smartwatch. Keep responses under 2 sentences.";

  JsonObject usrMsg = messages.add<JsonObject>();
  usrMsg["role"] = "user";
  usrMsg["content"] = userMessage;

  String requestBody;
  serializeJson(doc, requestBody);

  int httpCode = http.POST(requestBody);
  String response = "";

  if (httpCode == 200) {
    String payload = http.getString();
    JsonDocument resDoc;
    deserializeJson(resDoc, payload);
    response = resDoc["choices"][0]["message"]["content"]
      .as<String>();
  } else {
    response = "Error: " + String(httpCode);
  }

  http.end();
  return response;
}

Step 3: Managing Conversation History

GPT-4o is stateless — it does not remember previous messages unless you send them. For multi-turn conversations, you need to accumulate messages and send the full history with each request:

// Store conversation in PSRAM
String conversationHistory = "[]";

String callGPTWithHistory(String userMessage) {
  JsonDocument doc;
  deserializeJson(doc, conversationHistory);
  JsonArray messages = doc.to<JsonArray>();

  // Add the new user message
  JsonObject usrMsg = messages.add<JsonObject>();
  usrMsg["role"] = "user";
  usrMsg["content"] = userMessage;

  // Call the API with full history...
  // After getting the response, add it too:
  JsonObject astMsg = messages.add<JsonObject>();
  astMsg["role"] = "assistant";
  astMsg["content"] = aiResponse;

  // Save updated history
  serializeJson(doc, conversationHistory);

  // Trim if too long (keep last 10 messages)
  if (messages.size() > 12) {
    // Remove oldest non-system messages
    messages.remove(1);
    messages.remove(1);
  }

  return aiResponse;
}

Step 4: Memory and Token Management

On an ESP32, memory management is critical. Here are the key considerations:

Use PSRAM — Allocate large buffers in PSRAM with ps_malloc() instead of regular malloc(). The ArduinoJson library can use PSRAM with a custom allocator.
Keep max_tokens low — For wearable use, 100-200 tokens is enough. Longer responses waste API credits and take too long to speak via TTS.
Trim conversation history — Do not let the message array grow indefinitely. Keep the system prompt plus the last 8-10 messages.
Stream responses — For faster perceived latency, use the streaming API and process chunks as they arrive.

Step 5: Error Handling for Embedded

Cloud APIs fail in ways that desktop apps rarely experience. Your ESP32 code needs to handle Wi-Fi drops, API timeouts, rate limits (HTTP 429), and malformed responses gracefully. Always set an HTTP timeout, always check the response code before parsing JSON, and always have a fallback message for the user.

Cost Optimization

Every API call costs money. For a wearable that might be used 20-50 times per day:

Use GPT-4o mini for simple queries (10x cheaper than GPT-4o)
Route complex questions to GPT-4o and simple ones to mini based on input length or keywords
Cache common responses in NVS storage
Use our AI API Cost Calculator to estimate your monthly spend

What We Built With This

The ProjectOSAI Companion Watch uses this exact approach — GPT-4o over Wi-Fi from an ESP32-S3 — combined with Whisper for speech-to-text and OpenAI TTS for voice output. The full voice pipeline runs in about 2-4 seconds end-to-end. If you want to skip the coding and get a working AI companion on your wrist, check out the watch.