OpenAI

Overview

OpenAI's GPT models serve as an alternative cloud AI provider in the Prismatic Platform, complementing the primary Claude integration and local Ollama inference. The platform uses OpenAI primarily for embedding generation (text-embedding-3), classification tasks, and as a fallback provider when specialized model capabilities are needed. While Claude serves as the primary reasoning engine, OpenAI fills specific capability gaps, particularly in vector embedding generation where its text-embedding-3 family of models provides high-quality representations for semantic search.

The Prismatic Platform's multi-provider AI architecture ensures resilience -- if one provider is unavailable, the system automatically routes to alternatives. OpenAI's embedding models are particularly valuable for the platform's semantic search capabilities, generating vector representations of intelligence documents, agent descriptions, and security findings for similarity-based retrieval via Meilisearch. These embeddings transform text into numerical vectors that capture semantic meaning, enabling searches that understand intent rather than just matching keywords.

OpenAI integration follows the same provider abstraction used for Claude and Ollama, allowing seamless switching between providers based on task requirements, cost optimization, and availability. The PrismaticClaude.Providers module defines a common behaviour that all providers implement, making the AI layer genuinely provider-agnostic at the application level. Application code never calls OpenAI directly -- it requests capabilities through the provider router, which selects the optimal provider based on the task requirements.

Key Features

• GPT-4 Turbo: Advanced reasoning and code generation for complex analysis tasks requiring broad knowledge
• Embeddings: text-embedding-3-small (1536 dimensions) and text-embedding-3-large (3072 dimensions) for vector representations
• Function Calling: Structured output through tool definitions, enabling agents to invoke platform functions with type-safe parameters
• JSON Mode: Guaranteed valid JSON output for automated pipeline integrations, eliminating parsing failures
• Streaming: Token-by-token response streaming for real-time display in Phoenix LiveView interfaces
• Fine-Tuning: Custom model training on platform-specific datasets for domain-specialized classification
• Batch API: Cost-efficient batch processing for large-scale document analysis at reduced pricing
• Vision: GPT-4V image analysis for security screenshot review and UI testing automation

Platform Integration

OpenAI provides embedding generation and fallback inference through the platform's unified provider abstraction.

defmodule PrismaticClaude.Providers.OpenAI do
  @moduledoc """
  OpenAI provider implementing the unified AI provider behaviour.
  Primary use: embedding generation and fallback inference.
  """
  @behaviour PrismaticClaude.Providers.Behaviour

  @base_url "https://api.openai.com/v1"

  @impl true
  def generate_embedding(text, opts \\ []) do
    model = Keyword.get(opts, :model, "text-embedding-3-small")
    headers = auth_headers()

    body = Jason.encode!(%{input: text, model: model})

    case HTTPoison.post("#{@base_url}/embeddings", body, headers,
           recv_timeout: 30_000) do
      {:ok, %{status_code: 200, body: body}} ->
        %{"data" => [%{"embedding" => embedding}]} = Jason.decode!(body)
        {:ok, embedding}

      {:ok, %{status_code: 429}} ->
        {:error, :rate_limited}

      {:ok, %{status_code: status, body: body}} ->
        {:error, {:api_error, status, Jason.decode!(body)}}

      {:error, reason} ->
        {:error, reason}
    end
  end

  @impl true
  def complete(prompt, opts \\ []) do
    model = Keyword.get(opts, :model, "gpt-4-turbo")
    max_tokens = Keyword.get(opts, :max_tokens, 4096)

    body = Jason.encode!(%{
      model: model,
      messages: [%{role: "user", content: prompt}],
      max_tokens: max_tokens,
      response_format: %{type: "json_object"}
    })

    case HTTPoison.post("#{@base_url}/chat/completions", body, auth_headers(),
           recv_timeout: 60_000) do
      {:ok, %{status_code: 200, body: resp}} ->
        %{"choices" => [%{"message" => %{"content" => content}}]} = Jason.decode!(resp)
        {:ok, content}

      {:ok, %{status_code: 429}} ->
        {:error, :rate_limited}

      {:error, reason} ->
        {:error, reason}
    end
  end

  @impl true
  def available? do
    case HTTPoison.get("#{@base_url}/models", auth_headers(), recv_timeout: 5_000) do
      {:ok, %{status_code: 200}} -> true
      _ -> false
    end
  end

  defp auth_headers do
    [
      {"Authorization", "Bearer #{api_key()}"},
      {"Content-Type", "application/json"}
    ]
  end

  defp api_key, do: Application.get_env(:prismatic_claude, :openai)[:api_key]
end

The provider abstraction enables automatic fallback routing. If OpenAI returns a rate limit or error, the coordinator routes to the next available provider transparently.

defmodule PrismaticClaude.Providers.Router do
  @moduledoc """
  Routes AI requests to the optimal provider based on task type,
  availability, and configuration. Handles automatic fallback.
  """

  @spec route(atom(), keyword()) :: {:ok, module()} | {:error, :no_provider}
  def route(:embedding, _opts) do
    cond do
      PrismaticClaude.Providers.OpenAI.available?() -> {:ok, PrismaticClaude.Providers.OpenAI}
      PrismaticClaude.Providers.Ollama.available?() -> {:ok, PrismaticClaude.Providers.Ollama}
      true -> {:error, :no_provider}
    end
  end

  def route(:completion, opts) do
    cond do
      Keyword.get(opts, :sensitive, false) ->
        {:ok, PrismaticClaude.Providers.Ollama}
      PrismaticClaude.Providers.OpenAI.available?() ->
        {:ok, PrismaticClaude.Providers.OpenAI}
      PrismaticClaude.Providers.Ollama.available?() ->
        {:ok, PrismaticClaude.Providers.Ollama}
      true ->
        {:error, :no_provider}
    end
  end
end

Architecture

OpenAI occupies the cloud AI provider tier in the platform's multi-provider architecture.

The provider hierarchy for the platform follows a clear priority order.

Performance Characteristics

OpenAI API performance varies by model and endpoint. The platform tracks these metrics for provider selection optimization.

Embedding generation is the most cost-effective OpenAI operation for the platform, providing high-quality vector representations at minimal cost per document.

Configuration

OpenAI provider configuration is managed through the platform's runtime configuration system with support for environment-specific overrides.

# OpenAI provider configuration in config/runtime.exs
config :prismatic_claude, :openai,
  api_key: System.get_env("OPENAI_API_KEY"),
  organization: System.get_env("OPENAI_ORG_ID"),
  model: "gpt-4-turbo",
  embedding_model: "text-embedding-3-small",
  max_tokens: 4096,
  timeout: 30_000,
  retry_count: 3,
  retry_delay: 1_000,
  rate_limit: [
    requests_per_minute: 500,
    tokens_per_minute: 30_000
  ]

Rate limiting is enforced client-side to prevent exceeding OpenAI's API limits and triggering 429 responses.

# Client-side rate limiter for OpenAI API calls
config :prismatic_claude, :openai_rate_limiter,
  enabled: true,
  window_ms: 60_000,
  max_requests: 500,
  max_tokens: 30_000,
  backoff_strategy: :exponential,
  max_backoff_ms: 30_000

Best Practices

• Use embeddings for semantic search -- OpenAI's embedding models outperform keyword matching for semantic similarity across intelligence documents and agent descriptions
• Prefer JSON mode for structured outputs that feed into automated pipelines, avoiding fragile parsing of free-text responses
• Implement rate limit handling with exponential backoff, as OpenAI's API enforces strict per-minute token limits
• Cache embeddings aggressively in ETS or Redis since the same text always produces the same embedding vector
• Use the provider abstraction -- never call OpenAI directly from application code; always go through PrismaticClaude.Providers
• Monitor API costs -- track token usage per operation type to identify cost optimization opportunities; batch API offers 50% savings for non-urgent operations
• Handle all error codes -- implement specific handling for 429 (rate limit), 500 (server error), and 503 (overloaded) responses
• Use text-embedding-3-small by default -- the small model provides 90%+ of the large model's quality at 15% of the cost

Comparison with Alternatives

OpenAI complements Claude and Ollama in the platform's AI architecture: Claude provides the best reasoning for Elixir code, Ollama provides privacy for sensitive data, and OpenAI provides the best embedding models and JSON mode for structured data extraction.

Related Technologies

• Claude - Primary AI provider for reasoning and code generation
• Ollama - Local AI inference for offline and privacy-sensitive operations
• Meilisearch - Search engine that can consume OpenAI-generated embeddings for semantic search
• ETS - In-memory cache for embedding vectors and API response caching
• Redis - Distributed cache for embedding vectors across cluster nodes
• Elixir - Host language providing the HTTP client and provider abstraction

Related Apps

• prismatic_claude - Multi-provider AI layer hosting the OpenAI adapter
• prismatic_agents - Agent system that may route to OpenAI for specific tasks
• prismatic_storage_meilisearch - Search engine consuming OpenAI embeddings

Connect & Contribute

Created by Tomáš Korcak (korczis) | Open Source under GHL

• GitHub | GitLab | LinkedIn | Contact
• Developer Portal | Architecture | Meet the Creator