Adding Custom LLM Providers

Register custom model providers with Pi using extensions for proxies, custom endpoints, and OAuth flows

Extensions can register custom model providers via pi.registerProvider(). This enables proxies, self-hosted deployments, OAuth/SSO authentication, and custom API implementations.

Quick Start

Create an extension that registers a provider:

import type { ExtensionAPI } from "@mariozechner/pi-coding-agent";

export default function (pi: ExtensionAPI) {
  pi.registerProvider("my-provider", {
    baseUrl: "https://api.example.com",
    apiKey: "MY_API_KEY",
    api: "openai-completions",
    models: [
      {
        id: "my-model",
        name: "My Model",
        reasoning: false,
        input: ["text", "image"],
        cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
        contextWindow: 128000,
        maxTokens: 4096
      }
    ]
  });
}

Use Cases

Proxy

Redirect an existing provider through a corporate proxy:

pi.registerProvider("anthropic", {
  baseUrl: "https://proxy.example.com"
});

Custom Endpoint

Add a self-hosted or private deployment:

pi.registerProvider("my-llm", {
  baseUrl: "https://api.my-llm.com/v1",
  apiKey: "MY_LLM_API_KEY",
  api: "openai-completions",
  models: [/* ... */]
});

OAuth/SSO

Add enterprise authentication:

pi.registerProvider("corporate-ai", {
  baseUrl: "https://ai.corp.com/v1",
  api: "openai-responses",
  models: [/* ... */],
  oauth: {
    name: "Corporate AI (SSO)",
    async login(callbacks) { /* ... */ },
    async refreshToken(cred) { /* ... */ },
    getApiKey(cred) { return cred.access; }
  }
});

Override Existing Provider

When only baseUrl and/or headers are provided (no models), all existing models for that provider are preserved with the new endpoint.

Register New Provider

When models is provided, it replaces all existing models for that provider.

OAuth Support

Add authentication that integrates with /login:

Custom Streaming API

For providers with non-standard APIs, implement streamSimple:

import {
  type AssistantMessage,
  type AssistantMessageEventStream,
  type Context,
  type Model,
  type SimpleStreamOptions,
  calculateCost,
  createAssistantMessageEventStream,
} from "@mariozechner/pi-ai";

function streamMyProvider(
  model: Model<any>,
  context: Context,
  options?: SimpleStreamOptions
): AssistantMessageEventStream {
  const stream = createAssistantMessageEventStream();

  (async () => {
    const output: AssistantMessage = {
      role: "assistant",
      content: [],
      api: model.api,
      provider: model.provider,
      model: model.id,
      usage: {
        input: 0,
        output: 0,
        cacheRead: 0,
        cacheWrite: 0,
        totalTokens: 0,
        cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
      },
      stopReason: "stop",
      timestamp: Date.now(),
    };

    try {
      stream.push({ type: "start", partial: output });

      // Make API request and process response...
      // Push content events as they arrive...

      stream.push({
        type: "done",
        reason: output.stopReason as "stop" | "length" | "toolUse",
        message: output
      });
      stream.end();
    } catch (error) {
      output.stopReason = options?.signal?.aborted ? "aborted" : "error";
      output.errorMessage = error instanceof Error ? error.message : String(error);
      stream.push({ type: "error", reason: output.stopReason, error: output });
      stream.end();
    }
  })();

  return stream;
}

pi.registerProvider("my-provider", {
  baseUrl: "https://api.example.com",
  apiKey: "MY_API_KEY",
  api: "my-custom-api",
  models: [/* ... */],
  streamSimple: streamMyProvider
});

Study the existing provider implementations in packages/ai/src/providers/ before writing your own.

Complete Example

Here’s a complete custom provider with OAuth:

custom-provider-anthropic/index.ts

import Anthropic from "@anthropic-ai/sdk";
import type {
  Api,
  AssistantMessageEventStream,
  Context,
  Model,
  OAuthCredentials,
  OAuthLoginCallbacks,
  SimpleStreamOptions,
} from "@mariozechner/pi-ai";
import type { ExtensionAPI } from "@mariozechner/pi-coding-agent";

// OAuth implementation
async function loginAnthropic(callbacks: OAuthLoginCallbacks): Promise<OAuthCredentials> {
  const authUrl = "https://claude.ai/oauth/authorize?...";
  callbacks.onAuth({ url: authUrl });
  
  const authCode = await callbacks.onPrompt({ message: "Paste the authorization code:" });
  const tokens = await exchangeCodeForTokens(authCode);
  
  return {
    refresh: tokens.refreshToken,
    access: tokens.accessToken,
    expires: Date.now() + tokens.expiresIn * 1000 - 5 * 60 * 1000,
  };
}

async function refreshAnthropicToken(credentials: OAuthCredentials): Promise<OAuthCredentials> {
  const tokens = await refreshTokens(credentials.refresh);
  return {
    refresh: tokens.refreshToken,
    access: tokens.accessToken,
    expires: Date.now() + tokens.expiresIn * 1000 - 5 * 60 * 1000,
  };
}

function streamCustomAnthropic(
  model: Model<Api>,
  context: Context,
  options?: SimpleStreamOptions
): AssistantMessageEventStream {
  // Custom streaming implementation
  // See packages/ai/src/providers/anthropic.ts for reference
}

export default function (pi: ExtensionAPI) {
  pi.registerProvider("custom-anthropic", {
    baseUrl: "https://api.anthropic.com",
    apiKey: "CUSTOM_ANTHROPIC_API_KEY",
    api: "custom-anthropic-api",

    models: [
      {
        id: "claude-opus-4-5",
        name: "Claude Opus 4.5 (Custom)",
        reasoning: true,
        input: ["text", "image"],
        cost: { input: 5, output: 25, cacheRead: 0.5, cacheWrite: 6.25 },
        contextWindow: 200000,
        maxTokens: 64000,
      },
      {
        id: "claude-sonnet-4-5",
        name: "Claude Sonnet 4.5 (Custom)",
        reasoning: true,
        input: ["text", "image"],
        cost: { input: 3, output: 15, cacheRead: 0.3, cacheWrite: 3.75 },
        contextWindow: 200000,
        maxTokens: 64000,
      },
    ],

    oauth: {
      name: "Custom Anthropic (Claude Pro/Max)",
      login: loginAnthropic,
      refreshToken: refreshAnthropicToken,
      getApiKey: (cred) => cred.access,
    },

    streamSimple: streamCustomAnthropic,
  });
}

See packages/coding-agent/examples/extensions/custom-provider-anthropic/ for the complete working example.

Unregister Provider

Remove a provider:

pi.unregisterProvider("my-provider");

Built-in models that were overridden are restored.

Reference

Config Options

interface ProviderConfig {
  baseUrl?: string;           // API endpoint URL
  apiKey?: string;            // API key or env var name
  api?: Api;                  // API type for streaming
  streamSimple?: Function;    // Custom streaming implementation
  headers?: Record<string, string>;  // Custom headers
  authHeader?: boolean;       // Add Authorization: Bearer header
  models?: ProviderModelConfig[];    // Model definitions
  oauth?: OAuthProvider;      // OAuth configuration
}

Model Definition

interface ProviderModelConfig {
  id: string;                 // Model ID
  name: string;               // Display name
  api?: Api;                  // API override for this model
  reasoning: boolean;         // Supports extended thinking
  input: ("text" | "image")[];  // Supported inputs
  cost: {                     // Cost per million tokens
    input: number;
    output: number;
    cacheRead: number;
    cacheWrite: number;
  };
  contextWindow: number;      // Max context size
  maxTokens: number;          // Max output tokens
  headers?: Record<string, string>;  // Model-specific headers
  compat?: OpenAICompletionsCompat;  // Compatibility settings
}

Next Steps

• See Building Extensions for extension basics
• See Creating Skills for specialized workflows
• Check packages/coding-agent/examples/extensions/custom-provider-* for examples