Skip to content

OpenAI-Compatible APIs

OpenAIClient can be used directly with any API that implements the OpenAI chat completions interface. Use it to connect to providers like Groq, Together AI, Fireworks, Ollama, vLLM, LiteLLM, or any other OpenAI-compatible endpoint.

The named provider clients (MistralClient, GrokClient, VoyageClient) share the OpenAI-compatible base (_OpenAIBase) with OpenAIClient, not OpenAIClient itself.

Configuration

from padwan_ai import OpenAIClient

client = OpenAIClient(
    model="llama-3.3-70b-versatile",
    base_url="https://api.groq.com/openai/v1/",
    api_key="gsk-...",
)

API key fallback

OPENAI_API_KEY is only used when base_url points at api.openai.com. For any other host (Scaleway, a local vLLM, a proxy, ...) a missing api_key falls back to PADWAN_API_KEY, then "no-key-required" — your OpenAI key is never sent to a third-party endpoint. Pass api_key explicitly when the endpoint needs one.

Unified gateway via environment

Aggregators that serve OSS variants of many model families behind a single OpenAI-compatible endpoint and one token are supported without overriding base_url/api_key on every call — or juggling per-provider env vars. Set two environment variables to switch the LLMClient facade into gateway mode:

export PADWAN_BASE_URL="https://your-gateway.example.com/v1/"
export PADWAN_API_KEY="..."

from padwan_ai import LLMClient

# Every model — including gemini-/mistral-/grok- prefixed names — is routed
# through OpenAIClient against the gateway, using the single token.
async with LLMClient(model="gemini-2.5-flash") as client:
    response, usage = await client.complete_chat(
        [{"role": "user", "content": "Hello!"}]
    )

Resolution precedence:

  1. Explicit base_url / api_key arguments to LLMClient
  2. PADWAN_BASE_URL / PADWAN_API_KEY
  3. Native per-provider env vars (OPENAI_API_KEY, GEMINI_API_KEY, …) and model-name routing

Passing an explicit base_url disables gateway mode and restores native per-provider routing (so you can still point GeminiClient at a Gemini-protocol proxy). In gateway mode the token never falls back to OPENAI_API_KEY; if PADWAN_API_KEY is unset it defaults to "no-key-required" for local gateways. Even outside gateway mode, an explicit base_url with no api_key still prefers PADWAN_API_KEY over the provider's native env key, on both LLMClient and RealtimeClient.

Usage

Basic Chat

async with OpenAIClient(
    model="llama-3.3-70b-versatile",
    base_url="https://api.groq.com/openai/v1/",
    api_key="gsk-...",
) as client:
    response, usage = await client.complete_chat(
        [{"role": "user", "content": "Hello!"}]
    )
    print(response["content"])

Streaming

async with OpenAIClient(
    model="llama-3.3-70b-versatile",
    base_url="https://api.groq.com/openai/v1/",
    api_key="gsk-...",
) as client:
    stream = client.stream_chat([{"role": "user", "content": "Tell me a story"}])
    async for chunk in stream:
        print(chunk, end="", flush=True)

    if stream.usage:
        print(f"\nTokens used: {stream.usage['total']}")

Examples

Ollama (local)

async with OpenAIClient(
    model="llama3",
    base_url="http://localhost:11434/v1/",
    api_key="ollama",  # Ollama doesn't require a real key
) as client:
    response, usage = await client.complete_chat(
        [{"role": "user", "content": "Hello!"}]
    )

Together AI

async with OpenAIClient(
    model="meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo",
    base_url="https://api.together.xyz/v1/",
    api_key="...",
) as client:
    response, usage = await client.complete_chat(
        [{"role": "user", "content": "Hello!"}]
    )

vLLM

async with OpenAIClient(
    model="Qwen/Qwen3-0.6B",
    base_url="http://localhost:8000/v1/",
    api_key="token-abc123",
    on_thought=lambda t: print(f"[thinking] {t}"),
) as client:
    response, usage = await client.complete_chat(
        [{"role": "user", "content": "Hello!"}]
    )

Start vLLM with a --reasoning-parser (e.g. vllm serve Qwen/Qwen3-0.6B --reasoning-parser qwen3) so reasoning arrives in the reasoning field and is forwarded to on_thought instead of the answer text.

Subclassing

To create a reusable client for a specific provider, subclass OpenAIClient:

import dataclasses
import os
from typing import ClassVar
from padwan_ai import OpenAIClient
from padwan_ai.errors import LLMError


@dataclasses.dataclass
class GroqClient(OpenAIClient):
    provider: ClassVar[str] = "groq"
    model: str | None = "llama-3.3-70b-versatile"
    base_url: str = "https://api.groq.com/openai/v1/"

    def _get_default_api_key(self) -> str:
        api_key = os.environ.get("GROQ_API_KEY")
        if not api_key:
            raise LLMError(self.provider, "GROQ_API_KEY not set")
        return api_key