Multimodal Content

User messages can carry a list of content parts (text, images, and audio) instead of a plain string. Parts use the OpenAI content-part shape: OpenAI-compatible providers (OpenAI, Grok) receive them verbatim, the Gemini client converts them to inlineData parts, and the Mistral client rewrites audio parts to its plain base64 input_audio chunk.

Building parts

content_parts infers the part type per item:

from pathlib import Path

from padwan_ai import content_parts

parts = content_parts(
    "What is in this screenshot?",  # str -> text part
    Path("shot.png"),  # image extension -> base64 data: URL image part
    Path("voice.wav"),  # wav/mp3 extension -> base64 audio part
    Path("notes.md"),  # any other file -> inlined text file part
)

Strings are always text (never treated as paths), so message text that mentions a filename is safe; wrap files in Path to have them read. Files are classified by extension: an image MIME type yields an image part, a wav/mp3 audio MIME type an audio part (other audio formats raise ValueError), anything else is inlined as text.

The explicit builders remain for full control:

from padwan_ai import audio_part, image_part, text_file_part, text_part

parts = [
    text_part("What is in this screenshot?"),
    image_part("shot.png"),  # reads the file into a base64 data: URL
    audio_part("voice.wav"),  # reads the file into a base64 audio part
    text_file_part("notes.md"),  # inlines a text file, labelled with its name
]

image_part accepts a path (str | Path) and guesses the MIME type from the file name; pass mime= to override. Unknown extensions fall back to image/png. audio_part accepts a path (str | Path) and guesses the format from the extension (wav, mp3, flac, ogg, aac, aiff, m4a); fmt= overrides the guess and unknown extensions raise ValueError. Supported audio extensions take precedence over host MIME mappings. Format support varies per provider (see the audio table below). text_file_part reads the file as UTF-8 (encoding= overrides) and prefixes the text with --- <name> --- so the model can tell files apart.

Sending images

from padwan_ai import ConversationState, LLMClient, image_part, text_part

state = ConversationState()
state.add_user_message(
    [
        text_part("Describe this image."),
        image_part("shot.png"),
    ]
)

async with LLMClient("gpt-4o") as client:
    response, usage = await client.complete_chat(state.messages)

Checking vision support

supports_vision is a best-effort, curated check of whether a model accepts image input:

from padwan_ai import supports_vision

supports_vision("gpt-4o")  # True
supports_vision("mistral-large-latest")  # False - text-only
supports_vision("some-local-model")  # True - unknown models are attempted

Each provider package exposes its own supports_vision (e.g. padwan_ai.mistral.supports_vision); the top-level function dispatches to them in the same order as LLMClient routing. Unknown models default to True so the request is attempted and the provider surfaces the real error instead of the check guessing wrong.

Checking audio support

supports_audio is the audio counterpart of supports_vision, with the same dispatch and unknown-model behaviour; pass fmt= to also check a specific format:

from padwan_ai import supports_audio

supports_audio("gpt-audio")  # True
supports_audio("gpt-audio", "flac")  # False - OpenAI takes wav/mp3 only
supports_audio("gemini-2.5-flash", "flac")  # True
supports_audio("grok-4")  # False - no Grok chat model takes audio

Each audio-capable provider package also exposes its accepted formats as AUDIO_FORMATS (e.g. padwan_ai.gemini.AUDIO_FORMATS).

Limitations

• No video parts, image generation, or audio generation.
• Content parts are for user messages; assistant messages stay text-only.
• The Anthropic client passes message content through unconverted, so image and audio parts are not usable with Claude models yet (and the Messages API takes no audio anyway).