<@ULVA73B9P> can I use the marvin3 SDK with ollama...
# ask-marvin
t
@Marvin can I use the marvin3 SDK with ollama?
m
thought for 427.4 seconds
Yes. Marvin 3 uses Pydantic AI under the hood and works with any OpenAI‑compatible endpoint, including Ollama. Quickstart (local Ollama) 1) Start Ollama (defaults to http://localhost:11434) 2) Set the OpenAI‑compatible base URL (note the /v1 suffix):
Copy code
export OLLAMA_BASE_URL="<http://localhost:11434/v1>"
3) Use an Ollama model in Marvin:
Copy code
import marvin

agent = marvin.Agent(model="ollama:llama3.2")  # e.g., llama3.2, mistral, qwen2.5, etc.
print(agent.run("Write a haiku about Prefect"))
Explicit provider setup (if you want to be precise in code)
Copy code
import marvin
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.ollama import OllamaProvider

ollama = OpenAIChatModel(
    "llama3.2",
    provider=OllamaProvider(base_url="<http://localhost:11434/v1>")
)

agent = marvin.Agent(model=ollama)
print(agent.run("What's 2+2?"))
Notes - Use any Ollama model name you’ve pulled (e.g., llama3.2, mistral, deepseek-r1). - If you see connection errors, double‑check that the base URL includes /v1 and that Ollama is running. - You can also route through a LiteLLM proxy if you prefer. Links - Marvin 3 (GitHub) - Pydantic AI – OpenAI (Ollama‑compatible) model - Ollama If you share your target model and environment (local vs. remote), I can tailor the config.
t
@Marvin beautiful... my agent was claiming that you didn't support ollama but that sounded wrong. if i have a full ingestion + digestion pipeline running in prefect, using ollama and begging the model for typed responses, what's the difference between using Pydantic AI, Marvin3, or something else like instructor
m
🔄 Researching your answer... this may take a while 🔧 Using tool:
research_prefect_topic
📊 Tools used: •
research_prefect_topic
(1x)