DSPy

DSPy automates the optimization of language model prompts and weights, removing manual tuning from the development process. The framework handles prompt refinement systematically, improving reliability when building complex LLM-based systems.

This guide demonstrates capturing telemetry from DSPy applications using InteractiveAI.

Prerequisites

• InteractiveAI account with API credentials

• LLM provider credentials (OpenAI, Ollama, or other supported provider)

---

Installation

pip install interactiveai dspy openinference-instrumentation-dspy

---

Configuration

Set your API credentials as environment variables:

import os

# InteractiveAI credentials
# Obtain keys from Settings > API Keys in the dashboard
os.environ["INTERACTIVEAI_PUBLIC_KEY"] = "pk-ia-..."
os.environ["INTERACTIVEAI_SECRET_KEY"] = "sk-ia-..."
os.environ["INTERACTIVEAI_HOST"] = "https://app.interactiveai.com"

# Model provider credentials
os.environ["OPENAI_API_KEY"] = "sk-proj-..."

Initialize the client and confirm connectivity:

from interactiveai import Interactive

client = Interactive(
    public_key=os.environ["INTERACTIVEAI_PUBLIC_KEY"],
    secret_key=os.environ["INTERACTIVEAI_SECRET_KEY"],
    host=os.environ.get("INTERACTIVEAI_HOST", "https://app.interactiveai.com")
)

if client.auth_check():
    print("Connection established")
else:
    print("Authentication failed - verify credentials")

---

Enabling Trace Capture

DSPy has a dedicated OpenInference instrumentor that captures all LM calls automatically:

from openinference.instrumentation.dspy import DSPyInstrumentor

DSPyInstrumentor().instrument()

---

Configuring DSPy

Initialize a language model and set it as the default for DSPy operations:

import dspy

lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

---

Running DSPy Modules

With instrumentation active, all DSPy operations generate traces automatically.

Chain-of-Thought Reasoning

math = dspy.ChainOfThought("question -> answer: float")
math(question="Two dice are tossed. What is the probability that the sum equals two?")

Retrieval-Augmented Generation

def search_wikipedia(query: str) -> list[str]:
    results = dspy.ColBERTv2(url="http://20.102.90.50:2017/wiki17_abstracts")(query, k=3)
    return [x["text"] for x in results]

rag = dspy.ChainOfThought("context, question -> response")

question = "What's the name of the castle that David Gregory inherited?"
rag(context=search_wikipedia(question), question=question)

ReAct with Tool Usage

def evaluate_math(expression: str):
    return dspy.PythonInterpreter({}).execute(expression)

def search_wikipedia(query: str):
    results = dspy.ColBERTv2(url="http://20.102.90.50:2017/wiki17_abstracts")(query, k=3)
    return [x["text"] for x in results]

react = dspy.ReAct("question -> answer: float", tools=[evaluate_math, search_wikipedia])

pred = react(question="What is 9362158 divided by the year of birth of David Gregory of Kinnairdy castle?")
print(pred.answer)

source: https://dspy.ai/

---

Enriching Traces with Context

Layer the InteractiveAI SDK on top of DSPy instrumentation to attach identifiers and metadata:

from interactiveai import Interactive
import dspy

client = Interactive(
    public_key=os.environ["INTERACTIVEAI_PUBLIC_KEY"],
    secret_key=os.environ["INTERACTIVEAI_SECRET_KEY"],
    host=os.environ.get("INTERACTIVEAI_HOST", "https://app.interactiveai.com")
)

with client.start_as_current_span(name="dspy-reasoning-task") as span:
    
    client.update_current_trace(
        user_id="user_123",
        session_id="session_abc",
        tags=["dspy", "chain-of-thought"],
        metadata={"experiment": "variant_a", "environment": "production"}
    )
    
    # DSPy operations here
    math = dspy.ChainOfThought("question -> answer: float")
    result = math(question="What is the square root of 144?")
    
    client.update_current_trace(
        input="What is the square root of 144?",
        output=str(result)
    )

client.flush()

---

Trace Visibility

The InteractiveAI dashboard displays:

• LM calls with prompts and completions

• Chain-of-thought reasoning steps

• Tool invocations and retrieval operations

• Token usage and latency per operation