Haystack

Deepset created Haystack as a modular Python framework for constructing production-grade LLM applications. The architecture supports custom pipeline configurations for retrieval-augmented generation, advanced search systems, and other LLM workflows. Haystack connects with Hugging Face Transformers, Elasticsearch, OpenSearch, OpenAI, Cohere, Anthropic, and numerous other services.

This guide covers capturing telemetry from Haystack applications using InteractiveAI and OpenTelemetry instrumentation.

Prerequisites

• InteractiveAI account with API credentials

• OpenAI API key

• SerperDev API key (for web search functionality)

---

Installation

pip install haystack-ai interactiveai openinference-instrumentation-haystack

---

Configuration

Set your API credentials as environment variables:

import os

# InteractiveAI credentials
# Obtain keys from Settings > API Keys in the dashboard
os.environ["INTERACTIVEAI_PUBLIC_KEY"] = "pk-ia-..."
os.environ["INTERACTIVEAI_SECRET_KEY"] = "sk-ia-..."
os.environ["INTERACTIVEAI_HOST"] = "https://app.interactiveai.com"

# Model and tool provider credentials
os.environ["OPENAI_API_KEY"] = "sk-proj-..."
os.environ["SERPERDEV_API_KEY"] = "..."

Initialize the client and confirm connectivity:

from interactiveai import Interactive

client = Interactive(
    public_key=os.environ["INTERACTIVEAI_PUBLIC_KEY"],
    secret_key=os.environ["INTERACTIVEAI_SECRET_KEY"],
    host=os.environ.get("INTERACTIVEAI_HOST", "https://app.interactiveai.com")
)

if client.auth_check():
    print("Connection established")
else:
    print("Authentication failed - verify credentials")

---

Enabling Trace Capture

Haystack has a dedicated OpenInference instrumentor that captures pipeline operations automatically:

from openinference.instrumentation.haystack import HaystackInstrumentor

HaystackInstrumentor().instrument()

Once activated, all pipeline components and model calls generate spans that route to InteractiveAI.

---

Running a Haystack Application

Here's a working example using an agent with web search capabilities:

from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import ComponentTool
from haystack.components.websearch import SerperDevWebSearch
from interactiveai import Interactive

client = Interactive(
    public_key=os.environ["INTERACTIVEAI_PUBLIC_KEY"],
    secret_key=os.environ["INTERACTIVEAI_SECRET_KEY"],
    host=os.environ.get("INTERACTIVEAI_HOST", "https://app.interactiveai.com")
)

search_tool = ComponentTool(component=SerperDevWebSearch())

agent = Agent(
    chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
    system_prompt="You are a helpful research assistant.",
    tools=[search_tool],
)

result = agent.run(messages=[ChatMessage.from_user("What are the main features of the Haystack framework?")])

print(result["last_message"].text)

client.flush()

---

Enriching Traces with Context

Combine OpenInference instrumentation with the InteractiveAI SDK to attach identifiers and metadata:

from interactiveai import Interactive
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.tools import ComponentTool
from haystack.components.websearch import SerperDevWebSearch

client = Interactive(
    public_key=os.environ["INTERACTIVEAI_PUBLIC_KEY"],
    secret_key=os.environ["INTERACTIVEAI_SECRET_KEY"],
    host=os.environ.get("INTERACTIVEAI_HOST", "https://app.interactiveai.com")
)

with client.start_as_current_span(name="haystack-search-task") as span:
    
    client.update_current_trace(
        user_id="user",
        session_id="session",
        tags=["haystack", "web-search"],
        metadata={"experiment": "baseline", "environment": "production"}
    )
    
    search_tool = ComponentTool(component=SerperDevWebSearch())
    
    agent = Agent(
        chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
        system_prompt="You are a helpful research assistant.",
        tools=[search_tool],
    )
    
    query = "Latest developments in renewable energy"
    result = agent.run(messages=[ChatMessage.from_user(query)])
    
    client.update_current_trace(
        input=query,
        output=result["last_message"].text
    )

client.flush()

---

Trace Visibility

The InteractiveAI dashboard displays:

• Pipeline execution flow and component interactions

• Model requests with prompts and completions

• Tool invocations and search results

• Token consumption and latency for each step