OpenAI

Integrate OpenAI with Weave for tracing, evaluation, and monitoring

Do you want to experiment with OpenAI models on Weave without any set up? Try the LLM Playground.

Tracing

It’s important to store traces of LLM applications in a central database, both during development and in production. You’ll use these traces for debugging and to help build a dataset of tricky examples to evaluate against while improving your application.

Weave can automatically capture traces for the openai python library.

Start capturing by calling weave.init(<project-name>) with a project name of your choice. OpenAI will be automatically patched regardless of when you import it.

If you don’t specify a W&B team when you call weave.init(), your default entity is used. To find or update your default entity, refer to User Settings in the W&B Models documentation.

Automatic Patching

Weave automatically patches OpenAI whether imported before or after weave.init():

Python

from openai import OpenAI
import weave

# highlight-next-line
weave.init('emoji-bot')  # OpenAI is automatically patched!

client = OpenAI()
response = client.chat.completions.create(
  model="gpt-4",
  messages=[
    {
      "role": "system",
      "content": "You are AGI. You will be provided with a message, and your task is to respond using emojis only."
    },
    {
      "role": "user",
      "content": "How are you?"
    }
  ]
)

import { OpenAI } from 'openai';
import { wrapOpenAI } from '@wandb/weave';

const openai = wrapOpenAI(new OpenAI());

// This will now trace all calls to OpenAI
openai.chat.completions.create(
  {
    model: "gpt-4",
    messages: [
      {
        role: "system",
        content: "You are AGI. You will be provided with a message, and your task is to respond using emojis only."
      },
      {
        role: "user",
        content: "How are you?"
      }
    ]
  }
);

Explicit Patching (Optional)

You can still explicitly patch if you want fine-grained control:

import weave

# highlight-next-line
weave.init('emoji-bot')
# highlight-next-line
weave.integrations.patch_openai()  # Enable OpenAI tracing

from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
  model="gpt-4",
  messages=[
    {"role": "user", "content": "Make me a emoji"}
  ]
)

View a live trace

We capture the function calling tools for OpenAI Functions and OpenAI Assistants as well.

Structured Outputs

Weave also supports structured outputs with OpenAI. This is useful for ensuring that your LLM responses follow a specific format.

Python

from openai import OpenAI
from pydantic import BaseModel
import weave

class UserDetail(BaseModel):
    name: str
    age: int

client = OpenAI()
weave.init('extract-user-details')

completion = client.beta.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    messages=[
        {"role": "system", "content": "Extract the user details from the message."},
        {"role": "user", "content": "My name is David and I am 30 years old."},
    ],
    response_format=UserDetail,
)

user_detail = completion.choices[0].message.parsed
print(user_detail)

Async Support

Weave also supports async functions for OpenAI.

Python

from openai import AsyncOpenAI
import weave

client = AsyncOpenAI()
weave.init('async-emoji-bot')

async def call_openai():
    response = await client.chat.completions.create(
        model="gpt-4",
        messages=[
            {
                "role": "system", 
                "content": "You are AGI. You will be provided with a message, and your task is to respond using emojis only."
            },
            {
                "role": "user",
                "content": "How are you?"
            }
        ]
    )
    return response

# Call the async function
result = await call_openai()

Streaming Support

Weave also supports streaming responses from OpenAI.

Python

from openai import OpenAI
import weave

client = OpenAI()
weave.init('streaming-emoji-bot')

response = client.chat.completions.create(
    model="gpt-4",
    messages=[
        {
            "role": "system", 
            "content": "You are AGI. You will be provided with a message, and your task is to respond using emojis only."
        },
        {
            "role": "user",
            "content": "How are you?"
        }
    ],
    stream=True
)

for chunk in response:
    print(chunk.choices[0].delta.content or "", end="")

Tracing Function Calls

Weave also traces function calls made by OpenAI when using tools.

Python

from openai import OpenAI
import weave

client = OpenAI()
weave.init('function-calling-bot')

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The location to get the weather for"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "The unit to return the temperature in"
                    }
                },
                "required": ["location"]
            }
        }
    }
]

response = client.chat.completions.create(
    model="gpt-4",
    messages=[
        {
            "role": "user",
            "content": "What's the weather like in New York?"
        }
    ],
    tools=tools
)

print(response.choices[0].message.tool_calls)

Batch API

Weave also supports the OpenAI Batch API for processing multiple requests.

Python

from openai import OpenAI
import weave

client = OpenAI()
weave.init('batch-processing')

# Create a batch file
batch_input = [
    {
        "custom_id": "request-1",
        "method": "POST",
        "url": "/v1/chat/completions",
        "body": {
            "model": "gpt-4",
            "messages": [{"role": "user", "content": "Hello, how are you?"}]
        }
    },
    {
        "custom_id": "request-2", 
        "method": "POST",
        "url": "/v1/chat/completions",
        "body": {
            "model": "gpt-4",
            "messages": [{"role": "user", "content": "What's the weather like?"}]
        }
    }
]

# Submit the batch
batch = client.batches.create(
    input_file_id="your-file-id",
    endpoint="/v1/chat/completions",
    completion_window="24h"
)

# Retrieve the batch results
completed_batch = client.batches.retrieve(batch.id)

Assistants API

Weave also supports the OpenAI Assistants API for building conversational AI applications.

Python

from openai import OpenAI
import weave

client = OpenAI()
weave.init('assistant-bot')

# Create an assistant
assistant = client.beta.assistants.create(
    name="Math Assistant",
    instructions="You are a personal math tutor. Answer questions about math.",
    model="gpt-4"
)

# Create a thread
thread = client.beta.threads.create()

# Add a message to the thread
message = client.beta.threads.messages.create(
    thread_id=thread.id,
    role="user",
    content="What is 2+2?"
)

# Run the assistant
run = client.beta.threads.runs.create(
    thread_id=thread.id,
    assistant_id=assistant.id
)

# Get the assistant's response
messages = client.beta.threads.messages.list(thread_id=thread.id)

Cost Tracking

Weave automatically tracks the cost of your OpenAI API calls. You can view the cost breakdown in the Weave UI.

Cost tracking is available for all OpenAI models and is calculated based on the latest OpenAI pricing.

Tracing Custom Functions

You can also trace custom functions that use OpenAI by using the @weave.op decorator.

Python

from openai import OpenAI
import weave

client = OpenAI()
weave.init('custom-function-bot')

@weave.op
def generate_response(prompt: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4",
        messages=[
            {
                "role": "user",
                "content": prompt
            }
        ]
    )
    return response.choices[0].message.content

# This function call will be traced
result = generate_response("Hello, how are you?")

Next Steps

Now that you’ve set up tracing for OpenAI, you can:

1. View traces in the Weave UI: Go to your Weave project to see traces of your OpenAI calls
2. Create evaluations: Use your traces to build evaluation datasets
3. Monitor performance: Track latency, costs, and other metrics
4. Debug issues: Use traces to understand what’s happening in your LLM application

For more information on these topics, check out our evaluation guide and monitoring guide.