Skip to main content

OpenInference Guardrails Instrumentation

Project description

OpenInference guardrails Instrumentation

pypi

Python auto-instrumentation library for LLM applications implemented with Guardrails

Guards are fully OpenTelemetry-compatible and can be sent to an OpenTelemetry collector for monitoring, such as arize-phoenix.

Installation

pip install openinference-instrumentation-guardrails

Quickstart

This quickstart shows you how to instrument your guardrailed LLM application

Install required packages.

pip install openinference-instrumentation-guardrails guardrails-ai arize-phoenix opentelemetry-sdk opentelemetry-exporter-otlp

Start Phoenix in the background as a collector. By default, it listens on http://localhost:6006. You can visit the app via a browser at the same address. (Phoenix does not send data over the internet. It only operates locally on your machine.)

python -m phoenix.server.main serve

Install the TwoWords validator that's used in the Guard.

guardrails hub install hub://guardrails/two_words

Set up GuardrailsInstrumentor to trace your guardrails application and sends the traces to Phoenix at the endpoint defined below.

from openinference.instrumentation.guardrails import GuardrailsInstrumentor
from opentelemetry import trace as trace_api
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk import trace as trace_sdk
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
import os

os.environ["OPENAI_API_KEY"] = "YOUR_KEY_HERE"

endpoint = "http://127.0.0.1:6006/v1/traces"
tracer_provider = trace_sdk.TracerProvider()
tracer_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter(endpoint)))
trace_api.set_tracer_provider(tracer_provider)

GuardrailsInstrumentor().instrument()

Set up a simple example of LLM call using a Guard

from guardrails import Guard
from guardrails.hub import TwoWords
import openai

guard = Guard().use(
    TwoWords(),
)

response = guard(
    llm_api=openai.chat.completions.create,
    prompt="What is another name for America?",
    model="gpt-3.5-turbo",
    max_tokens=1024,
)

print(response)

More Info

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Built Distribution

File details

Details for the file openinference_instrumentation_guardrails-0.1.9.tar.gz.

File metadata

File hashes

Hashes for openinference_instrumentation_guardrails-0.1.9.tar.gz
Algorithm Hash digest
SHA256 1bb0ed5b3587f3f6658c980836efbec3616fca7185b464409b123ea0dfe83ee4
MD5 5fd58d5de99805ad56ccf709a1273958
BLAKE2b-256 363803a06449a694ee97ad8842704032ca4a6283037312c9ab985bf2654568db

See more details on using hashes here.

File details

Details for the file openinference_instrumentation_guardrails-0.1.9-py3-none-any.whl.

File metadata

File hashes

Hashes for openinference_instrumentation_guardrails-0.1.9-py3-none-any.whl
Algorithm Hash digest
SHA256 16a4d682a1b3a2c75ac9ff1ae1ba5cc1c19f657e90e6d40a5056b6273758209c
MD5 3db56af16080ac912749fa85a22a046f
BLAKE2b-256 21a82d12beae8816cd01ff79d32618e26da40402b24af1038a9c1c37a853f25b

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page
close