Skip to content

Arize Toolkit

Arize Toolkit Logo

Welcome to the documentation for Arize Toolkit, a Python client for interacting with the Arize AI API. To get started with the toolkit, check out the Quickstart Guide 🚀 .

Command-Line Interface

The CLI wraps all Client functionality so you can manage Arize resources directly from the terminal. See the CLI Reference for full details.

# Install
pip install arize_toolkit[cli]

# One-time setup
arize_toolkit config init

# List models
arize_toolkit models list

# JSON output for scripting
arize_toolkit --json models list | jq '.[].name'
Command Group Description Docs
config Manage configuration profiles Configuration
models / projects List models, check volume, pull performance metrics Models
monitors Create, list, copy, and delete monitors Monitors
prompts Manage prompt templates and versions Prompts
traces List traces, get spans, and discover columns Traces
spaces / orgs / users Organization and space navigation Spaces
custom-metrics Create and manage custom metrics Custom Metrics
evaluators Manage LLM and code evaluators Evaluators
dashboards Create and view dashboards Dashboards
datasets Retrieve datasets and example data Datasets
imports Manage file and table import jobs Data Imports

Extensions

Claude Code Plugins

  • Plugin Marketplace - Install AI-assisted skills for querying, managing, and developing with the Arize platform

Disclaimer

Although this package is used for development work with and within the Arize platform, it is not an Arize product. It is a open source project developed and maintained by an Arize Engineer. Feel free to add issues or reach out for help in the Arize community Slack channel.

Overview

Arize Toolkit is a set of tools packaged as a Python client that lets you easily interact with Arize AI APIs. Here's a quick overview of the main features in the current release:

  • Access and manage models
  • Retrieve performance metrics over a time period
  • Retrieve inference volume over a time period
  • Create, copy, and manage custom metrics
  • Create, copy, and manage monitors and alerting
  • Work with LLM features like prompts and annotations
  • Import data from cloud storage (S3, GCS, Azure) and databases (BigQuery, Snowflake, Databricks)
  • Create, update, and delete data import jobs with full lifecycle management

Installation

pip install arize_toolkit

Client Setup

The Client class is the entrypoint for interacting with the toolkit. It provides maintains the connection information for making requests to the Arize APIs, and offers a wide range of operations for interacting with models, monitors, dashboards, and more.

API Key

To create a client, you need to provide your Arize API key. Use this reference to get your API key from the Arize UI.

Arize UI Path

Organization and Space

You will also need to provide an organization name and space name. To give some context, models are scoped to a space, and the space is scoped to an organization. These can be found by navigating to the Arize UI and looking at the upper left path in the Projects & Models page. They will be in the format organization/space.

For the example below, the organization is Demo Models and the space is Demo Model Manager.

Arize UI Path

For On Prem deployments

For SaaS users, the default API endpoint is always going to be https://api.arize.com. If you are using an on prem deployment of Arize, you will need to provide the api_url parameter. This parameters should just be the base url of your Arize instance.

For Developers