Documentation

Tokoscope wraps your existing LLM client in two lines and gives you full token visibility, automatic compression, semantic caching, and cost attribution — with zero infrastructure changes.

Quick start

The fastest way to get started:

// 1. Install
// npm install tokoscope

// 2. Wrap your client
import OpenAI from 'openai'
import { wrap } from 'tokoscope'

const client = wrap(new OpenAI(), {
  apiKey: 'ts_live_...'
})

// 3. Use exactly as before
const res = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Hello' }]
})

# 1. Install
# pip install tokoscope

# 2. Wrap your client
from openai import OpenAI
from tokoscope import wrap

client = wrap(OpenAI(), api_key='ts_live_...')

# 3. Use exactly as before
response = client.chat.completions.create(
  model='gpt-4o',
  messages=[{'role': 'user', 'content': 'Hello'}]
)

That's it. Every API call is now tracked automatically. Visit your dashboard to see live data.

Installation

JavaScript / Node.js

npm install tokoscope
# or
yarn add tokoscope

Python

pip install tokoscope

API key

Get your API key from app.tokoscope.com/settings. It starts with ts_live_.

Store it as an environment variable — never hardcode it in your source:

TOKOSCOPE_API_KEY=ts_live_...

const client = wrap(new OpenAI(), {
  apiKey: process.env.TOKOSCOPE_API_KEY
})

OpenAI

import OpenAI from 'openai'
import { wrap } from 'tokoscope'

const client = wrap(new OpenAI(), {
  apiKey: process.env.TOKOSCOPE_API_KEY,
  userId: 'user_123' // optional
})

const res = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Hello' }]
})

from openai import OpenAI
from tokoscope import wrap

client = wrap(OpenAI(),
  api_key='ts_live_...',
  user_id='user_123'  # optional
)

response = client.chat.completions.create(
  model='gpt-4o',
  messages=[{'role': 'user', 'content': 'Hello'}]
)

# Async support
response = await client.chat.completions.acreate(
  model='gpt-4o',
  messages=[{'role': 'user', 'content': 'Hello'}]
)

Anthropic

import Anthropic from '@anthropic-ai/sdk'
import { wrap } from 'tokoscope'

const client = wrap(new Anthropic(), {
  apiKey: process.env.TOKOSCOPE_API_KEY
})

const res = await client.messages.create({
  model: 'claude-sonnet-4-6',
  max_tokens: 1024,
  messages: [{ role: 'user', content: 'Hello' }]
})

from anthropic import Anthropic
from tokoscope import wrap

client = wrap(Anthropic(), api_key='ts_live_...')

response = client.messages.create(
  model='claude-sonnet-4-6',
  max_tokens=1024,
  messages=[{'role': 'user', 'content': 'Hello'}]
)

Gemini

import { GoogleGenerativeAI } from '@google/generative-ai'
import { wrap } from 'tokoscope'

const genAI = wrap(new GoogleGenerativeAI(process.env.GEMINI_KEY), {
  apiKey: process.env.TOKOSCOPE_API_KEY
})

const model = genAI.getGenerativeModel({ model: 'gemini-2.5-flash' })
const result = await model.generateContent('Hello')

import google.generativeai as genai
from tokoscope import wrap

model = wrap(
  genai.GenerativeModel('gemini-2.5-flash'),
  api_key='ts_live_...'
)
result = model.generate_content('Hello')

Per-user tracking

Pass a userId to attribute token usage to individual end users of your app. Users appear in the Users tab with individual token usage, cost, and waste scores.

const client = wrap(new OpenAI(), {
  apiKey: process.env.TOKOSCOPE_API_KEY,
  userId: currentUser.id // any string identifier
})

Caching

Tokoscope automatically caches responses using two layers:

• Exact match — identical prompts return cached responses instantly
• Semantic match — prompts with 85%+ similarity return cached responses using OpenAI embeddings

Cache TTL is 7 days. Clear cache anytime from Settings.

Prompt compression

Prompts with a waste score above 30% are automatically compressed before being tracked. The compressed version is stored alongside the original and visible in the Prompts tab.

Compression uses Claude Haiku and typically reduces prompt length by 40–90% without changing model behavior.

Streaming

Streaming is fully supported across OpenAI, Anthropic, and Gemini. Set stream: true (or stream=True in Python) as usual. Tokoscope passes every chunk through unchanged and tracks token usage once the stream completes.

OpenAI

const stream = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Hello' }],
  stream: true
})

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || '')
}
// Tracking fires automatically once the stream ends

stream = client.chat.completions.create(
  model='gpt-4o',
  messages=[{'role': 'user', 'content': 'Hello'}],
  stream=True
)

for chunk in stream:
  delta = chunk.choices[0].delta.content if chunk.choices else None
  if delta:
    print(delta, end='', flush=True)
# Tracking fires automatically once the stream ends

Anthropic

Both messages.create(stream=True) and the messages.stream() helper are supported.

# Option 1 — create(stream=True)
stream = client.messages.create(
  model='claude-sonnet-4-6',
  max_tokens=1024,
  messages=[{'role': 'user', 'content': 'Hello'}],
  stream=True
)
for event in stream:
  if event.type == 'content_block_delta':
    print(event.delta.text, end='', flush=True)

# Option 2 — the .stream() helper (context manager)
with client.messages.stream(
  model='claude-sonnet-4-6',
  max_tokens=1024,
  messages=[{'role': 'user', 'content': 'Hello'}]
) as stream:
  for text in stream.text_stream:
    print(text, end='', flush=True)
# Tracking fires from the final message usage

Gemini

stream = model.generate_content(
  'Hello',
  stream=True
)
for chunk in stream:
  print(chunk.text, end='', flush=True)
# Tracking reads usage_metadata from the final chunk

Mistral

Both chat.complete() and chat.stream() are supported. Pass stream() directly — Tokoscope wraps it automatically.

// Non-streaming
const result = await client.chat.complete({
  model: 'mistral-large-latest',
  messages: [{ role: 'user', content: 'Hello' }]
})

// Streaming
const stream = await client.chat.stream({
  model: 'mistral-large-latest',
  messages: [{ role: 'user', content: 'Hello' }]
})

for await (const chunk of stream) {
  process.stdout.write(chunk.data?.choices[0]?.delta?.content || '')
}
// Tracking fires automatically once the stream ends

# Non-streaming
result = client.chat.complete(
  model='mistral-large-latest',
  messages=[{'role': 'user', 'content': 'Hello'}]
)

# Streaming
stream = client.chat.stream(
  model='mistral-large-latest',
  messages=[{'role': 'user', 'content': 'Hello'}]
)
for chunk in stream:
  delta = chunk.data.choices[0].delta.content if chunk.data.choices else None
  if delta:
    print(delta, end='', flush=True)

# Async streaming
stream = await client.chat.stream_async(
  model='mistral-large-latest',
  messages=[{'role': 'user', 'content': 'Hello'}]
)
async for chunk in stream:
  print(chunk.data.choices[0].delta.content or '', end='')

Budget alerts + webhooks

Set a monthly spend threshold in Settings. When your spend crosses the threshold, Tokoscope fires an email alert and — if configured — a webhook POST to any HTTPS endpoint. Alerts fire once per calendar month.

Setting up a Slack alert

1. In Slack, go to Apps → Incoming Webhooks → Add to Slack
2. Choose a channel and click Allow
3. Copy the webhook URL (starts with https://hooks.slack.com/services/...)
4. Paste it into the Webhook Alert field in Settings
5. Click Send test to verify delivery

Tokoscope automatically detects Slack URLs and sends a formatted block message with spend, threshold, provider, and a link to your dashboard.

Custom webhook endpoint

Any HTTPS URL works. Tokoscope sends a POST request with a JSON payload when your budget threshold is crossed:

{
  "event": "budget_alert",
  "userId": "usr_...",
  "email": "you@example.com",
  "budgetThreshold": 10,
  "monthlySpend": 10.0142,
  "triggeredAt": "2026-07-14T12:00:00.000Z",
  "provider": "openai",
  "model": "gpt-4o-mini"
}

Testing your webhook

Use the Send test button in Settings to fire a test payload with "test": true immediately, without waiting for your budget threshold to be crossed. The test payload uses your configured threshold value with a spend $0.01 above it.

wrap() options

Python uses snake_case: api_key and user_id.

Supported models & pricing

VS Code Extension

The Tokoscope VS Code extension lets you see token counts, waste scores, and cost estimates for any LLM prompt directly in your editor — without leaving VS Code.

Features

• Status bar token counter — live token count and cost for the current file or selection
• Analyze current file — waste score, suggestions, and cost across all models
• Analyze selection — right-click any text to analyze it
• Auto-analyze on save — optional, disabled by default

Setup

# 1. Open Command Palette
Cmd+Shift+P (Mac) / Ctrl+Shift+P (Windows)

# 2. Run
Tokoscope: Set API Key

# 3. Paste your key from app.tokoscope.com/settings
ts_live_...

VS Code extension

Tokoscope is available as a VS Code extension. See token counts, waste scores, and cost estimates for any prompt or file — without leaving your editor.

What it does

• Status bar counter — live token count and cost for the current file or selection
• Analyze current file — waste score, suggestions, and cost across all models
• Analyze selection — right-click any text to analyze it
• Open dashboard — jump to app.tokoscope.com from the command palette

Setup

1. Cmd+Shift+P → Tokoscope: Set API Key
2. Enter your key from app.tokoscope.com/settings
3. Done — token counts appear in the status bar

Commands

Changelog

• v0.9.0 (Python) / v0.8.0 (JS) — Mistral AI support: chat.complete(), chat.stream(), chat.complete_async(), and chat.stream_async(). Full token tracking and non-streaming cache support.
• v0.8.0 (Python) / v0.7.0 (JS) — Streaming support for Anthropic and Gemini. Anthropic supports create(stream=True) and the .stream() context manager. Gemini supports generate_content(stream=True).
• v0.7.0 (Python) — Streaming support for OpenAI
• VS Code v0.1.0 — VS Code extension published to marketplace
• v0.6.0 — Streaming support for OpenAI (JS). Python: semantic caching, async support via acreate()
• v0.5.0 — Semantic caching (85% similarity threshold) for JS SDK
• v0.4.0 — Gemini support (JavaScript + Python)
• v0.3.0 — 7-day exact match caching, cache stats in dashboard
• v0.2.0 — Per-user token tracking via userId
• v0.1.0 — Initial release: OpenAI + Anthropic, tracking, compression, budget alerts