LLM usage metering: track tokens per end-user, across providers

The problem

Your AI app calls OpenAI (or Anthropic, or Gemini). Different users send different prompt sizes, hit different models, and pay you on different plans.

You need to know: how many tokens has each user used this month, across all of their requests, across every provider you call? And: when they hit their plan limit, can the next call return a clean 429 instead of running anyway?

Building this yourself means a counter table, a periodic reset job, atomic increments under concurrency, and an admin UI to debug it. Most teams underestimate how hard the concurrency story is - until two parallel requests both pass canUse().

The solution

AIPricingLab models LLM usage as composite events. One LLM call becomes one event with prompt_tokens and completion_tokens metadata, and AIPricingLab routes it into every limit group it matches - total tokens, per-model tokens, per-tier tokens, anything you defined.

Atomic reserve / commit / release means parallel requests can't both pass the same quota. Reservations auto-release after 60 seconds if your code crashes mid-call.

You don't need to write the counter logic, the period reset, the dashboard, or the developer SDK. Drop in @vevee/sdk, define limit groups in the dashboard, and ship.

Example

After your OpenAI call, you know the actual token counts. Send them as one composite event with prompt and completion as metadata. AIPricingLab counts it against every matching limit group on the user's plan.

import OpenAI from "openai";
import { createClient } from "@vevee/sdk";

const vevee = createClient({ apiKey: process.env.VEVEE_KEY! });
const openai = new OpenAI();

export async function chat(userId: string, prompt: string) {
  // Reserve up to a 4k token bound atomically.
  const r = await vevee.reserve(userId, "llm.tokens", 4000, { model: "gpt-4o" });
  if (!r.allowed) {
    throw new Error(`limit_reached: ${r.reasons?.join(",")}`);
  }

  try {
    const res = await openai.chat.completions.create({
      model: "gpt-4o",
      messages: [{ role: "user", content: prompt }],
    });
    const used =
      (res.usage?.prompt_tokens ?? 0) +
      (res.usage?.completion_tokens ?? 0);

    // Refund the unused portion.
    await vevee.commit(r.reservationId!);
    if (used < 4000) {
      await vevee.track(userId, "llm.tokens.refund", 4000 - used, {
        reservationId: r.reservationId!,
      });
    }
    return res.choices[0]?.message;
  } catch (err) {
    await vevee.release(r.reservationId!);
    throw err;
  }
}

Frequently asked questions

Other use cases