Token efficient IDs for AI agents

Where UUIDs cost ~23 tokens and get hallucinated by LLMs, id-agent produces memorable word-based IDs at ~14 tokens with equivalent collision resistance. The first ID library built for the context window, not the database.

• Human-readable -- word-based IDs that humans and LLMs can actually remember
• Token-efficient -- every word in the wordlist is exactly 1 BPE token on o200k_base
• Collision-safe -- configurable entropy from ~12 to ~192 bits
• Validated inputs -- zod-powered schema validation on all public APIs

Token Cost Comparison

Install

npm install id-agent

pnpm add id-agent

Quick Start

import { idAgent } from 'id-agent'

// Generate a random ID (8 words, ~96 bits entropy)
const id = idAgent()
// => "urd-antes-sorry-pac-dire-total-expire-going"

// With a type prefix
const taskId = idAgent({ prefix: 'task' })
// => "task_slide-exact-cede-bury-linge-ease-bean-impact"

// Fewer words for short-lived IDs
const short = idAgent({ words: 3 })
// => "front-reject-tho"

API Reference

idAgent(opts?)

Generate a random, human-readable ID.

import { idAgent } from 'id-agent'

idAgent()                          // 8 words, ~96 bits
idAgent({ words: 5 })             // 5 words, ~60 bits
idAgent({ prefix: 'user' })       // "user_cloud-train-scope-frame-match-level-paint-field"

Options:

Invalid options throw a ZodError with a descriptive message.

idAgent.from(input, opts?)

Generate a deterministic ID from a string input using HMAC-SHA256. Same input always produces the same ID.

const id = await idAgent.from('user@example.com')
// Always returns the same ID for the same input

const namespaced = await idAgent.from('user@example.com', {
  namespace: 'my-app',
  prefix: 'user',
  words: 5,
})

Options:

parse(id)

Parse any id-agent ID into its components. Supports both hyphen-separated and underscore-separated words. Returns null for unrecognized formats.

import { parse } from 'id-agent'

parse('task_storm-delta-stone')
// => { prefix: 'task', words: ['storm', 'delta', 'stone'], wordCount: 3, bits: 36, raw: 'task_storm-delta-stone', format: 'readable' }

parse('task_storm_delta_stone')
// => { prefix: 'task', words: ['storm', 'delta', 'stone'], wordCount: 3, bits: 36, raw: 'task_storm_delta_stone', format: 'readable' }

validate(id)

Check if a string is a valid id-agent ID. Validates that all words exist in the WORDLIST.

import { validate } from 'id-agent'

validate('storm-delta-stone')
// => { valid: true, prefix: undefined, wordCount: 3 }

validate('task_jump-notaword')
// => { valid: false, reason: 'unknown words: notaword' }

validate('INVALID')
// => { valid: false, reason: 'contains uppercase characters' }

createAliasMap(opts)

Create a bidirectional alias map for token reduction in LLM contexts. Maps long IDs to short word-based aliases with full replace/restore support.

import { createAliasMap } from 'id-agent'

const aliases = createAliasMap({ words: 3 })
aliases.set('8cdda07b-85d2-459c-8a2a-83c8f9245dbe')
// => "storm-delta-stone" (3 random words from WORDLIST)

aliases.get('storm-delta-stone')
// => "8cdda07b-85d2-459c-8a2a-83c8f9245dbe"

// Replace all UUIDs in text before sending to LLM
const text = 'Process 8cdda07b-85d2-459c-8a2a-83c8f9245dbe then 6ba7b810-9dad-11d1-80b4-00c04fd430c8'
const shortened = aliases.replace(text, {
  pattern: /[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/gi
})
// => "Process storm-delta-stone then cloud-train-scope"

// Restore originals in LLM output
const restored = aliases.restore(shortened)
// => original text

Options:

entries() returns [original, alias] pairs (not [alias, original]). Use get(alias) to look up the original from an alias.

detectDuplicates(opts)

Scan text for duplicate IDs using a regex pattern. Pure function -- no filesystem access.

import { detectDuplicates } from 'id-agent'

const dupes = detectDuplicates({
  pattern: /[a-z]+(?:-[a-z]+)+/,
  text: 'Found storm-delta-stone in file A and storm-delta-stone in file B',
})
// => [{ id: 'storm-delta-stone', count: 2 }]

// Also accepts an array of strings
const dupes2 = detectDuplicates({
  pattern: /task_[a-z]+(?:-[a-z]+)+/,
  text: ['const x = "task_red-fox-run"', 'const y = "task_red-fox-run"'],
})

Options:

WORDLIST

Direct access to the curated 4096-word list. Every word is exactly 1 BPE token on o200k_base. The array is frozen (immutable).

import { WORDLIST } from 'id-agent'

WORDLIST.length          // => 4096
Object.isFrozen(WORDLIST) // => true

The Math

Entropy

Each word is drawn uniformly from a curated 4096-word list (2^12). Every position in the ID is an independent random selection:

Entropy per word = log2(4096) = 12 bits
Total entropy    = words * 12 bits
ID space         = 4096^words = 2^(words * 12)

This holds regardless of individual word length -- a 3-character word and a 6-character word both contribute exactly 12 bits because the attacker must guess from the same 4096-word pool.

Collision Probability (Birthday Paradox)

The probability of at least one collision among n randomly generated IDs:

P(collision) ≈ n^2 / (2 * 2^b)

where b = total bits of entropy

This is the birthday paradox approximation, valid when P is small (P < 0.01).

The default (8 words, 96 bits) is safe for over 300 trillion items before reaching a 50% collision chance. For context, most applications will never generate more than a few million IDs.

Worked Example

For the default 8-word ID at 1 million items:

b = 8 * 12 = 96 bits
n = 1,000,000

P ≈ (10^6)^2 / (2 * 2^96)
  = 10^12 / (2 * 7.92 * 10^28)
  = 10^12 / (1.58 * 10^29)
  = 6.3 * 10^-18

That's roughly 1 in 158 quadrillion -- effectively zero.

Token Cost (Measured)

All measurements on o200k_base (GPT-4o, GPT-4.1, o1, o3) using tiktoken. Token counts vary slightly per ID due to BPE merge behavior with hyphens -- values below are averages over 100 samples:

Why Not Just Fewer Words?

The right word count depends on your scale. The default of 8 is deliberately conservative (global-safe). But if you're building something smaller:

Token Efficiency: Why Words Beat Hex

BPE tokenizers (used by all major LLMs) were trained on natural language. Short English words are single tokens by design. UUIDs are hex strings that split unpredictably:

"storm-delta-stone"  =>  4 tokens (3 words + separators)
"dc193952-186a-4645" =>  11 tokens (same 18 characters!)

id-agent's wordlist is curated so every word is exactly 1 BPE token on o200k_base. The hyphens add ~1 token per 6 words due to BPE merge behavior. This is why word-based IDs are fundamentally more token-efficient than random hex/alphanumeric strings.

How It Works

id-agent uses a curated wordlist of 4096 English words, each verified as a single BPE token on the o200k_base tokenizer (used by GPT-4o, GPT-4.1). Words are 3-6 characters, filtered for offensive terms and homophones.

Random IDs use crypto.getRandomValues() (CSPRNG). Deterministic IDs use HMAC-SHA256 via the Web Crypto API, mapping the hash to wordlist indices.

All public API inputs are validated with zod schemas. Invalid options throw with descriptive error messages.

License

MIT