Shared Infrastructure

AI Gateway

CloudGrid provides a built-in AI gateway that routes to Claude models with automatic metering, budgeting, and identity management.

Declare AI access

requires:
  - ai

Install the SDK

npm install @cloudgrid-io/runtime

(The old @cloudgrid-io/ai package still works — it re-exports @cloudgrid-io/runtime.)

Use the SDK

import { runtime } from '@cloudgrid-io/runtime';

const ai = runtime.ai;  // Zero-config: identity auto-detected, no keys in code

// Chat
const response = await ai.chat({
  prompt: 'Summarize this document',
  model: 'claude-sonnet',        // claude-haiku | claude-sonnet | claude-opus
  max_tokens: 1024,
});
console.log(response.text);

// Streaming
for await (const chunk of ai.chat({ prompt: 'Tell me a story', stream: true })) {
  if (chunk.type === 'text') process.stdout.write(chunk.text);
}

// Vision
import { readFile } from 'fs/promises';
const image = await readFile('./screenshot.png');
const analysis = await ai.vision({
  image,
  prompt: 'What is in this image?',
  model: 'claude-sonnet',
});

// Generate (single prompt, no streaming)
const output = await ai.generate({
  prompt: 'Write a haiku about deployment',
  model: 'claude-haiku',
});

// Embeddings — for search / RAG. Use this instead of a personal OpenAI key or a
// third-party vector service; the platform provides the model (1536-dim, matches
// the managed pgvector substrate). Single string → one vector; array → a batch.
const vec = await ai.embeddings({ text: 'hello world' });
// vec.values (number[]), vec.model, vec.dim, vec.tokens
const vecs = await ai.embeddings({ text: ['a', 'b', 'c'] }); // array in input order

Identity resolution

The SDK auto-detects identity:

• In production: Reads the platform-mounted token at /var/run/cloudgrid/identity/token
• In local dev (grid dev): Uses ~/.cloudgrid/credentials + .cloudgrid/link.json

No configuration needed.

Model aliases

Error handling

import { runtime, CloudGridAIError } from '@cloudgrid-io/runtime';
const ai = runtime.ai;

try {
  const response = await ai.chat({ prompt: 'Hello' });
} catch (err) {
  if (err instanceof CloudGridAIError) {
    if (err.code === 'AI_BUDGET_EXCEEDED') {
      // Monthly token cap reached
    }
  }
}

Budget and metering

Per-entity monthly token caps are enforced by the platform. Check usage:

grid info my-app --json

When the cap is exceeded, the gateway returns HTTP 429 with code AI_TOKEN_CAP_EXCEEDED.