> ## Documentation Index
> Fetch the complete documentation index at: https://tbd-6fc993ce-hypeship-browser-pools-docs-refresh.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# Scale
> Recommended practices for scaling in production
## Overview
This guide covers how to run Kernel in production at scale — which architecture to build around browser creation, and when a browser pool is the right tool. It assumes you're comfortable [creating](/introduction/create) and [controlling](/introduction/control) browsers; for the mechanics of standing up a pool and acquiring from it, see [Browser Pools](/browsers/pools).
## Why a browser pool
A [browser pool](/browsers/pools) keeps a set of identically-configured browsers ready for immediate use. Compared to creating browsers on demand, it gives you:
* **Low-latency acquisition** — the browser is already booted with your configuration applied (including settings like custom viewports, extensions, and kiosk-mode live view that otherwise [restart Chromium](/browsers/performance#troubleshooting-latency) on a fresh browser), so `acquire` hands you one that's ready to drive.
* **Reserved, pre-configured capacity** — a fixed set of browsers on your exact configuration, ready before traffic arrives.
* **Higher creation throughput** — acquiring from a pool isn't subject to the [rate limit](/info/pricing#rate-limiting) on `browsers.create()` that high-volume workloads hit.
The tradeoff: a browser pool counts against your concurrency limit whether or not its browsers are currently acquired — a pool sized to 40 holds 40 of your limit. Idle pooled browsers aren't billed, but they hold the slot.
## When to use a pool vs on-demand
Reach for a **browser pool** when:
* you're running the same workload repeatedly, in production
* acquisition latency matters — a cold start is unacceptable (for example, a synchronous, user-facing action)
* traffic is steady or high-frequency enough to keep reserved browsers utilized
* you're hitting the `browsers.create()` rate limit at volume
Stick with **on-demand `browsers.create()`** when:
* volume is low, bursty, one-off, or you're still developing
* each session needs a different configuration (a pool is one fixed config)
* you need a GPU browser (not available in pools)
Concurrency and request patterns are how you *size* a pool once you've decided to use one — not a threshold that gates whether pools are worth it. Even a small pool pays off when acquisition latency matters and demand is steady.
## Sizing
Watch `available_count` and target 10–20% available under normal load, resizing before traffic peaks rather than during them. See [Sizing a browser pool](/browsers/pools#sizing-a-browser-pool) for the full guidance.
## Architecture patterns
### Direct browser creation (POC)
For proof-of-concept work and early production systems with modest concurrency needs, creating browsers on-demand is the simplest approach.
**When to use:**
* Low or unpredictable volume
* Infrequent or one-off workloads
* Early development and testing
```typescript theme={null}
import Kernel from '@onkernel/sdk';
import { chromium } from 'playwright';
const kernel = new Kernel();
async function processTask(taskData: any) {
// Create browser with extended timeout for long tasks
const session = await kernel.browsers.create({
stealth: true,
timeout_seconds: 3600, // Destroy browser after 1 hour of inactivity
});
try {
const browser = await chromium.connectOverCDP(session.cdp_ws_url);
const context = browser.contexts()[0];
const page = context.pages()[0];
// Your automation logic here
await page.goto(taskData.url);
// ... perform work ...
return { success: true, data: /* results */ };
} catch (error) {
console.error('Task failed:', error);
return { success: false, error: error.message };
} finally {
// Always clean up
await kernel.browsers.deleteByID(session.session_id);
}
}
```
### Single browser pool (scaling)
For production systems with consistent, high-frequency workloads, a browser pool allows you to access higher concurrency plus predictable performance.
**When to use:**
* Consistent, high-frequency workloads on a fixed configuration
* Steady request patterns, or latency-sensitive acquisition
```typescript theme={null}
import Kernel from '@onkernel/sdk';
import { chromium } from 'playwright';
const kernel = new Kernel();
const POOL_NAME = 'production-pool';
// Initialize pool once (typically in deployment/startup)
async function initializePool() {
await kernel.browserPools.create({
name: POOL_NAME,
size: 25, // Balance cost and availability
timeout_seconds: 300, // Destroy browsers after 5 minutes of inactivity
stealth: true,
headless: false, // headless: true for cost savings if no live view needed
});
}
async function processTask(taskData: any) {
let session;
try {
// Acquire browser (returns immediately if available)
session = await kernel.browserPools.acquire(POOL_NAME, {
acquire_timeout_seconds: 30, // Wait up to 30s for availability
});
const browser = await chromium.connectOverCDP(session.cdp_ws_url);
const context = browser.contexts()[0];
const page = context.pages()[0];
// Perform work
await page.goto(taskData.url);
// ... automation logic ...
return { success: true, data: /* results */ };
} catch (error) {
console.error('Task failed:', error);
return { success: false, error: error.message };
} finally {
// Critical: Always release back to pool
if (session) {
await kernel.browserPools.release(POOL_NAME, {
session_id: session.session_id,
reuse: true, // Reuse for efficiency
});
}
}
}
```
**Key considerations:**
* Pool size should match your typical concurrency
* Always release browsers in a `finally` block to prevent browser pool exhaustion
* Set `acquire_timeout_seconds` based on your SLA requirements
### Queue-based processing (high scale)
For systems exceeding browser pool capacity or with unpredictable bursts, implement a task queue to manage workloads gracefully.
**When to use:**
* Request volume exceeds a single browser pool's capacity
* Highly variable traffic patterns
* Need to prioritize certain tasks
* Want to decouple request ingestion from processing
```typescript theme={null}
import { Queue } from 'bullmq'; // or any queue system
import Kernel from '@onkernel/sdk';
const kernel = new Kernel();
const POOL_NAME = 'production-pool';
const POOL_SIZE = 50;
// Task queue configuration
const taskQueue = new Queue('browser-tasks', {
connection: { /* Redis config */ }
});
// Worker that processes tasks
async function startWorker() {
const worker = new Worker('browser-tasks', async (job) => {
let session;
try {
// Acquire browser with reasonable timeout
session = await kernel.browserPools.acquire(POOL_NAME, {
acquire_timeout_seconds: 120,
});
const browser = await chromium.connectOverCDP(session.cdp_ws_url);
const context = browser.contexts()[0];
const page = context.pages()[0];
// Process job data
await page.goto(job.data.url);
const result = await page.evaluate(() => /* extract data */);
return { success: true, data: result };
} catch (error) {
// Handle errors with retry logic
if (error.message.includes('timeout')) {
throw new Error('RETRY'); // BullMQ will retry
}
throw error;
} finally {
if (session) {
await kernel.browserPools.release(POOL_NAME, {
session_id: session.session_id,
reuse: true,
});
}
}
}, {
connection: { /* Redis config */ },
concurrency: POOL_SIZE, // Match pool size
});
return worker;
}
// Add tasks to queue
async function submitTask(taskData: any, priority?: number) {
await taskQueue.add('process', taskData, {
priority: priority || 5,
attempts: 3,
backoff: {
type: 'exponential',
delay: 2000,
},
});
}
```
**Queue-specific considerations:**
* Set worker concurrency to match or slightly exceed browser pool size
* Implement proper retry logic for transient failures
* Monitor queue depth to scale browser pools dynamically
* Use priority queues for different SLAs