Skip to content
Cap'n Web

Batch + pipelining

Three dependent RPC calls in a single HTTP round trip, running live, with the source that produces it.

Updated View as Markdown
// The two strategies being compared, and the fake network they run over.
// No DOM in this file -- main.js does the wiring, so this stays readable as
// an answer to "what is the actual difference between the two approaches?".
import { newHttpBatchRpcSession } from './vendor/capnweb.js';

export const RPC_URL = new URL('/rpc', location.href).href;

const JITTER_MS = 40;
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

/**
 * Run `fn` with `fetch` wrapped so each RPC POST is counted and padded with
 * simulated uplink and downlink latency. Restores the real fetch afterwards so
 * a failed run cannot leave the page in a patched state.
 *
 * Latency is simulated on this side on purpose: the server does exactly the
 * same work in both columns, so the difference you see is round trips and
 * nothing else.
 */
async function withSimulatedNetwork(rttMs, fn) {
	const realFetch = globalThis.fetch.bind(globalThis);
	const latency = () => rttMs + Math.random() * JITTER_MS;
	let posts = 0;

	globalThis.fetch = async (input, init) => {
		const url = input instanceof Request ? input.url : String(input);
		const method = init?.method ?? (input instanceof Request ? input.method : 'GET');
		if (url.startsWith(RPC_URL) && method === 'POST') {
			posts++;
			await sleep(latency());
			const response = await realFetch(input, init);
			await sleep(latency());
			return response;
		}
		return realFetch(input, init);
	};

	const started = performance.now();
	try {
		const value = await fn();
		return { value, posts, ms: performance.now() - started };
	} finally {
		globalThis.fetch = realFetch;
	}
}

// One session. `user` is never awaited before being used, so `user.id` is sent
// as a pipelined reference rather than a resolved value.
export const pipelined = (rttMs) =>
	withSimulatedNetwork(rttMs, async () => {
		const api = newHttpBatchRpcSession(RPC_URL);
		const user = api.authenticate('cookie-123');
		const profile = api.getUserProfile(user.id);
		const notifications = api.getNotifications(user.id);
		const [u, p, n] = await Promise.all([user, profile, notifications]);
		return { user: u, profile: p, notifications: n };
	});

// Three sessions, each awaited before the next can be built.
export const sequential = (rttMs) =>
	withSimulatedNetwork(rttMs, async () => {
		const user = await newHttpBatchRpcSession(RPC_URL).authenticate('cookie-123');
		const profile = await newHttpBatchRpcSession(RPC_URL).getUserProfile(user.id);
		const notifications = await newHttpBatchRpcSession(RPC_URL).getNotifications(user.id);
		return { user, profile, notifications };
	});
/playground/batch-pipelining/index.htmlOpen

Authenticate a user, then fetch that user’s profile and notifications, both of which need the user ID that the first call returns. Pipelined, all three travel in one HTTP request. Written the ordinary way, they take three.

Drag the latency slider and run it again. The server does identical work in both columns; the only difference is how many times the browser has to cross the network.

What makes it one round trip

The three calls are issued against a single session, and the second and third are built from a promise that has not resolved yet:

const api = newHttpBatchRpcSession(RPC_URL);

const user = api.authenticate('cookie-123');       // not awaited
const profile = api.getUserProfile(user.id);       // uses user.id anyway
const notifications = api.getNotifications(user.id);

const [u, p, n] = await Promise.all([user, profile, notifications]);

user.id is not a string here. It is a reference to a field of a result the server has not produced yet. Cap’n Web sends that reference as part of the same batch, and the server substitutes the real value when it gets there. Nothing has to come back to the client in between.

The sequential version awaits each call before building the next, so the client cannot know user.id until a full round trip has completed:

const user = await newHttpBatchRpcSession(RPC_URL).authenticate('cookie-123');
const profile = await newHttpBatchRpcSession(RPC_URL).getUserProfile(user.id);
const notifications = await newHttpBatchRpcSession(RPC_URL).getNotifications(user.id);

Same data, same server work, three times the network cost, and that cost grows with the length of the chain, which is the part that hurts on a slow connection.

Where the latency comes from

Two separate knobs, deliberately kept apart:

  • Server-side work: per-method delays set by DELAY_AUTH_MS, DELAY_PROFILE_MS and DELAY_NOTIFS_MS. Identical in both modes, so it is not what the demo measures.
  • Network round trips: simulated in the browser by the slider. This is the part pipelining removes.

Keeping the round-trip cost on the client means the deployed Worker adds no artificial delay, and the page can change it without a redeploy.

How this page runs

The demo above is the example’s unmodified Worker and browser client, bundled together into one page. A small shim replaces fetch for the /rpc path and hands the request straight to the Worker’s fetch handler:

globalThis.fetch = async (input, init) => {
  const request = new Request(input, init);
  if (new URL(request.url).pathname === '/rpc') {
    return await worker.fetch(request, ENV, ctx);
  }
  return upstream(input, init);
};

Everything above that line is untouched: the same session setup, the same batch encoding, the same newWorkersRpcResponse on the other end. Only the transport hop is gone, which is why the round-trip counts mean what they say and why these docs deploy as static files.

Run it yourself

npm run build   # the examples resolve capnweb to dist/
npx wrangler dev --cwd examples/batch-pipelining --ip 127.0.0.1 --port 8788

The terminal client runs the same comparison against a server in a separate process:

node examples/batch-pipelining/server-node.mjs   # in one shell
node examples/batch-pipelining/client.mjs        # in another

Next

Edit this page
Navigation

Type to search…

↑↓ navigate↵ selectEsc close