Paint by Language Model — Programmer API

Canvas

Size: 800 × 600 pixels
Background: white (#FFFFFF)
Coordinate origin: (0, 0) = top-left corner; positive X goes right, positive Y goes down
Bottom-right corner: (800, 600)

For LLMs

This API allows a language model or external script to programmatically paint on the canvas by calling methods on window.paintByLanguageModel in the browser console, or in a Playwright / Puppeteer script aimed at the /draw page.

The object is registered when the draw page mounts and removed when it unmounts. Tool-setting calls update the React state that backs the toolbar UI, so changes are reflected visually in real time.

Scripted / Playwright contexts — async state: Tool-setting methods such as selectStrokeType(), setColor(), and setThickness() trigger React state updates, which are asynchronous. Always set all tool properties first, then await sleep(50) before placing clicks. sleep(0) is NOT reliable — React schedules renders via its own internal scheduler and may not have re-rendered the canvas component by the time a setTimeout(0) resolves. Use const sleep = (ms) => new Promise(r => setTimeout(r, ms)) and await sleep(50) for consistent results.

Clicks required per stroke type:

Stroke type	Commits after
line, arc, burn, dodge	2 clicks (start → end)
circle, splatter	2 clicks (centre → radius point)
polyline, dry-brush, chalk, wet-brush	≥2 clicks to add points, then doubleClick() to commit

Quick Start

// Open /draw in the browser, then paste this into the DevTools console:

const api = window.paintByLanguageModel;
const sleep = (ms) => new Promise(r => setTimeout(r, ms));

// All tool-setting calls (selectStrokeType, setColor, setThickness, etc.) trigger
// async React state updates. Call all setters first, then await sleep(0) to let
// React flush them, then place your clicks.

await (async () => {
  // 1. Configure the tool
  api.selectStrokeType("line");
  api.setColor("#1a1a1a");
  api.setThickness(4);
  await sleep(50); // wait for all setters to flush before clicking

  // 2. Draw a line — click start point, then end point
  api.click(100, 300);
  api.click(700, 300);

  // Done! A horizontal line is now on the canvas.
})();

Method Reference

Tool Config

selectStrokeType

window.paintByLanguageModel.selectStrokeType(type: string): void

Set the active stroke type. Changes are reflected in the toolbar UI. IMPORTANT for scripted / Playwright contexts: this call triggers a React state update, which is asynchronous. If you call click() or doubleClick() in the same synchronous block the clicks may fire against the previously-active stroke type. Always set ALL tool properties first, then await sleep(50) before placing clicks. sleep(0) / setTimeout(0) is NOT reliable — React may not have re-rendered the canvas by then.

Parameters

Name	Type	Description
type	string	One of: "line", "arc", "polyline", "circle", "splatter", "dry-brush", "chalk", "wet-brush", "burn", "dodge"

Example

// Correct pattern in a script:
const sleep = (ms) => new Promise(r => setTimeout(r, ms));
api.selectStrokeType("circle");
api.setColor("#ffe066");
api.setThickness(3);
await sleep(50); // wait ~3 render frames for ALL setters to flush
api.click(400, 300); // centre
api.click(440, 300); // radius

setColor

window.paintByLanguageModel.setColor(hex: string): void

Set the stroke colour. Accepts any CSS hex colour string.

Parameters

Name	Type	Description
hex	string	Hex colour string, e.g. "#ff6600" or "#3a7bd5"

Example

window.paintByLanguageModel.setColor("#ff6600");

setOpacity

window.paintByLanguageModel.setOpacity(value: number): void

Set the stroke opacity. Changes take effect for the next committed stroke.

Parameters

Name	Type	Description
value	number	Opacity between 0.0 (transparent) and 1.0 (fully opaque)

Example

window.paintByLanguageModel.setOpacity(0.5);

setThickness

window.paintByLanguageModel.setThickness(px: number): void

Set the stroke thickness in canvas pixels.

Parameters

Name	Type	Description
px	number	Stroke thickness in pixels (1–50)

Example

window.paintByLanguageModel.setThickness(8);

setTypeParam

window.paintByLanguageModel.setTypeParam(key: string, value: unknown): void

Override a type-specific parameter for the current stroke type. Use getTypeParamSchema(type) to discover available keys.

Parameters

Name	Type	Description
key	string	Parameter key, e.g. "arc_start_angle", "fill", "splatter_count", "brush_width", "bristle_count", "gap_probability", "chalk_width", "grain_density", "softness", "flow", "intensity", "dot_size_min", "dot_size_max"
value	unknown	Value to set for the parameter

Example

window.paintByLanguageModel.setTypeParam("fill", true);

Canvas Interactions

click

window.paintByLanguageModel.click(x: number, y: number): void

Simulate a canvas click at logical pixel coordinates (x, y). The number of clicks required to commit a stroke depends on the active type: • line, arc, burn, dodge — 2 clicks (start → end) • circle, splatter — 2 clicks (centre → radius point) • polyline, dry-brush, chalk, wet-brush — ≥2 clicks (add points) then doubleClick() to commit

Parameters

Name	Type	Description
x	number	X coordinate in canvas pixels
y	number	Y coordinate in canvas pixels

Example

window.paintByLanguageModel.click(100, 150);

doubleClick

window.paintByLanguageModel.doubleClick(x: number, y: number): void

Simulate a canvas double-click at (x, y). Commits multi-point strokes (polyline, dry-brush, chalk, wet-brush).

Parameters

Name	Type	Description
x	number	X coordinate in canvas pixels
y	number	Y coordinate in canvas pixels

Example

window.paintByLanguageModel.doubleClick(200, 200);

cancelStroke

window.paintByLanguageModel.cancelStroke(): void

Discard any in-progress (uncommitted) stroke without committing it. Clears the overlay preview.

Example

window.paintByLanguageModel.cancelStroke();

Canvas Management

clearCanvas

window.paintByLanguageModel.clearCanvas(): void

Clear all committed strokes from the canvas. Does not show a confirmation dialog (unlike the toolbar Clear button).

Example

window.paintByLanguageModel.clearCanvas();

getStrokes

window.paintByLanguageModel.getStrokes(): object[]

Return a deep copy of the current committed stroke array as plain JSON-serialisable objects.

Returns: Array of EnrichedStroke objects (plain JSON)

Example

const strokes = window.paintByLanguageModel.getStrokes();

loadStrokes

window.paintByLanguageModel.loadStrokes(drawingJson: string): void

Load a drawing from a JSON string (same format as the Download JSON feature). Replaces the current canvas content.

Parameters

Name	Type	Description
drawingJson	string	JSON string in DrawingData format (exported by downloadJSON)

Example

window.paintByLanguageModel.loadStrokes(JSON.stringify(drawingData));

downloadJSON

window.paintByLanguageModel.downloadJSON(): void

Programmatically trigger a browser file-download of the current drawing as a .json file.

Example

window.paintByLanguageModel.downloadJSON();

downloadJPG

window.paintByLanguageModel.downloadJPG(): void

Programmatically trigger a browser file-download of the current canvas as a .jpg file.

Example

window.paintByLanguageModel.downloadJPG();

getCanvasImageDataUrl

window.paintByLanguageModel.getCanvasImageDataUrl(): string

Return the current canvas as a base-64 data:image/png;base64,... string. Useful for passing to another LLM for evaluation.

Returns: Base-64 encoded PNG data URL string

Example

const dataUrl = window.paintByLanguageModel.getCanvasImageDataUrl();

Introspection

getState

window.paintByLanguageModel.getState(): object

Return a snapshot of the current tool state: activeType, color, opacity, thickness, typeParams, and strokeCount.

Returns: { activeType: string, color: string, opacity: number, thickness: number, typeParams: object, strokeCount: number }

Example

const state = window.paintByLanguageModel.getState();

getStrokeTypes

window.paintByLanguageModel.getStrokeTypes(): string[]

Return the list of all valid stroke type names.

Returns: Array of stroke type name strings

Example

const types = window.paintByLanguageModel.getStrokeTypes();

getTypeParamSchema

window.paintByLanguageModel.getTypeParamSchema(type: string): object

Return the parameter schema and defaults for a given stroke type. Enables discovery of what setTypeParam keys are valid.

Parameters

Name	Type	Description
type	string	A valid stroke type name (see getStrokeTypes())

Returns: Record of parameter key → default value for the given stroke type

Example

const schema = window.paintByLanguageModel.getTypeParamSchema("splatter");

Worked Example — Sunset

A complete script that paints a recognisable sunset scene. Copy-paste it into the DevTools console while the /draw page is open.

// Sunset scene — horizon line, sun circle, sky splatters
// Canvas: 800 × 600 px  |  (0, 0) = top-left
//
// Tool-setting calls (selectStrokeType, setColor, setOpacity, setThickness) all
// trigger React state updates that are asynchronous. The correct pattern is:
//   1. Call all setters for a section
//   2. await sleep(50)  ← wait ~3 render frames for React to flush all queued state
//   3. Then place clicks
//
// sleep(0) / setTimeout(0) is NOT sufficient — React schedules renders via its
// own internal scheduler and may not have re-rendered the canvas component by
// the time a setTimeout(0) resolves. Use sleep(50) for reliable results.
//
// Two-click stroke types (line, arc, circle, splatter, burn, dodge) also need
// await sleep(50) between consecutive strokes of the same type, because the
// first stroke's commit triggers a React state update that must settle before
// the next stroke's first click can register correctly.

const api = window.paintByLanguageModel;
const sleep = (ms) => new Promise(r => setTimeout(r, ms));

await (async () => {

  // ── Sky background gradient effect (wide splatters) ────────────────────────
  // splatter = 2 clicks: centre point, then radius point
  api.selectStrokeType("splatter");
  api.setColor("#ffb347");      // warm orange
  api.setOpacity(0.6);
  api.setThickness(1);
  api.setTypeParam("splatter_count", 80);
  api.setTypeParam("dot_size_min", 3);
  api.setTypeParam("dot_size_max", 10);
  await sleep(50);
  api.click(400, 100);          // centre
  api.click(440, 100);          // radius point → commits splatter

  api.setColor("#ff6b6b");      // coral/red
  api.setOpacity(0.4);
  api.setTypeParam("splatter_count", 60);
  await sleep(50);               // flush colour change + wait for previous commit to settle
  api.click(200, 180);          // centre
  api.click(240, 180);          // radius point → commits splatter

  await sleep(50);
  api.click(600, 180);          // centre
  api.click(640, 180);          // radius point → commits splatter

  // ── Sun — filled yellow circle near horizon ─────────────────────────────────
  // circle = 2 clicks: centre point, then radius point
  api.selectStrokeType("circle");
  api.setColor("#ffe066");       // bright yellow
  api.setOpacity(1.0);
  api.setThickness(3);
  api.setTypeParam("fill", true);
  await sleep(50);
  api.click(400, 310);           // centre of sun
  api.click(450, 310);           // radius point (50 px radius)

  // ── Horizon line ────────────────────────────────────────────────────────────
  // line = 2 clicks: start point, then end point
  api.selectStrokeType("line");
  api.setColor("#cc3300");       // deep red horizon
  api.setOpacity(0.9);
  api.setThickness(3);
  await sleep(50);
  api.click(0, 360);             // left edge
  api.click(800, 360);           // right edge → commits line

  // ── Water / sea — horizontal wet-brush strokes below horizon ────────────────
  // wet-brush = multi-point: ≥2 clicks to add points, then doubleClick() to commit
  api.selectStrokeType("wet-brush");
  api.setColor("#1a3a5c");       // dark ocean blue
  api.setOpacity(0.8);
  api.setThickness(12);
  api.setTypeParam("softness", 4);
  api.setTypeParam("flow", 0.7);
  await sleep(50);
  api.click(0, 400);
  api.click(200, 395);
  api.click(400, 400);
  api.click(600, 395);
  api.doubleClick(800, 400);     // commits the stroke

  api.setOpacity(0.5);
  await sleep(50);
  api.click(0, 430);
  api.click(300, 425);
  api.click(600, 430);
  api.doubleClick(800, 430);

  // ── Sun reflection on water — three short vertical lines ────────────────────
  // Each line is 2 clicks. await sleep(0) between pairs so each commit settles
  // before the next stroke's first click fires.
  api.selectStrokeType("line");
  api.setColor("#ffe066");
  api.setOpacity(0.7);
  api.setThickness(2);
  await sleep(50);
  api.click(380, 365);
  api.click(380, 480);           // commits line 1
  await sleep(50);
  api.click(400, 365);
  api.click(400, 490);           // commits line 2
  await sleep(50);
  api.click(420, 365);
  api.click(420, 480);           // commits line 3

  // ── Atmospheric haze with burn ───────────────────────────────────────────────
  // burn = 2 clicks: start point, then end point
  api.selectStrokeType("burn");
  api.setThickness(80);
  api.setTypeParam("intensity", 0.15);
  await sleep(50);
  api.click(200, 360);           // left of horizon
  api.click(600, 360);           // right of horizon → commits burn

  // Done — a simple sunset is painted on the canvas.
  // Call window.paintByLanguageModel.getCanvasImageDataUrl() to export it.

})();