Ai Keeper/Guides/Lab and evaluation
← All workflows
Collection 10 · Test before trust

Lab and evaluation

Send raw requests, exercise stateful Responses, compare models, benchmark performance, inspect web work, build Canvas artifacts, and keep regression checks in QA Lab.

7 workflowsBeginner-first

Find one workflow

Filter this collection by name, outcome, provider, engine, channel, mode, or command.

7 shown
Workflow 01

Send a raw request in Playground

Test an endpoint and payload without writing a separate script.

Intermediate5–10 minResult · Request and response are inspectable✓ Source-audited workflow
Current Lab Playground with request method, endpoint, headers, JSON body, and response panel.
Start with a harmless non-streaming request and inspect the response before testing stateful or tool payloads.
Before you start
  • A Ready route.
  • Know which capability you are testing.

  1. Open Lab > Playground and choose the target instance/model and endpoint type.

  2. Start with the generated example payload. Change only the prompt and one parameter.

  3. Send and inspect status, headers, body, timing, and streaming events if enabled.

  4. For embeddings or reranking, choose the matching specialist instance and verify vector/ranking output rather than chat text.

  5. Copy a working request only after it succeeds here.

You are done when
The response has the expected status and schema for the selected endpoint, and System > Requests shows the same call.
If something looks wrong
400 responseRead the returned validation message and compare payload shape to the generated example.
Testing raw instance URLUse the proxy target when you want to validate real app routing and transforms.
Workflow 02

Test the stateful Responses API

Exercise response state, compacted history, and WebSocket transport against a local runtime.

Advanced10–20 minResult · A multi-turn response continues by state ID✓ Source-audited workflow
Lab navigation showing Responses beside Playground, Compare, Benchmark, Browser, Canvas, and QA Lab.
Use Responses for stateful calls and Playground for a raw one-request baseline.
Before you start
  • A Ready compatible route.
  • Basic chat completion already works.

  1. Open Lab > Responses and select the model.

  2. Send a first request and record the response/state identifier shown by the UI.

  3. Send a follow-up linked to the previous response instead of resending the whole history.

  4. Test compacted-history payload handling with the built-in example, then switch transport to WebSocket when available.

  5. Inspect events and ensure the response remains tied to the same process generation.

You are done when
The follow-up remembers the first turn, compacted input is accepted, and WebSocket events complete without crossing to a stale process.
If something looks wrong
Instance restarted mid-sessionStart a new stateful response; generation-pinned state should not be silently reused across processes.
Workflow 03

Compare two models side by side

Run one prompt with matched settings and judge quality, format, and behavior fairly.

Beginner5–20 minResult · A documented model choice✓ Source-audited workflow
Current Lab hub; Compare sits alongside Playground in the tab bar.
All seven Lab tools share this tab bar — Playground, Responses, Compare, Benchmark, Browser, Canvas, QA Lab.
Before you start
  • Two Ready chat models/routes.
  • One representative prompt and success criteria.

  1. Open Lab > Compare and choose model A and model B.

  2. Use the same system context, prompt, output limit, temperature/profile, and tool availability for both.

  3. Run the prompt and wait for both sides to finish.

  4. Score correctness, instruction following, useful detail, format, latency, and any tool behavior. Do not choose only by writing style.

  5. Repeat with at least one different representative prompt before changing a production route.

You are done when
You can explain the choice using observed criteria and more than one prompt, not a single subjective impression.
If something looks wrong
One model had different tools/contextThe comparison is invalid. Match inputs and policy before interpreting output quality.
Workflow 04

Benchmark TTFT, throughput, and latency

Measure repeatable performance after warm-up and watch memory pressure while the run executes.

Intermediate10–20 minResult · A repeatable performance baseline✓ Source-audited workflow
Current Lab hub tab bar including Benchmark.
Benchmark is where TTFT and throughput are measured; do not infer speed from chat feel.
Before you start
  • A stable Ready instance.
  • No unrelated heavy workloads if you want comparable numbers.

  1. Open Lab > Benchmark, choose the instance, prompt size, output size, and repeat count.

  2. Run one warm-up that you do not treat as the final number.

  3. Run the repeat set while watching Monitor for memory/CPU pressure.

  4. Record time to first token, tokens per second, and end-to-end latency together.

  5. After a tuning change, rerun the exact same benchmark and environment.

You are done when
Repeated warm runs are in a reasonably consistent range and comparisons use the same prompt/output settings.
If something looks wrong
Cold and warm results mixedLabel them separately; model load and first compilation/cache work distort the first run.
Only tokens/sec improvedCheck TTFT, memory, and output quality too.
Workflow 05

Create and inspect a managed browser session

Open a page, review the visible state, perform one action, and audit what the model did.

Intermediate5–15 minResult · A browser action is visible and logged✓ Source-audited workflow
Lab Browser showing a loaded isolated research board, connected state, profile, URL bar, and New Tab.
The local demonstration page proves navigation completed while keeping the session disposable and free of personal browsing data.
Before you start
  • A web task that is allowed by the site and your policy.
  • No need to expose credentials to the model unless explicitly approved.

  1. Open Lab > Browser and create/select a browser profile and session.

  2. Navigate to a harmless page and inspect the preview/DOM state before acting.

  3. Run one simple action such as following a link or filling a non-sensitive test field.

  4. Review the LLM action log and screenshot/preview to confirm what changed.

  5. Stop or reset the session when finished.

You are done when
The visible page reflects the intended single action and the action log attributes it correctly.
If something looks wrong
Model wants to submit/purchase/sendRequire explicit confirmation and inspect the final target and payload.
Page changed unexpectedlyStop and capture the current state before retrying; do not stack blind clicks.
Workflow 06

Create and snapshot a Canvas artifact

Render live HTML, charts, or interactive output, inspect source, and preserve a known-good snapshot.

Intermediate10–30 minResult · A live artifact and snapshot exist✓ Source-audited workflow
Lab Canvas showing a generated release-readiness dashboard with summary cards and a request-volume chart.
Use Source or Snapshot after a visual artifact renders correctly; Reset is available when you want to start over.
Before you start
  • HTML/chart/interactive source from Chat, an agent, or your own edit.

  1. Open Lab > Canvas and create or load the artifact source.

  2. Render/preview it and interact with the important controls.

  3. Open the source view and correct any visible error rather than repeatedly asking the model to regenerate everything.

  4. Create a snapshot when the artifact is working. Make one change, then use history/reset to prove recovery.

  5. Export or hand the artifact to the next workflow only after the snapshot test passes.

You are done when
The artifact renders, key interactions work, source is inspectable, and the known-good snapshot can be restored.
If something looks wrong
Arbitrary script from an untrusted sourceReview code and use an isolated test artifact. A live Canvas is executable content, not a static screenshot.
Workflow 07

Create a QA regression check

Save a focused probe so a model, prompt, route, or workflow change can be tested again later.

Advanced15–30 minResult · A test suite detects a deliberate failure✓ Source-audited workflow
Lab navigation showing QA Lab as the quality-check destination.
Establish a passing Playground request first, then preserve it as a repeatable QA check.
Before you start
  • A behavior with a clear expected result.
  • A stable test input that contains no sensitive production data.

  1. Open Lab > QA Lab and create a suite named for the behavior, not the implementation.

  2. Add one small probe with input, target, expected condition, and useful failure message.

  3. Run it against the known-good setup and save the result.

  4. Temporarily change the test expectation or target so the probe fails, confirming it is capable of detecting a problem.

  5. Restore the correct setup and rerun before relying on the suite.

You are done when
The probe passes in the intended setup, fails in the controlled bad setup, and passes again after restoration.
If something looks wrong
Test only checks HTTP 200Assert the behavior or content you care about; a successful status can still contain the wrong answer or missing tool call.

Keep going

← Images, video, voice, screen, and cameraPrevious collectionRemote access and distributed use →Next collection