Testing an endpoint

Why test an endpoint

Use Endpoint test console to check that your registered endpoint responds, measure its behavior under load, and test prompt sizes before offering capacity. Tests make real requests and may consume your supplier quota.

Open the test console

Sign in to the provider workspace and open Endpoint test console. Under Endpoint under test, select one of your registered endpoints. Check its model, adapter, base address, availability and declared context before starting a test.

If no endpoint is registered, follow Provider onboarding in Supply & fulfillment, then return to the console. The tests available depend on the selected endpoint's model type. A suspended endpoint cannot be tested.

Run a chat test

In Chat test, enter a Test prompt and Max output tokens, then select Send test. The response appears as it arrives. Choose Cancel to stop the test.

The results distinguish received text chunks from reported token usage. A chunk can contain several tokens; a chunk count is not a token count. Missing usage is shown as unavailable.

• End-to-end rate includes the initial waiting time and the response time.
• First content shows how long the first response text took to arrive.
• Decode-window rate measures reported output tokens over the response period after the first content.

Embedding endpoints have a separate test that shows the returned dimensions, a small vector preview and reported usage.

Run a stream load test

In Stream load test, enter your prompt and output limit, choose Parallel streams, then start the run. The available counts are 1, 5, 8, 10, 25 and 50. Choose a count your supplier allows.

Review active, completed, failed and cancelled streams separately. Live / peak chunk rate measures chunks per second. End-to-end throughput and Total completion tokens use reported token counts; missing reports remain unavailable.

Completed streams retain their measured results. Stop all cancels the run. Compare tests using the same endpoint, model, prompt, output limit and parallel stream count.

Probe the context window

In Context window, choose a Probe cap (tokens) and select Probe context window. The test tries progressively larger prompts and may consume quota for each step.

Review each Accepted or Rejected result and the highlighted Max accepted prompt size. The console compares the result with the endpoint's declared context. A bounded test below the declared size does not establish the model's maximum.

Review the result

A successful test shows observed behavior at that time. It does not establish a continuing capacity commitment. Publish only capacity you can sustain; see Publishing capacity.

For a failed or interrupted test, review the displayed result before trying again. Contact the team with the endpoint reference, test type, approximate time and error if you need help. Keep passwords, API keys and sensitive prompts out of support messages.