Documentation Index
Fetch the complete documentation index at: https://docs.guardway.ai/llms.txt
Use this file to discover all available pages before exploring further.
What this is for
Providers are the upstream LLM services your gateway routes to (OpenAI, Anthropic, AWS Bedrock, Azure OpenAI, your own self-hosted vLLM, etc.). Each provider is scoped to a single gateway — that gateway holds its credentials and forwards requests to it. You can move a provider to a different gateway later from the provider edit dialog. Use this page to add a provider to a gateway and check that the gateway can reach it.Options
Provider presets
Guardway ships presets for the most common providers. Each preset pre-fills the base URL, request adapter, and any provider-specific quirks.Custom (OpenAI-compatible)
Add Provider dialog
| Field | Required | Notes |
|---|---|---|
| Gateway | Yes | The gateway this provider belongs to. Each provider is scoped to exactly one gateway; move it later from the provider edit dialog. |
| Preset | No | Pick one to pre-fill Type, API Base, and any provider-specific quirks. Leave empty if you’ll configure manually. |
| Name | Yes | Internal identifier shown in routing rules and logs. |
| Type | Yes | Adapter to use. Defaults to OpenAI. Set automatically when you pick a preset. |
| Display Name | No | Friendly label shown in dashboards. Falls back to Name when empty. |
| Enabled | — | Toggle. Disable to stop routing to this provider without deleting it. |
| API Base | Usually | Base URL of the upstream endpoint (e.g. https://api.example.com/v1). Pre-filled by presets; editable for Custom and for regional endpoints (Azure, Bedrock). |
| API Key | Usually | The provider’s secret. Encrypted at rest and pushed only to the attached gateway. Optional for local presets (Ollama, LM Studio). |
| Custom Model Names | No | Comma-separated or newline-separated list of model identifiers to expose. Useful when a provider doesn’t return a discoverable model list. |
Enabled Endpoints
Toggle which OpenAI-compatible endpoints this provider serves. Defaults: Chat, Completion, Embeddings on; Audio, Images, Moderation off. Disable any your account doesn’t pay for so routing rules skip it cleanly.Advanced Settings
| Field | Default | Notes |
|---|---|---|
| Requests/min | 1000 | Per-provider request rate limit enforced by the gateway. |
| Tokens/min | 100000 | Per-provider token rate limit. |
| Max Retries | 3 | How many times to retry a failed upstream call before giving up. |
| Delay (ms) | 1000 | Initial backoff between retries. |
| Request Timeout (ms) | 30000 | Hard timeout on the upstream call. |
| Custom Headers (JSON) | empty | Extra headers sent on every request, e.g. {"X-Custom-Header": "value"}. |
| Config (JSON) | empty | Provider-specific defaults merged into requests, e.g. {"temperature": 0.7, "max_tokens": 1000}. |
| Health Check URL | empty | Optional URL the gateway pings to determine provider health. Falls back to a lightweight upstream call when empty. |
Provider row state
Each row in the providers list shows:- Health —
healthy,degraded,down, orunknown. - Latency — rolling p50 to the provider.
- Models — count discovered (zero until you sync; see Models).
- Enabled — toggle to stop routing to this provider without deleting it.
How to configure
Open Configuration → Providers
Open Configuration → Providers from the dashboard sidebar and click Add Provider.
Pick the gateway and preset
Select the Gateway this provider belongs to, then pick a Preset (or leave empty for a manual setup). The preset pre-fills Type and API Base.
Fill in the basics
Enter a Name, optionally a Display Name, paste the API Key, and override the API Base if your environment needs a regional or self-hosted URL. Add Custom Model Names when the provider can’t auto-discover its model list.
Pick the endpoints to enable
Under Enabled Endpoints, leave on only the modalities your account supports — Chat, Completion, Embeddings, Audio, Images, Moderation.
(Optional) Tune Advanced Settings
Expand Advanced Settings to set per-provider Rate Limits, Retry Configuration, Request Timeout, Custom Headers (JSON), Config (JSON) defaults, and a Health Check URL.
Save and verify health
Click Save Provider. The row appears in the list with health Unknown until the gateway runs its first health check; it then flips to healthy (or surfaces an error). If it stays unknown for more than a minute, check the gateway can reach the provider’s API Base from its network.