What this is for
Providers are the upstream LLM services your gateway routes to (OpenAI, Anthropic, AWS Bedrock, Azure OpenAI, your own self-hosted vLLM, etc.). Each provider is scoped to a single gateway — that gateway holds its credentials and forwards requests to it. You can move a provider to a different gateway later from the provider edit dialog. Use this page to add a provider to a gateway and check that the gateway can reach it.Options
Provider presets
Guardway ships presets for the most common providers. Each preset pre-fills the base URL, request adapter, and any provider-specific quirks.Custom (OpenAI-compatible)
Add Provider dialog
| Field | Required | Notes |
|---|---|---|
| Gateway | Yes | The gateway this provider belongs to. Each provider is scoped to exactly one gateway; move it later from the provider edit dialog. |
| Preset | No | Pick one to pre-fill Type, API Base, and any provider-specific quirks. Leave empty if you’ll configure manually. |
| Name | Yes | Internal identifier shown in routing rules and logs. |
| Type | Yes | Adapter to use. Defaults to OpenAI. Set automatically when you pick a preset. |
| Display Name | No | Friendly label shown in dashboards. Falls back to Name when empty. |
| Enabled | — | Toggle. Disable to stop routing to this provider without deleting it. |
| API Base | Usually | Base URL of the upstream endpoint (e.g. https://api.example.com/v1). Pre-filled by presets; editable for Custom and for regional endpoints (Azure, Bedrock). |
| API Key | Usually | The provider’s secret. Encrypted at rest and pushed only to the attached gateway. Optional for local presets (Ollama, LM Studio). |
| Custom Model Names | No | Comma-separated or newline-separated list of model identifiers to expose. Useful when a provider doesn’t return a discoverable model list. |
Enabled Endpoints
Toggle which OpenAI-compatible endpoints this provider serves. Defaults: Chat, Completion, Embeddings on; Audio, Images, Moderation off. Disable any your account doesn’t pay for so routing rules skip it cleanly.Advanced Settings
| Field | Default | Notes |
|---|---|---|
| Requests/min | 1000 | Per-provider request rate limit enforced by the gateway. |
| Tokens/min | 100000 | Per-provider token rate limit. |
| Max Retries | 3 | How many times to retry a failed upstream call before giving up. |
| Delay (ms) | 1000 | Initial backoff between retries. |
| Request Timeout (ms) | 30000 | Hard timeout on the upstream call. |
| Custom Headers (JSON) | empty | Extra headers sent on every request, e.g. {"X-Custom-Header": "value"}. |
| Config (JSON) | empty | Provider-specific defaults merged into requests, e.g. {"temperature": 0.7, "max_tokens": 1000}. |
| Health Check URL | empty | Optional URL the gateway pings to determine provider health. Falls back to a lightweight upstream call when empty. |
Provider row state
Each row in the providers list shows:- Health —
healthy,degraded,down, orunknown. - Latency — rolling p50 to the provider.
- Models — count discovered (zero until you sync; see Models).
- Enabled — toggle to stop routing to this provider without deleting it.
How to configure
Open Configuration → Providers
Open Configuration → Providers from the dashboard sidebar and click Add Provider.
Pick the gateway and preset
Select the Gateway this provider belongs to, then pick a Preset (or leave empty for a manual setup). The preset pre-fills Type and API Base.
Fill in the basics
Enter a Name, optionally a Display Name, paste the API Key, and override the API Base if your environment needs a regional or self-hosted URL. Add Custom Model Names when the provider can’t auto-discover its model list.
Pick the endpoints to enable
Under Enabled Endpoints, leave on only the modalities your account supports — Chat, Completion, Embeddings, Audio, Images, Moderation.
(Optional) Tune Advanced Settings
Expand Advanced Settings to set per-provider Rate Limits, Retry Configuration, Request Timeout, Custom Headers (JSON), Config (JSON) defaults, and a Health Check URL.
Save and verify health
Click Save Provider. The row appears in the list with health Unknown until the gateway runs its first health check; it then flips to healthy (or surfaces an error). If it stays unknown for more than a minute, check the gateway can reach the provider’s API Base from its network.