> ## Documentation Index
> Fetch the complete documentation index at: https://docs.guardway.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Providers

> Add an LLM provider to Guardway, pick a preset, paste an API key, and attach it to a gateway.

## What this is for

Providers are the upstream LLM services your gateway routes to (OpenAI, Anthropic, AWS Bedrock, Azure OpenAI, your own self-hosted vLLM, etc.). Each provider is **scoped to a single gateway** — that gateway holds its credentials and forwards requests to it. You can move a provider to a different gateway later from the provider edit dialog.

Use this page to add a provider to a gateway and check that the gateway can reach it.

## Options

### Provider presets

Guardway ships presets for the most common providers. Each preset pre-fills the base URL, request adapter, and any provider-specific quirks.

export const Provider = ({slug, name, ext = 'svg'}) => <div className="flex flex-col items-center justify-center gap-3 p-4 rounded-xl border border-gray-200 dark:border-gray-800">
    <div className="h-12 w-32 flex items-center justify-center">
      <img src={`/images/providers/${slug}-light.${ext}`} alt={name} className="block dark:hidden max-h-full max-w-full object-contain" />
      <img src={`/images/providers/${slug}-dark.${ext}`} alt={name} className="hidden dark:block max-h-full max-w-full object-contain" />
    </div>
    <div className="text-sm font-medium">{name}</div>
  </div>;

<div className="grid grid-cols-2 sm:grid-cols-3 md:grid-cols-4 gap-3 my-6 not-prose">
  <Provider slug="openai" name="OpenAI" />

  <Provider slug="anthropic" name="Anthropic" />

  <Provider slug="google-gemini" name="Google Gemini" />

  <Provider slug="groq" name="Groq" ext="png" />

  <Provider slug="mistral" name="Mistral" />

  <Provider slug="deepseek" name="DeepSeek" />

  <Provider slug="xai" name="xAI (Grok)" />

  <Provider slug="perplexity" name="Perplexity" />

  <Provider slug="together" name="Together AI" />

  <Provider slug="fireworks" name="Fireworks AI" />

  <Provider slug="openrouter" name="OpenRouter" />

  <Provider slug="cohere" name="Cohere" />

  <Provider slug="aws-bedrock" name="AWS Bedrock" />

  <Provider slug="azure-openai" name="Azure OpenAI" />

  <Provider slug="huggingface" name="HuggingFace TGI" />

  <Provider slug="nvidia-nim" name="NVIDIA NIM" />

  <Provider slug="ollama" name="Ollama" />

  <Provider slug="lm-studio" name="LM Studio" ext="png" />

  <Provider slug="vllm" name="vLLM" />

  <div className="flex flex-col items-center justify-center gap-3 p-4 rounded-xl border border-gray-200 dark:border-gray-800">
    <div className="text-2xl">
      {`</>`}
    </div>

    <div className="text-sm font-medium">Custom (OpenAI-compatible)</div>
  </div>
</div>

<Tip>
  Anything that speaks an OpenAI-compatible API works with the **Custom** preset — point it at any base URL (a self-hosted vLLM, a partner endpoint, etc.).
</Tip>

### Add Provider dialog

| Field                  | Required | Notes                                                                                                                                                          |
| ---------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Gateway**            | Yes      | The gateway this provider belongs to. Each provider is scoped to exactly one gateway; move it later from the provider edit dialog.                             |
| **Preset**             | No       | Pick one to pre-fill **Type**, **API Base**, and any provider-specific quirks. Leave empty if you'll configure manually.                                       |
| **Name**               | Yes      | Internal identifier shown in routing rules and logs.                                                                                                           |
| **Type**               | Yes      | Adapter to use. Defaults to **OpenAI**. Set automatically when you pick a preset.                                                                              |
| **Display Name**       | No       | Friendly label shown in dashboards. Falls back to **Name** when empty.                                                                                         |
| **Enabled**            | —        | Toggle. Disable to stop routing to this provider without deleting it.                                                                                          |
| **API Base**           | Usually  | Base URL of the upstream endpoint (e.g. `https://api.example.com/v1`). Pre-filled by presets; editable for Custom and for regional endpoints (Azure, Bedrock). |
| **API Key**            | Usually  | The provider's secret. Encrypted at rest and pushed only to the attached gateway. Optional for local presets (Ollama, LM Studio).                              |
| **Custom Model Names** | No       | Comma-separated or newline-separated list of model identifiers to expose. Useful when a provider doesn't return a discoverable model list.                     |

#### Enabled Endpoints

Toggle which OpenAI-compatible endpoints this provider serves. Defaults: **Chat**, **Completion**, **Embeddings** on; **Audio**, **Images**, **Moderation** off. Disable any your account doesn't pay for so routing rules skip it cleanly.

#### Advanced Settings

| Field                     | Default  | Notes                                                                                                              |
| ------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------ |
| **Requests/min**          | `1000`   | Per-provider request rate limit enforced by the gateway.                                                           |
| **Tokens/min**            | `100000` | Per-provider token rate limit.                                                                                     |
| **Max Retries**           | `3`      | How many times to retry a failed upstream call before giving up.                                                   |
| **Delay (ms)**            | `1000`   | Initial backoff between retries.                                                                                   |
| **Request Timeout (ms)**  | `30000`  | Hard timeout on the upstream call.                                                                                 |
| **Custom Headers (JSON)** | empty    | Extra headers sent on every request, e.g. `{"X-Custom-Header": "value"}`.                                          |
| **Config (JSON)**         | empty    | Provider-specific defaults merged into requests, e.g. `{"temperature": 0.7, "max_tokens": 1000}`.                  |
| **Health Check URL**      | empty    | Optional URL the gateway pings to determine provider health. Falls back to a lightweight upstream call when empty. |

### Provider row state

Each row in the providers list shows:

* **Health** — `healthy`, `degraded`, `down`, or `unknown`.
* **Latency** — rolling p50 to the provider.
* **Models** — count discovered (zero until you sync; see [Models](/platform/configuration/models)).
* **Enabled** — toggle to stop routing to this provider without deleting it.

## How to configure

<Frame caption="Providers list">
  <img src="https://mintcdn.com/fcguardwayai/rJTQ_bXDRs9Cgazf/images/screenshots/platform/configuration/providers-list.png?fit=max&auto=format&n=rJTQ_bXDRs9Cgazf&q=85&s=3281504d376d0626d917f0fb3f344006" alt="Providers list" width="2998" height="1620" data-path="images/screenshots/platform/configuration/providers-list.png" />
</Frame>

<Steps>
  <Step title="Open Configuration → Providers">
    Open **Configuration → Providers** from the dashboard sidebar and click **Add Provider**.
  </Step>

  <Step title="Pick the gateway and preset">
    Select the **Gateway** this provider belongs to, then pick a **Preset** (or leave empty for a manual setup). The preset pre-fills **Type** and **API Base**.

    <Frame caption="Add Provider — top fields">
      <img src="https://mintcdn.com/fcguardwayai/rJTQ_bXDRs9Cgazf/images/screenshots/platform/configuration/provider-add.png?fit=max&auto=format&n=rJTQ_bXDRs9Cgazf&q=85&s=725bd3bd7fd6a616c53cb61e9d96c491" alt="Add Provider — top fields" width="1178" height="1662" data-path="images/screenshots/platform/configuration/provider-add.png" />
    </Frame>
  </Step>

  <Step title="Fill in the basics">
    Enter a **Name**, optionally a **Display Name**, paste the **API Key**, and override the **API Base** if your environment needs a regional or self-hosted URL. Add **Custom Model Names** when the provider can't auto-discover its model list.
  </Step>

  <Step title="Pick the endpoints to enable">
    Under **Enabled Endpoints**, leave on only the modalities your account supports — **Chat**, **Completion**, **Embeddings**, **Audio**, **Images**, **Moderation**.
  </Step>

  <Step title="(Optional) Tune Advanced Settings">
    Expand **Advanced Settings** to set per-provider **Rate Limits**, **Retry Configuration**, **Request Timeout**, **Custom Headers (JSON)**, **Config (JSON)** defaults, and a **Health Check URL**.
  </Step>

  <Step title="Save and verify health">
    Click **Save Provider**. The row appears in the list with health **Unknown** until the gateway runs its first health check; it then flips to **healthy** (or surfaces an error). If it stays **unknown** for more than a minute, check the gateway can reach the provider's API Base from its network.
  </Step>
</Steps>

## Related

* [Models](/platform/configuration/models) — discover and enable specific models from a provider.
* [API Keys](/platform/configuration/api-keys) — restrict which providers a key can call.
* [Routing](/platform/configuration/routing) — fan out across providers for fallback or cost.