Table of Contents

Gateway routing, CORS, and unavailable tenants settings

The gateway is the HTTP entry point of a Neos deployment. It authenticates users, resolves tenant context for multitenant clusters, applies CORS policies to cross-origin requests, and routes requests to the correct service.

This article covers gateway deployment settings for routing, tenant availability synchronization and CORS configuration for browser and mobile clients.

Request flow overview

For a multitenant deployment, a typical flow is:

  1. The browser calls the gateway.
  2. The gateway applies authentication and tenant resolution.
  3. The gateway routes traffic to frontend, backend, or supporting services.
  4. For tenant-sensitive routes, the gateway checks its unavailable-tenants cache.

When the gateway starts in multitenant mode, it initializes this cache from Tenant Management and then keeps it synchronized with:

  • Redis pub/sub updates (push model)
  • periodic refresh calls to Tenant Management only when pub/sub synchronization is unavailable (fallback pull model)

This combination provides low-latency updates in normal mode and keeps convergence guarantees during transient pub/sub failures.

Sticky sessions for notifications

SignalR notifications require end-to-end affinity across the full request path. In practice, this means sticky sessions are needed at two levels.

The first level is the ingress. The incoming client request must keep reaching the same gateway pod, typically through an ingress affinity cookie. Without this, two requests from the same browser session can land on two different gateway pods.

The second level is the backend service used by the gateway for notifications traffic. Once a request has reached a given gateway pod, that gateway pod must keep reaching the same backend pod. This is provided by a dedicated Kubernetes sticky Service.

Both levels are necessary. If only the backend Service is sticky, affinity is guaranteed only from one gateway pod to one backend pod. It does not guarantee that the same end user session will keep using the same gateway pod. For SignalR negotiation and subsequent notifications traffic, that partial affinity is not enough.

The stable routing chain must therefore be:

  1. client session -> same ingress target
  2. ingress -> same gateway pod
  3. gateway pod -> same backend pod through the sticky Kubernetes Service

With the Neos Helm chart, this double affinity is already configured automatically for notifications routes:

  • ingress affinity is enabled on notifications ingresses with a dedicated session cookie
  • backend affinity is enabled through dedicated -sticky Kubernetes Service resources used by gateway notification routes

Detailed multitenant flow

For tenant-aware routes, the gateway checks whether the requested tenant is currently available before routing to the backend. If Tenant Management has signaled that the tenant is unavailable because its data persistence is not running or because the tenant itself is inactive, the gateway returns a 503 immediately without forwarding the request.

The authorized-tenants list returned by Tenant Management can still contain tenants with status Inactive. This is intentional: authorization resolution and availability blocking are separated concerns. The gateway authorizes first, then enforces fail-closed availability with the unavailable-tenants cache, which is what produces the 503 for inactive tenants.

This protection relies on a local in-memory cache. In normal mode, the cache is synchronized via Redis pub/sub and Tenant Management pushes fresh snapshots whenever tenant availability changes. If the subscription cycle is not available, the gateway automatically falls back to periodic pull refresh from Tenant Management until pub/sub becomes operational again.

Observability

Gateway unavailable-tenants metrics are documented in Monitoring.

See in particular the gateway metric neos_unavailable_tenant and its labels for blocked tenant diagnosis.

X-Forwarded-Prefix header

The gateway sets the X-Forwarded-Prefix header when a prefix value is available from one of the following sources, evaluated in priority order:

  1. An upstream X-Forwarded-Prefix header already present on the incoming request (for example set by Cloudflare or an ingress controller) — forwarded as-is.
  2. The ASP.NET Core PathBase of the current request, when set by middleware.
  3. The deployment configuration: for clusters accessed via a URL prefix (Prefix configuration key) or as a nested cluster, the value is resolved to the full prefix path with a leading slash — for example /documentation or /app/tracking.

When none of these sources provides a value (for example a cluster served at the root with no prefix and no upstream header), the header is not set on the forwarded request. Do not assume X-Forwarded-Prefix is always present.

This header allows cluster backends and frontends to correctly reconstruct absolute URLs (Swagger server URL, Stimulsoft report routes, etc.) without being aware of the deployment topology.

Where to set configuration

Gateway .NET settings are provided through configuration providers (for example, JSON files and environment variables). In Kubernetes with Helm, the usual approach is to define settings in a secret mounted into the gateway pod (gateway.envSecret).

See also:

Unavailable-tenants timing settings

All delays and request timeout used by unavailable-tenants synchronization can be configured through IConfiguration with the keys below.

Configuration key Default value Description
UnavailableTenants:RequestTimeoutSeconds 60 Timeout (seconds) for calls to Tenant Management (methods/getunavailabletenants).
UnavailableTenants:Subscriber:RetryDelaySeconds 5 Delay (seconds) before retrying Redis subscriber cycle after a failure.
UnavailableTenants:PeriodicRefresh:BaseDelaySeconds 15 Base delay (seconds) used by fallback pull refresh cycles.
UnavailableTenants:PeriodicRefresh:MaxDelaySeconds 60 Maximum delay (seconds) reached by backoff when fallback refresh fails repeatedly.
UnavailableTenants:PeriodicRefresh:BackoffMultiplier 2 Multiplication factor applied to fallback delay after refresh failures.
UnavailableTenants:PeriodicRefresh:JitterPercent 20 Randomization factor (percent) applied to fallback delay to avoid synchronized retries across pods.

Allowed values and behavior

  • RequestTimeoutSeconds, Subscriber:RetryDelaySeconds, BaseDelaySeconds, and MaxDelaySeconds must be strictly positive.
  • MaxDelaySeconds lower than BaseDelaySeconds is automatically aligned to BaseDelaySeconds.
  • BackoffMultiplier should be greater than or equal to 1.
  • JitterPercent is clamped between 0 and 90.

If a value is missing or invalid, the gateway falls back to the default listed above.

Environment variable mapping

When using environment variables, use standard .NET key mapping (: becomes __).

UnavailableTenants__RequestTimeoutSeconds=60
UnavailableTenants__Subscriber__RetryDelaySeconds=5
UnavailableTenants__PeriodicRefresh__BaseDelaySeconds=15
UnavailableTenants__PeriodicRefresh__MaxDelaySeconds=60
UnavailableTenants__PeriodicRefresh__BackoffMultiplier=2
UnavailableTenants__PeriodicRefresh__JitterPercent=20

Example secret for gateway

Example Kubernetes secret containing these settings:

kubectl create secret generic neos-gateway-settings `
  --from-literal=UnavailableTenants__RequestTimeoutSeconds=60 `
  --from-literal=UnavailableTenants__Subscriber__RetryDelaySeconds=5 `
  --from-literal=UnavailableTenants__PeriodicRefresh__BaseDelaySeconds=15 `
  --from-literal=UnavailableTenants__PeriodicRefresh__MaxDelaySeconds=60 `
  --from-literal=UnavailableTenants__PeriodicRefresh__BackoffMultiplier=2 `
  --from-literal=UnavailableTenants__PeriodicRefresh__JitterPercent=20

Then reference this secret in Helm values through gateway.envSecret.

Gateway CORS configuration

Gateway CORS is configurable through IConfiguration under Gateway:Cors.

The policy is global for the gateway and is applied by middleware.

By default, CORS is disabled. In production, use explicit allow lists for origins.

Configuration key Default value Description
Gateway:Cors:Enabled false Enables CORS middleware in the gateway.
Gateway:Cors:DevelopmentAllowAll false Development-only override. When true in development, allows any origin, method, and header (credentials remain disabled).
Gateway:Cors:AllowAnyOrigin false Allows all origins. Not recommended for production.
Gateway:Cors:AllowAnyMethod false Allows all HTTP methods.
Gateway:Cors:AllowAnyHeader false Allows all request headers.
Gateway:Cors:AllowCredentials false Enables credentialed cross-origin requests. Keep disabled unless strictly required.
Gateway:Cors:AllowedOrigins [] List of allowed origins (scheme + host + optional port).
Gateway:Cors:AllowedMethods GET, POST, PUT, PATCH, DELETE, OPTIONS Allowed HTTP methods when AllowAnyMethod=false.
Gateway:Cors:AllowedHeaders Authorization, Content-Type, Accept, Origin Allowed request headers when AllowAnyHeader=false.
Gateway:Cors:PreflightMaxAgeSeconds 600 Preflight cache duration in seconds (Access-Control-Max-Age).

Allowed values and behavior

  • Trailing slashes in AllowedOrigins are normalized (for example https://api.example.com/ becomes https://api.example.com).
  • Empty list values are ignored.
  • PreflightMaxAgeSeconds must be strictly positive, otherwise it is ignored.
  • AllowAnyOrigin=true combined with AllowCredentials=true is rejected by design and credentials are disabled.
  • If CORS is enabled with no allowed origins and AllowAnyOrigin=false, cross-origin requests are denied.

Development mode

The Gateway:Cors:DevelopmentAllowAll setting is intended for local development and integration testing.

When Gateway:Cors:DevelopmentAllowAll=true and the gateway runs in Development environment:

  • CORS is forced on, even if Gateway:Cors:Enabled=false.
  • Any origin, method, and header are allowed.
  • Credentials remain disabled for safety.

In non-development environments, this flag is ignored.

Warning

Do not rely on DevelopmentAllowAll for staging or production. Use explicit allow lists (AllowedOrigins, AllowedMethods, and AllowedHeaders) instead.

Development-only example:

Gateway__Cors__DevelopmentAllowAll=true
ASPNETCORE_ENVIRONMENT=Development

Environment variable mapping

When using environment variables, use standard .NET key mapping (: becomes __).

Gateway__Cors__Enabled=true
Gateway__Cors__DevelopmentAllowAll=false
Gateway__Cors__AllowAnyOrigin=false
Gateway__Cors__AllowAnyMethod=false
Gateway__Cors__AllowAnyHeader=false
Gateway__Cors__AllowCredentials=false
Gateway__Cors__AllowedOrigins__0=https://mobile.example.com
Gateway__Cors__AllowedMethods__0=GET
Gateway__Cors__AllowedMethods__1=POST
Gateway__Cors__AllowedMethods__2=OPTIONS
Gateway__Cors__AllowedHeaders__0=Authorization
Gateway__Cors__AllowedHeaders__1=Content-Type
Gateway__Cors__AllowedHeaders__2=Accept
Gateway__Cors__AllowedHeaders__3=Origin
Gateway__Cors__PreflightMaxAgeSeconds=600

Example secret for CORS

kubectl create secret generic neos-gateway-settings `
  --from-literal=Gateway__Cors__Enabled=true `
  --from-literal=Gateway__Cors__AllowAnyOrigin=false `
  --from-literal=Gateway__Cors__AllowAnyMethod=false `
  --from-literal=Gateway__Cors__AllowAnyHeader=false `
  --from-literal=Gateway__Cors__AllowCredentials=false `
  --from-literal=Gateway__Cors__AllowedOrigins__0=https://mobile.example.com `
  --from-literal=Gateway__Cors__AllowedMethods__0=GET `
  --from-literal=Gateway__Cors__AllowedMethods__1=POST `
  --from-literal=Gateway__Cors__AllowedMethods__2=OPTIONS `
  --from-literal=Gateway__Cors__AllowedHeaders__0=Authorization `
  --from-literal=Gateway__Cors__AllowedHeaders__1=Content-Type `
  --from-literal=Gateway__Cors__AllowedHeaders__2=Accept `
  --from-literal=Gateway__Cors__AllowedHeaders__3=Origin `
  --from-literal=Gateway__Cors__PreflightMaxAgeSeconds=600

Reference this secret in Helm values through gateway.envSecret.

Operational recommendation

Start with default values, then adapt gradually:

  • increase RequestTimeoutSeconds if Tenant Management latency is high
  • increase BaseDelaySeconds to reduce background traffic
  • increase MaxDelaySeconds to smooth retry load during prolonged outages
  • keep JitterPercent greater than 0 in multi-pod deployments

Troubleshooting

Unexpected 503 for a tenant request

  1. Confirm request is routed through multitenant gateway path.
  2. Check current metric samples for neos_unavailable_tenant matching tenant/cluster and inspect data_persistence_state and tenant_status labels.
  3. Verify latest snapshot for the cluster in Tenant Management (GET methods/getunavailabletenants).
  4. Check gateway logs for warmup or subscriber failures.
  5. If needed, force a fresh publication from Tenant Management (manual republish method below).

If you want to customize the rendered 503 page returned by the gateway, see Custom status code pages in production.

Verify Redis publication path

  1. Confirm Tenant Management can connect to Redis.
  2. Confirm gateway subscriber is connected to channel neos:unavailable-tenants:snapshot:v1.
  3. Validate message payload uses UnavailableTenants contract (not legacy TenantIdentifiers) and includes DataPersistenceState and TenantStatus for each item, including tenants marked Inactive even when their persistence is still Running.
  4. If pub/sub is unstable, the gateway automatically switches to fallback periodic pull refresh while transport is being fixed.

Recovery actions from Tenant Management

Recommended order during incidents:

  1. Trigger POST /api/v1/methods/resetalltenantcache to rebuild tenant resolution stores.
  2. If immediate snapshot synchronization is still required, call POST /api/v1/methods/republishunavailabletenantssnapshots.
  3. Re-check gateway metric and retry impacted request.

Notes:

  • resetalltenantcache already triggers automatic unavailable-tenants republication after rebuild in Tenant Management backend logic.
  • the Tenant Management UI action for ResetAllCache currently also performs an explicit call to RepublishUnavailableTenantsSnapshots after reset completion.
  • manual republishunavailabletenantssnapshots is an explicit admin action for immediate re-sync.