Skip to main content

Kubernetes Anycast with BGP Controller

This guide covers deploying globally distributed anycast services on NetActuate Managed Kubernetes using the open-source NetActuate BGP Controller. The controller automates BGP session provisioning, prefix announcement, and health-based failover across your clusters.

What This Guide Covers​

The NetActuate BGP Controller is a Kubernetes DaemonSet that:

  • Provisions BGP sessions automatically via the NetActuate API
  • Configures your BGP backend (MetalLB, Calico, Cilium, or BIRD) to peer with NetActuate routers
  • Announces your anycast prefixes from each cluster location
  • Withdraws prefixes when health checks fail, shifting traffic to the next nearest POP
  • Re-announces prefixes when services recover

The result is a globally distributed anycast architecture where traffic routes to the nearest healthy cluster automatically, with no manual BGP session management.

Prerequisites​

  • A NetActuate account with:
    • A BGP group and group ID (visible in the portal under Networking > Anycast)
    • An allocated ASN
    • At least one anycast prefix (e.g., a /24)
    • VM IP-based API authentication enabled (contact your account manager to enable this)
  • Kubernetes clusters deployed on NetActuate VMs at one or more locations
  • A supported BGP backend installed on each cluster:
    • MetalLB v0.13+ in CRD mode (recommended for most deployments)
    • Calico v3.x with BGP dataplane enabled
    • Cilium v1.13+ with --enable-bgp-control-plane=true

Note: The controller also supports a BIRD backend for bare-metal and VM deployments outside Kubernetes. See the BIRD backend section below.

Step 1: Install a BGP Backend​

If you do not already have a BGP backend installed, MetalLB is the recommended starting point.

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.8/config/manifests/metallb-native.yaml

Wait for MetalLB pods to be ready:

kubectl wait --namespace metallb-system \
  --for=condition=ready pod \
  --selector=app=metallb \
  --timeout=90s

Calico​

If your cluster already uses Calico for networking with BGP enabled, no additional installation is needed. Set controller.backend: calico in the Helm values.

Cilium​

If your cluster uses Cilium, ensure BGP control plane is enabled (--enable-bgp-control-plane=true). Set controller.backend: cilium in the Helm values.

Step 2: Configure Authentication​

The controller needs to communicate with the NetActuate API to discover the node it's running on and provision BGP sessions.

When running as a DaemonSet with hostNetwork: true, the NetActuate API identifies the calling VM by its source IP address. No API key is required.

This method requires VM IP-based authentication to be enabled on your NetActuate account. Contact your account manager to enable it if you have not already.

Leave netactuate.existingSecret empty in the Helm values when using this method.

API Key Authentication (fallback)​

If VM IP-based auth is unavailable, create a Kubernetes secret with your API key:

kubectl create secret generic netactuate-api-key \
  --from-literal=api-key=YOUR_VAPI2_KEY \
  -n kube-system

Then set netactuate.existingSecret: "netactuate-api-key" in the Helm values.

Step 3: Install the BGP Controller​

Clone the repository and install with Helm:

git clone https://github.com/netactuate/netactuate-bgp-controller.git
cd netactuate-bgp-controller

helm install bgp-controller ./deploy/helm/bgp-controller \
  -n kube-system \
  --set bgp.localASN=YOUR_ASN \
  --set bgp.groupId=YOUR_GROUP_ID \
  --set "bgp.prefixes[0]=YOUR.PREFIX.0.0/24"

Replace YOUR_ASN, YOUR_GROUP_ID, and the prefix with your actual values from the NetActuate portal.

For more complex configurations, use a values file:

helm install bgp-controller ./deploy/helm/bgp-controller \
  -n kube-system \
  -f my-values.yaml

Example Values File​

bgp:
  localASN: 64512
  groupId: 100
  prefixes:
    - "198.51.100.0/24"
  redundantSessions: false

controller:
  backend: metallb
  reconcileInterval: 30s
  withdrawDelay: 30s
  reannounceDelay: 60s

healthChecks:
  - name: my-app
    url: "http://my-app.default.svc.cluster.local:8080/health"
    interval: 10s
    timeout: 5s
    failureThreshold: 3
    successThreshold: 2

Step 4: Deploy a Service​

Deploy a service that uses a LoadBalancer IP from your anycast prefix:

kubectl apply -f examples/hello-world.yaml

Or create your own LoadBalancer service. The controller's IPAddressPool makes your anycast prefix available to any service of type LoadBalancer.

Verify the service received an IP from your prefix:

kubectl get svc hello-anycast

Step 5: Verify​

Check controller status​

kubectl get pods -n kube-system -l app.kubernetes.io/name=bgp-controller
kubectl logs -n kube-system -l app.kubernetes.io/name=bgp-controller --tail=50

Check BGP resources (MetalLB example)​

kubectl get bgppeers,ipaddresspools,bgpadvertisements -n metallb-system

Check controller health endpoint​

kubectl exec -n kube-system deploy/bgp-controller -- wget -qO- http://localhost:8080/status | jq .

The status response shows the controller state, node info, BGP session status, health check results, and announced prefixes.

Verify in the NetActuate portal​

Navigate to Networking > Anycast > BGP Sessions and confirm:

  • Sessions show "Established" status
  • The customer IP matches your node IP
  • The group matches your configured groupId

Test externally​

curl http://198.51.100.1/
traceroute -n 198.51.100.1

Traffic should route to the nearest POP announcing the prefix.

Health-Based Failover​

The controller monitors HTTP endpoints you configure and automatically withdraws the anycast prefix when all health checks fail. This shifts traffic to the next nearest healthy POP.

How It Works​

  1. When all configured health checks exceed their failureThreshold, the controller deletes the BGP advertisement resource.
  2. The BGP backend stops announcing the prefix. Routes withdraw across the internet (typically 30-90 seconds).
  3. Anycast traffic shifts to the next nearest POP still announcing.
  4. When any health check reaches its successThreshold, the controller recreates the advertisement and re-announces the prefix.

Route Dampening Prevention​

BGP routers penalize prefixes that flap rapidly, potentially suppressing them for 30-60 minutes. The controller includes safeguards to prevent this:

SettingDefaultPurpose
controller.withdrawDelay30sAbsorbs transient failures before withdrawing
controller.reannounceDelay60sPrevents re-announcing into unstable conditions
controller.minAdvertisementDuration5mPrevents rapid flap cycles

Test Failover​

Scale down your service to trigger withdrawal:

kubectl scale deploy my-service --replicas=0

Watch the controller logs to see the health check failures and prefix withdrawal:

kubectl logs -n kube-system -l app.kubernetes.io/name=bgp-controller -f

Verify the advertisement was removed:

kubectl get bgpadvertisements -n metallb-system

Scale back up to trigger recovery:

kubectl scale deploy my-service --replicas=1

Multi-Location Deployment​

For global anycast, deploy the BGP controller on Kubernetes clusters at multiple NetActuate locations. Each cluster announces the same prefix from its location. The controller handles BGP session provisioning and failover independently at each site.

Architecture​

Users worldwide
      |
  [Anycast DNS / IP]
      |
  +---+---+---+---+
  |       |       |
LAX     AMS     SIN
K8s     K8s     K8s
cluster cluster cluster
  |       |       |
BGP     BGP     BGP
ctrl    ctrl    ctrl

Each BGP controller instance:

  • Discovers which NetActuate VM it's running on
  • Provisions BGP sessions for that location
  • Announces the shared anycast prefix
  • Independently monitors local service health
  • Withdraws the prefix if local services fail

No coordination between sites is needed. Each controller operates independently, and BGP routing handles traffic distribution globally.

Per-Location Configuration​

Use the same Helm values at every location. The controller automatically discovers the correct node and location via the NetActuate API. The only required settings are your ASN, group ID, and prefix, which are the same across all locations.

BIRD Backend (Bare Metal and VMs)​

For deployments outside Kubernetes, the controller supports BIRD 2.x as a backend. It writes configuration files directly to disk and signals BIRD to reload.

Set controller.backend: bird in the Helm values (or run the controller binary directly on the VM).

The controller manages these BIRD config files:

  • /etc/bird/peers.d/peer_{sessionID}.conf for each BGP peer
  • /etc/bird/filters.d/anycast.conf for the export filter
  • /etc/bird/static.d/anycast_routes.conf for static routes

Your bird.conf must include these directories. The controller calls birdc configure to reload after changes.

For VM-based anycast without Kubernetes, also see the Configuring Anycast guide for manual BGP session setup with BIRD2.

Configuration Reference​

For the full configuration reference, including all Helm values, backend-specific settings, and status reporting options, see the BGP Controller README on GitHub.

Key Helm Values​

ValueRequiredDefaultDescription
bgp.localASNYesYour allocated BGP ASN
bgp.groupIdYesNetActuate BGP group ID
bgp.prefixesYesIPv4 anycast prefixes (e.g., ["198.51.100.0/24"])
bgp.prefixesV6No[]IPv6 anycast prefixes
bgp.redundantSessionsNofalseEnable multi-router HA sessions
controller.backendNometallbBGP backend: metallb, calico, cilium, or bird
netactuate.existingSecretNo""Kubernetes secret for API key auth (empty = IP-based auth)

Troubleshooting​

Controller stuck in DISCOVERING state​

The controller cannot identify itself via the NetActuate API.

  • Verify VM IP-based authentication is enabled on your account
  • Confirm the DaemonSet has hostNetwork: true
  • Ensure the controller is running on NetActuate infrastructure

BGP sessions created but MetalLB not peering​

  • Verify the node hostname label matches the nodeSelectors on the BGPPeer resource: kubectl get nodes --show-labels | grep hostname
  • Confirm MetalLB was installed in CRD mode
  • Check that MetalLB speaker pods are running in metallb-system
  • Ensure TCP port 179 (BGP) is open between the node and the NetActuate router

Health checks always failing​

  • Use full Kubernetes DNS names for health check URLs: http://my-service.my-namespace.svc.cluster.local:8080/health
  • Increase failureThreshold if the service takes time to become ready
  • Increase healthChecks[].timeout for slow-responding endpoints

Enable debug logging​

Set LOG_LEVEL=debug in the controller environment to see detailed API calls and resource reconciliation.

Need Help?​

Contact support@netactuate.com or open a support ticket from the portal.