Load Balancer Setup — Route Traffic Reliably Across Your Services

The load balancer choice depends on your deployment model and protocol requirements. AWS ALB is the default for ECS and EKS workloads — it integrates natively with target groups, auto-scaling, and WAF. Nginx works everywhere and handles HTTP, TCP, and UDP with fine-grained control over routing and buffering. HAProxy excels at raw performance and is the right choice for high-throughput TCP proxying. Traefik is built for container environments with automatic service discovery from Docker or Kubernetes labels.

For most teams running on AWS with containerized workloads, ALB is the right choice. It is managed (no patching, no scaling), integrates with ACM for free SSL certificates, and supports advanced routing (path-based, host-based, header-based, query-string-based). For teams running on bare metal or VMs, Nginx or HAProxy provides equivalent functionality without cloud vendor lock-in.

For Kubernetes, the choice is between cloud load balancers (ALB via the AWS Load Balancer Controller) and in-cluster ingress controllers (Nginx Ingress, Traefik). We typically recommend cloud load balancers for external traffic (better DDoS protection, native WAF integration) and in-cluster ingress for internal service-to-service routing.

AWS ALB: We provision ALBs via Terraform with HTTPS listeners, ACM certificates, and target groups for each service. Routing rules direct traffic based on hostname (api.example.com to the API service, app.example.com to the frontend) or path (/api/* to the API, everything else to the frontend). Health checks are configured with appropriate thresholds — not the defaults, which are often too aggressive and mark healthy instances as unhealthy during deployments.

Health Checks: The health check endpoint is critical. We configure a dedicated /health endpoint that checks database connectivity, Redis connectivity, and any critical external dependencies. The endpoint returns 200 when the service is ready to accept traffic and 503 when it is not. Health check intervals, thresholds, and timeouts are tuned based on your application's startup time and failure characteristics.

SSL/TLS: SSL terminates at the load balancer. We configure TLS 1.2+ with modern cipher suites, HSTS headers, and automatic certificate renewal via ACM or Let's Encrypt. For end-to-end encryption requirements, we configure SSL between the load balancer and the backend using self-signed certificates or a private CA.

Connection Management: We tune keepalive timeouts, idle timeouts, and connection draining settings. The ALB's idle timeout must be higher than your application's keepalive timeout to prevent 502 errors during load balancer connection reuse. We set deregistration_delay to allow in-flight requests to complete during deployments.

WAF Integration: For public-facing applications, we attach AWS WAF to the ALB with managed rule groups (SQL injection, XSS, known bad inputs) and rate-limiting rules. Custom rules block traffic patterns specific to your application's threat model.

A production-ready load balancer setup:

• Load balancer provisioning — ALB, Nginx, HAProxy, or Traefik configured for your architecture
• SSL/TLS — certificate provisioning, TLS 1.2+, HSTS, automatic renewal
• Routing rules — host-based and path-based routing to your services
• Health checks — application-aware health endpoints with tuned thresholds
• Connection tuning — keepalive, idle timeout, and drain settings optimized for your traffic
• WAF rules — managed and custom rules for common web attacks
• Monitoring — request rate, error rate, latency percentiles, and active connection dashboards
• Runbook — troubleshooting guide for common load balancer issues (502s, 504s, health check failures)