c1.6 Protect the endpoint with AWS WAF and rate limiting

Put a shield in front of the model endpoint

AI inference endpoints are attractive targets: attackers probe them with injection payloads, and abusers hammer them to run up token bills or exhaust capacity. AWS WAF sits in front of your API (via CloudFront, Application Load Balancer, or API Gateway) and inspects every request before it reaches your application and the model. Two layers matter most here:

• Managed rule groups catch common web attacks and known-bad inputs.
• Rate-based rules throttle any single client that sends requests too fast.

Rate-limit abusive callers

A rate-based rule counts requests per aggregation key (client IP by default) over an evaluation window and blocks a key that exceeds your limit until it slows down. This caps runaway token spend and blunts denial-of-wallet attacks:

{
  "Name": "RateLimitPerIp",
  "Priority": 1,
  "Statement": {
    "RateBasedStatement": {
      "Limit": 300,
      "EvaluationWindowSec": 300,
      "AggregateKeyType": "IP"
    }
  },
  "Action": { "Block": {} },
  "VisibilityConfig": {
    "SampledRequestsEnabled": true,
    "CloudWatchMetricsEnabled": true,
    "MetricName": "RateLimitPerIp"
  }
}

Here a client IP is blocked once it exceeds 300 requests in any 5-minute window. Tune Limit to just above your legitimate peak per client.

Aggregate on something better than IP when you can

Behind NAT or a proxy many users share one IP, so pure-IP limiting is coarse. WAF can aggregate on custom keys — a forwarded IP, an API key header, or a session identifier — so you throttle the actual abuser instead of everyone behind the same gateway. Pair the rate rule with a scope-down statement so it only counts the requests that hit your inference path (e.g. /v1/generate), not static assets.

Start in count mode

Deploy new rules in Count mode first and watch the CloudWatch metrics; once you confirm they are not catching legitimate traffic, switch the action to Block. This avoids taking your own users offline with a mistuned limit.