300 Requests in 5 Minutes: How We Chose the Web Application Firewall (WAF) Limit for the Token Route

https://hackernoon.imgix.net/images/QEQBwoE0mIVieVY9Bb1uvEcDMVd2-4p83m48.jpeg

Daniil Romashov, Site Reliability Engineer (SRE)

A single client in a loop could potentially consume all of Lambda's concurrency. Therefore, we've implemented IP address-based rate limiting on the CloudFront edge. The number was taken from one week of logs. We then spent 16 days making final adjustments, including the test runner, a second route, notifications, and CI permissions.

30 for everyone

The access token issuance route is behind CloudFront. The request is then sent to API Gateway, which in turn sends it to Lambda. This lambda function has 30 concurrent executions reserved per region. One client is stuck in a loop on the route, sending approximately 20 requests per second and consuming the entire budget. The remaining users are receiving 429 responses from Lambda. Without tokens, nothing will work.

I wrote this as a comment at the beginning of the waf.tf file. Then I started writing the rule itself....

Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE

Read more