Dating back to the early days of the Internet, firewalls were part of the foundation upon which modern web app security was built. Web application firewalls (WAFs) are at the center of many organizations’ security infrastructures because of their capabilities in blocking malicious HTTP traffic.
There’s just one problem: when it comes to bot traffic in particular, WAFs just aren’t sufficient. The sophisticated attack techniques of bad bots have far outpaced any incremental improvements in WAF bot management technology. Cracks are starting to show, and organizations have a choice: they can either incrementally patch the holes and hope bots don’t get through, or adopt a purpose-built, modern solution that is up to the task.
What WAF Bot Management Can and Can’t Do
First, let’s back up and talk about how WAFs work. A WAF is a type of reverse-proxy server that acts as a shield between a web application and the Internet. User requests must pass through the WAF before hitting your server. Using preset policies, the WAF filters out malicious traffic from legitimate traffic, preventing successful attacks against your site.
WAFs are good at protecting you from familiar threats, such as cross-site scripting, SQL injection, buffer overflow and DDoS attacks. Where they falter is in recognizing unknown bot threats in real-time. They cannot recognize bots that piggyback on the identities of real humans and mimic their behavior. Nor can they identify botnets that rotate through thousands of different IP addresses to bypass IP-based rules.
Today’s bots are highly distributed, don’t carry attack signatures and target flaws in your site logic rather than known vulnerabilities. They are extremely complex, continuously evolving and re-tooling to get around WAF policies. WAF bot management capabilities are merely patches on top of a larger security system, and WAFs don’t learn in real-time. Bots, on the other hand, are evolving in real time to evade security systems. Binary, pre-set rules just can’t keep up.