I often implement large list of IP and URL whitelisting/HTTP header based controls on F5 using iRules and Data Groups. What I found is “Data Groups” are one of the easiest way to handle a large number of matching keys and values!  As per F5 official documents – data group is the simplest way to maintain a list of permanently matched keys and values. A large list IP address or URL or any string/integer matching can be easily fit within iRule using data groups. Add/Remove entries within data groups are super easy.

Some old documents mentioned that data groups have impact on performance – truth is since TMOS v10.x and above data groups have minimum performance impact. F5 did some testing on performance using data groups and here’s some of the results (copied from F5 site):

The testing was done using 10,000 CPS, 1 HTTP request per TCP connection.

Baseline: TCP + HTTP profile + Blank iRule (when RULE_INIT { } )

Baseline results: Total CPU % used = 23%; TMM CPU % used = 16%; TMM Mem (MB) = 254

Check using iRule that I used in the video against datagroup with 1,000 entries (and no match found, so the entire group is forced to be searched).

Results: Total CPU % used = 25%; TMM CPU % used = 18%; TMM Mem (MB) = 259

Check against datagroup with 2,000 entries (no match found).

Results: Total CPU % used = 25%; TMM CPU % used = 18%; TMM Mem (MB) = 261

Check against datagroup with 10,000 entries (no match found).

Results: Total CPU % used = 26%; TMM CPU % used = 18%; TMM Mem (MB) = 277

So, even with a datagroup with 10,000 entries, the performance hit is minimal.

Here is an example of how to use data groups within iRules; lets say – to whitelist a list of IP address and block requests at TCP layer (TCP three-way handshake which happens before HTTP) – (i)we will create two test data groups and (ii)then bundle them together in an iRule and (iii)finally apply to a virtual server.

i. create data group A – iRules > Data Group List – create “test_data_group_A”; set type Address; enter all the IP address or you can import list file or add them to “/config/bigip.conf” (reload configuration if you edit the /config/bigip.conf manually). This is what it looks like –

ltm data-group internal /Common/test_data_group_A {
 records {
 1.1.1.1/32 { }
 1.1.1.2/32 { }
 1.1.1.3/32 { }
 2.2.2.1/32 { }
 2.2.2.22/32 { }
 }
 type ip
}

ii. follow the same above and create “test_data_group_B”
iii. create an iRule and put them together to allow requests only from the listed IPs (use “class match” command to refer to a data group) –

when CLIENT_ACCEPTED {
if {[class match [IP::client_addr] equals test_data_group_IP_A] } 
{}
elseif {[class match [IP::client_addr] equals test_data_group_IP_B] }
{} 
else
{drop }
}

iv. Apply the above iRule to a virtual server

Thats all! the above should only allow the IPs listed on the Data Groups.

In case if it is required to add/remove new IPs – then only add them to the data groups; no need to touch the iRule anymore.