Technical notes, tips, howtos, etc

Thursday, February 12, 2009

Splunk: get list or count of addresses

I need to see a list of unique IPs that generated errors. The IP of our visitors are tracked in a X-Forwarded-For header as our load balancers and firewalls override the original in the IP packet. I already had Splunk set to drop the value into a variable named xfwd:

Transforms.conf:
[ezlmxfwd]
REGEX = X-Forwarded-For: ([0-9.]+)
FORMAT = xfwd::$1


Props.conf:
REPORT-xfwd = xfwd


Now for the searches. First, a list of IPs, with a count of how many times each showed up:
* | stats count by xfwd


Next, a count of distinct IPs:
* | stats distinct_count(xfwd)


More interestingly, a list and a distinct count for /24 addresses:
* | rex field=xfwd "(?[0-9]+\.[0-9]+\.[0-9]+)" | stats count by classc
* | rex field=xfwd "(?[0-9]+\.[0-9]+\.[0-9]+)" | stats distinct_count(classc)


The 'rex' command is a little more involved. It's saying to search through the field xfwd and apply the given regex. The string between the angle-brackets is the name that will be given the grouping's (inside the parens) match. Essentially we are creating a new field on the fly using regex.

.

No comments: