I've added predicate tests to grok's pattern match system. These predicates allow you to specify an additional requirement on any matched patterns. Here's the grammar:

  '%' pattern_name [ ':' subname ] [ operator value ] '%'

The difference is that now you can put operator and values on the end of the pattern. The following are valid operators: < > <= >= == ~

== < > <= >=

Match equals, less than, etc. Should be obvious. One special note is that if both the match and predicate values are numbers, then the comparison is done using perl's numerical compare operators. Otherwise, string comparators are used (eq, lt, gt, etc).

~

Regular expression match.

Still confused? Let's run through some examples.

1. Let's find out what's going on in our auth.log on any day from 20:00 to 20:09:

   % sudo cat /var/log/auth.log | ./grok -m '%TIME~/^20:0[0-9]/%'
   Sep 15 20:05:24 nightfall sshd[503]: Server listening on :: port 22.
   Sep 15 20:05:24 nightfall sshd[503]: Server listening on 0.0.0.0 port 22.
   Sep 15 20:07:31 nightfall login: login on ttyv0 as jls
   Nov 12 20:09:42 nightfall xscreensaver[647]: FAILED LOGIN 1 ON DISPLAY ":0.0", FOR "jls"
   Nov 26 20:07:18 nightfall sshd[494]: Server listening on :: port 22.
   Nov 26 20:07:18 nightfall sshd[494]: Server listening on 0.0.0.0 port 22.
     

2. How about looking through 'netstat -s' output for big numbers? Yes, you can use awk for this particular example.

   % netstat -s | ./grok -m "%NUMBER>100000%"
           130632 total packets received
           130465 packets for this host
           114759 packets sent from this host
     

3. Let's look in "all.log" (all syslog stuff goes here) for sshd lines with an IP starting with '83.'

   % ./grok -m "%SYSLOGBASE~/sshd/% .* %IP~/^83\./%" -r "%SYSLOGDATE% %IP%" < all.log
   Oct 17 09:54:37 83.170.72.199
   Oct 17 09:54:53 83.170.72.199
   Oct 17 09:56:02 83.170.72.199
   <snip some output >
   Apr 16 06:54:52 83.14.104.202
   Apr 16 06:54:53 83.14.104.202
   Apr 16 06:54:54 83.14.104.202

If you're interested in playing with this new feature, download grok-20070226.

This seems pretty powerful. Next feature I need to add is the ability to add predicates to patterns after they've been specified. Something like this would be sweet:

% ./grok -m "%APACHELOG%" -p "%NUMBER:RESPONSE==404%"
< some output showing you all apache log entries with response code 404 >

Something like that, which would let you modify the %NUMBER:RESPONSE% pattern to add a predicate requiring that it be 404.