Search this site


Metadata

Articles

Projects

Presentations

Project Updates

I spent this weekend working on implementing automatic pattern discovery in the C version of grok. I updated the C and Ruby API to be able to use this discovery:
% irb -rrubygems -rgrok
>> grok = Grok.new
>> grok.add_patterns_from_file("/usr/local/share/grok/patterns/base")
>> pattern = grok.discover("There's no place like 127.0.0.1")
=> "\\QThere's no place like \\E%{IP}\\Q\\E"
>> grok.compile(pattern)
>> grok.match("There's no place like 24.55.33.55").captures
=> {"IP"=>["24.55.33.55"]}
>> grok.match("There's no place like 234.345.12.4")
=> false
This will most likely get used in logstash to provide a helping hand in building log patterns. It could also be used on events that have no pre-defined pattern so we can at least attempt to parse the log line.

On a more important note, this was one of the few remaining features the perl grok implementation had that the C version did not.

You can download the latest version of grok here. You can also install the ruby grok gem with gem install jls-grok.

On another project node, I pushed a new fex release that includes some small fixes and also an rpm spec.