Ruby on Rails Log Analyzer – RAWK

October 11, 2006

Rail’s Analyzer with Klass

Download it: rawk.rb
run ruby rawk.rb -h for help.

I wanted to analyze Spongecell’s log file to see where the most of our cpu time was being used. I checked out rails-analyzer.rubyforge.org and after navigating myself in circles following the links I finally decided to download some stuff, install some gems, and give it a try. It didn’t work.

I didn’t understand why I needed whole packages just to look at one log file. I could get a lot of information from the logs just using grep. So I started to write a tool in awk until Tom told me I should use ruby.

Of course I should use ruby.

Rawk is the result. Many sets of data are presented and grouped in many ways. The first thing you will want to optimize are probably the actions that your server sepnds the most time on. I ran rawk on our log a few minutes ago:

sum.gif

The four most expensive requests for Spongecell are the poller (for keeping your calendar display up to date), iCalendar feeds from Spongecell, the normal web view of Spongecell, and RSS feeds.

Please give this tool a try and let me know what you think. I’m happy to include improvements or make modifications.

** This script is released as beerware, something I first heard about on a mac bittorrent client. They defined it as software that is free and if you like it you should buy yourself a beer.

About these ads

25 Responses to “Ruby on Rails Log Analyzer – RAWK”

  1. Tom Lester Says:

    Thanks for the script. I gave it a try and it worked well. One recommendation is to change the name to something that reflects the purpose of the tool instead of its previous implementation. Cheers.

  2. Chris Hobbs Says:

    Thanks for the comment. You’re right, perhaps Rawc – Rail Analyzer with Class – would have caused slightly less cringing from ex-awk users.

  3. topfunky Says:

    I’m using the Robot Co-Op’s libraries. There are a few things that would be great in Rawc:

    * A little more abstraction so you could use it to parse logs and then pump that data into a database for graphing over time
    * An overall average requests per second
    * A real-time option where it tails a live log and shows the actual requests per second being served up, like rails_stat
    * A gem? (see Hoe for an easy way to make a Rubygem and register it as a command-line app)

  4. MIke Bailey Says:

    Thanks for the script – it was great to be able to quickly confirm that our latest RoR app doesn’t have any slow pages. :-)

  5. Marcus Says:

    Wow. That was painfully simple to use and extremely useful. Thanks!


  6. [...] Ruby on Rails Log Analyzer – RAWK « ckh sponge Saving for the future – could be handy to have. (tags: rails ruby RoR rawk analyzer tools performance log) [...]

  7. Blizzo Says:

    This is nice, I was looking for a simple way to see how much traffic my new toy-app was getting and rails-analyzer looked like a lot of work and configuration for what I needed. This worked like a charm and was very speedy.

  8. topfunky Says:

    rawk.rb is a great idea and I used it for a few months, but I needed all the features of pl_analyze.

    I wrote an alternate logger that can be used simply with pl_analyze. I use it on every one of my sites now.

    http://nubyonrails.com/articles/a-hodel-3000-compliant-logger-for-the-rest-of-us


  9. [...] Hobbs’ Rails Analyzer with Klass, or Rawk, is a painless rails log analyzer.  Its a great tool to keep on hand.  I know there is a [...]

  10. RussianGeek Says:

    Thank you! Simple, clear and useful.

  11. J. Chufar Says:

    Great tool! Just what I needed this morning to grab profile of activity.

    Tks!

  12. Peter De Berdt Says:

    Rails > 2.0 uses the BufferedLogger instead of the normal logger, one way is to switch back to the old logger, but I wanted to find a way to stick with the new BufferedLogger.

    Although not the most elegant of solutions, but you can just patch the add method of the BufferedLogger in case you want to keep it.

    Create a file custom_logger.rb in your initializers folder and paste the content of the pastie in it (it only puts pid in the log message in production mode, it’s less ugly):

    http://pastie.caboo.se/171954


  13. Please have a look at request-log-analyzer, another tool to analyze your Rails log files. I may not be fully objective as the author, but I think the reports request-log-analyzer producers are superior to those of rawk. Moreover, no special logger is needed.

    Another advantage is that you can adjust the log parser easily to support log messages that are unique to your applications (locales, what user is logged in, etc).

    See the request-log-analyzer project wiki for more information.


  14. [...] looking at the logs, using rawk to analyze the top offenders, I was unable to find any particular controllers or actions that were [...]

  15. Mathieu Says:

    it seems to be parsing the log properly but shows no numbers
    the columns on the right are all zeros

    what is wrong?

  16. Chris Hobbs Says:

    Do you have the latest version of rawk?

    http://github.com/whatcould/rawk/tree/master

    Rails 2 changed the log format so that could be the problem.

  17. Mathieu Says:

    1.2 is the version I have

  18. jaycode Says:

    Hi, mine always return 0 for sum, max, median etc. How to fix this?

  19. lothar59 Says:

    same problem as jaycode, i got only 0 for sum, max, median etc. can anyone help ?

  20. lothar59 Says:

    can’t figure out this problem anywhere, it would be much appreciated any help on that.

  21. lothar59 Says:

    I took the one from Chris Hobbs on github it worked. Seems like the link rawk1.2 the one from Download it doesn’t. ty

  22. ishu3101 Says:

    When I run the script, I get the following message.

    Printing report for total request times

    Request Count Sum Max Median Avg Min Std
    rawk1.2.rb:42:in `/’: divided by 0 (ZeroDivisionError)
    from rawk1.2.rb:42:in `average’
    from rawk1.2.rb:57:in `to_s’
    from rawk1.2.rb:211:in `print_stats’
    from rawk1.2.rb:137:in `initialize’
    from rawk1.2.rb:238:in `new’
    from rawk1.2.rb:238:in `’

    How do you fix this?

  23. Mark Says:

    I took 1.2 from github and ran on my system. All times were zeros.

    Investigating the actual logs vs. the code I found that the code was expecting floats for the times… MY log had all integer values.

    To fix I modified the handful of lines that did the regexp parsing for time_string. I changed it from “time_string = $_[/DB: \d+\.\d+/]” to “time_string = $_[/DB: \d+/]” thus removing the search from including a decimal point followed by one or more digits. Remember to also change “time_string = time_string[/\d+\.\d+/] if time_string” to “time_string = time_string[/\d+/] if time_string” for the same reason.

    To the author – THANK YOU!! I was going to have my students figure this out. Now I can have them focus on doing the actual work – figuring out what they can improve!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: