Take the 2-minute tour ×

i have tests that i ran which can take up to 15m at a time. during these 15m, a log file is periodically written to. however, most of the content is useless.

in response to this i have a python script that parses out the useless text and displays the relevant data.

what i'm trying to achieve is similar to what tail -f log_file, constantly updating the terminal with the newest additions to a file. i was thinking that if a python script ran as a process, it could parse the log file whenever the tests write to it, then the python script can go to sleep until interrupted again once the log file is written to.

any ideas how one can achieve this?

i already have a script that does the parsing, i just don't know how to make it do it continually and efficiently.

share|improve this question
What platform?​ –  Ignacio Vazquez-Abrams Jul 15 '11 at 19:38
using a mac osx –  tipu Jul 15 '11 at 20:26

4 Answers 4

up vote 3 down vote accepted

You could just have the script filter standard input, and pipe tail -f through it. When you're waiting on stdin, your script will sleep, so it's plenty efficient.


python long_running_script.py && tail -f log_file | python filter_logs.py

Your script can be something like

while true:
    line = sys.stdin.readline()
    if filter_line(line): print line
share|improve this answer
wait wait wait i can pipe the result straight into a python script instead of a log file to begin with? –  tipu Jul 15 '11 at 20:28

looks like you need something like "pytailer": http://code.google.com/p/pytailer/

While I never used it myself, last example looks like what you want.

share|improve this answer

any ideas how one can achieve this?

This should be pretty easy to do. Most of what you want is already part of your OS.

python test.py | python log_parser.py

Be sure your tests write their log to stdout instead of some other file. This is often easy to do with small changes to the logging configuration.

share|improve this answer

Having implemented almost this exact tool, I had great success using the inotify capability in twisted

share|improve this answer

Your Answer


