Take the 2-minute tour ×

I would like to monitor progress of my Pig jobs. Most of the work is done in UDFs written in Jython. Is there a way to define/increment a Hadoop counter from within a Jython UDF?

Thanks in advance.

share|improve this question

2 Answers 2

up vote 2 down vote accepted

I can't check it right now (sorry for untested code), but in Java UDFs(which are quite similar) with Pig 0.8 it should go like this:

public class INC_COUNTER extends EvalFunc<DataBag> {

    public DataBag exec(Tuple input) throws IOException {

        PigStatusReporter reporter = PigStatusReporter.getInstance();
        if (reporter != null) {
           reporter.getCounter(Counters.EXAMPLE).increment(1);//Counters.EXAMPLE is an enum value
        return null;

I hope this works and you can transform this code into the solution for Jython UDF.

share|improve this answer

You could add a counter to your Hadoop configuration using setInt("<counter>", <value>). Then everytime the UDF runs call and increment the value using setInt("<counter>", getInt("<counter>") + 1). I did something similar once in a Hive UDF I think.

In the Java docs.

share|improve this answer
My UDFs are written in Jython, how can I get the right instance of Hadoop configuration from within a regular Jython function? –  Bolo Feb 7 '13 at 19:56

Your Answer


