Take the 2-minute tour ×

When inside tracing function, debugging a function call, is it possible to somehow retrieve the calling expression?

I can get calling line number from traceback object but if there are several function calls (possibly to the same function) on that line (eg. as subexpression in a bigger expression) then how could I learn where this call came from? I would be happy even with the offset from start of the source line.

traceback.tb_lasti seems to give more granual context (index of last bytecode tried) -- is it somehow possible to connect a bytecode to its exact source range?

EDIT: Just to clarify -- I need to extract specific (sub)expression (the callsite) from the calling source line.

share|improve this question
will this help? stackoverflow.com/questions/13903751/… –  Alexey Kachayev Dec 19 '12 at 12:03
@Alexey: unfortunately no, the source line is not precise enough for me, I need to extract specific subexpression. –  Aivar Dec 19 '12 at 12:46
maybe if you paste the source line, associated bytecodes, and the instruction you are debugging, we can help you work out where in the source that instruction is coding for. –  IfLoop Dec 19 '12 at 13:24
@TokenMacGuy: I'm not debugging any specific code, I want to create a visual expression debugger on top of Python tracing facilities. –  Aivar Dec 19 '12 at 13:34
add comment

3 Answers

Traceback frames have a line number too:

lineno = traceback.tb_lineno

You can also reach the code object, which will have a name, and a filename:

name = traceback.tb_frame.f_code.co_name
filename = traceback.tb_frame.f_code.co_filename

You can use the filename and line number, plus the frame globals and the linecache module to efficiently turn that into the correct source code line:

line = linecache.getline(filename, lineno, traceback.tb_frame.f_globals)

This is what the traceback module uses to turn a traceback into a useful piece of information, in any case.

Since bytecode only has a line number associated with it, you cannot directly lead the bytecode back to the precise part of a source code line; you'd have to parse that line yourself to determine what bytecode each part would emit then match that with the bytecode of the code object.

You could do that with the ast module, but you can't do that on a line-by-line basis as you'd need scope context to generate the correct bytecodes for local versus cell versus global name look-ups, for example.

share|improve this answer
The thing is that line number (or source line itself) is not granular enough for me, I need to extract specific callsite from that line. –  Aivar Dec 19 '12 at 12:40
@Aivar: then you'd have to parse out the bytecode (from the .f_code object plus the offset). That's a lot more involved. –  Martijn Pieters Dec 19 '12 at 12:41
@Martin: I don't doubt it's involved, but do you know, does bytecode contain source locations at line level only or at character level? From Python documentation I got the impression that bytecode instructions have only source line numbers associated with them. –  Aivar Dec 19 '12 at 12:45
@Aivar: it has line numbers only. –  Martijn Pieters Dec 19 '12 at 12:52
add comment

Unfortunately, compiled bytecode has lost its column offsets; the bytecode index to line number mapping is contained in the co_lnotab line number table. The dis module is a nice way of looking at the bytecode and interpreting co_lnotab:

>>> dis.dis(compile('a, b, c', '', 'eval'))
  1           0 LOAD_NAME                0 (a)
              3 LOAD_NAME                1 (b)
              6 LOAD_NAME                2 (c)
              9 BUILD_TUPLE              3
             12 RETURN_VALUE        
  ^-- line number

However, there's nothing stopping us from messing with the line number:

>>> a = ast.parse('a, b, c', mode='eval')
>>> for n in ast.walk(a):
...     if hasattr(n, 'col_offset'):
...         n.lineno = n.lineno * 1000 + n.col_offset
>>> dis.dis(compile(a, '', 'eval'))
1000           0 LOAD_NAME                0 (a)

1003           3 LOAD_NAME                1 (b)

1006           6 LOAD_NAME                2 (c)
              9 BUILD_TUPLE              3
             12 RETURN_VALUE        

Since compiling code directly should be the same as compiling via ast.parse, and since messing with line numbers shouldn't affect the generated bytecode (other than the co_lnotab), you should be able to:

  • locate the source file
  • parse it with ast.parse
  • munge the line numbers in the ast to include the column offsets
  • compile the ast
  • use the tb_lasti to search the munged co_lnotab
  • convert the munged line number back to (line number, column offset)
share|improve this answer
Thanks! This looks promising, I will give it a try. –  Aivar Dec 19 '12 at 16:00
add comment
up vote 0 down vote accepted

That's how I finally solved the problem: I instrumented each function call in the original program by wrapping it in a call to a helper function together with information about the source location of the original call. Actually I was interested in controlling the evaluation of each subexpression in the program, so I wrapped each subexpression.

More precisely: when I had an expression e in the original program, it became

_after(_before(location_info), e)

in the instrumented program. The helpers were defined like this:

def _before(location_info):
    return location_info

def _after(location_info, value):
    return value

When tracer reported the call to _before, I knew that it's about to evaluate the expression at location represented by location_info (tracing system gives me access to local variables/parameters, that's how I got to know the value of location_info). When tracer reported the call to _after, I knew that the expession indicated by location_info was just evaluated and the value is in value.

I could have written the execution "event handling" right into those helper functions and bypass the tracing system altogether, but I needed it for other reasons as well, so I used those helpers only for triggering a "call" event in tracing system.

The result can be seen here: https://bitbucket.org/aivarannamaa/thonny (pre-alpha at the moment).

share|improve this answer
add comment

Your Answer


