[ACCEPTED]-Profiling in Python: Who called the function?-profiling

I almost always view the output of the cProfile 5 module using Gprof2dot, basically it converts the 4 output into a graphvis graph (a .dot file), for 3 example:

It makes it very easy to determine 2 which function is slowest, and which function[s] called 1 it.

Usage is:

python -m cProfile -o output.pstats path/to/your/script arg1 arg2
gprof2dot.py -f pstats output.pstats | dot -Tpng -o output.png

That may not answer your question directly, but 10 will definitely help. If use the profiler 9 with option --sort cumulative it will sort 8 the functions by cumulative time. Which 7 is helpful to detect not only heavy functions 6 but the functions that call them.

python -m cProfile --sort cumulative myScript.py

There is 5 a workaround to get the caller function:

import inspect
print inspect.getframeinfo(inspect.currentframe().f_back)[2]

You 4 can add as many f_back as you want in case 3 you want the caller caller etc If you want 2 to calculate frequent calls you can do this:

record = {}

caller = inspect.getframeinfo(inspect.currentframe().f_back)[2]
record[caller] = record.get(caller, 0) + 1

Then 1 print them by order of frequency:

print sorted(record.items(), key=lambda a: a[1])

inspect.stack() will give you the current caller stack.

0

You might want to take a look at pycallgraph.

0

It is possible to do it using profiler cProfile in 6 standard library.
In pstats.Stats (the profiler result) there 5 is method print_callees (or alternatively print_callers).

Example code:

import cProfile, pstats
pr = cProfile.Profile()
pr.enable()

# ... do something ...

pr.disable()
ps = pstats.Stats(pr).strip_dirs().sort_stats('cumulative')
ps.print_callees()

Result 4 will be something like:

Function                           called...
                                       ncalls  tottime  cumtime
ElementTree.py:1517(_start_list)   ->   24093    0.048    0.124  ElementTree.py:1399(start)
                                        46429    0.015    0.041  ElementTree.py:1490(_fixtext)
                                        70522    0.015    0.015  ElementTree.py:1497(_fixname)
ElementTree.py:1527(_data)         ->   47827    0.017    0.026  ElementTree.py:1388(data)
                                        47827    0.018    0.053  ElementTree.py:1490(_fixtext)

On the left you have 3 the caller, on the right you have the callee.
(for 2 example _fixtext was called from _data 47827 times and 1 from _start_list 46429 times)

See also:

• docs.python.org/..#print_callees - show call hierarchy. Group by the caller. (used above)
• docs.python.org/..#print_callers - show call hierarchy. Group by the callee.

Couple of notes:

• Your code needs to be edited for this (insert those profile statements).
  (i.e. not possible to use from command line like python -m cProfile myscript.py. Though it is possible to write separate script for that)
• A bit unrelated, but strip_dirs() must go before sort_stats() (otherwise sorting does not work)

I have not used cProfile myself, but most 4 profilers give you a call hierarchy.
Googling 3 I found this slides about cProfile. Maybe that 2 helps. Page 6 looks like cProfile does provide 1 a hierarchy.

Sorry I'm not familiar with Python, but 25 there's a general method that works, assuming you can 24 manually interrupt execution at a random 23 time.

Just do so, and display the call stack. It 22 will tell you, with high probability, what 21 you want to know. If you want to be more 20 certain, just do it several times.

It works 19 because the guilty caller has to be on the 18 call stack for the fraction of time that's 17 being wasted, which exposes it to your interrupts 16 for that much of the time, whether it is 15 spread over many short calls or a few lengthy 14 ones.

NOTE: This process is more like diagnosis 13 than measurement. Suppose that bad call 12 is wasting 90% of the time. Then each time 11 you halt it, the probability is 90% that 10 the bad call statement is right there on 9 the call stack for you to see, and you will 8 be able to see that it's bad. However, if 7 you want to exactly measure the wastage, that's 6 a different problem. For that, you will 5 need a lot more samples, to see what % of 4 them contain that call. Or alternatively, just 3 fix the guilty call, clock the speedup, and 2 that will tell you exactly what the wastage 1 was.

Pycscope does this. I just found it today, so 7 I can't speak to how good it is, but the 6 few examples I've tried have been pretty 5 good (though not perfect).

https://pypi.python.org/pypi/pycscope/

You would use 4 this to generate a cscope file and then 3 a cscope plugin from an editor, VIM specifically. I 2 tried using it with vanilla cscope, it seems 1 that plain cscope gets confused.