Print callgraph of a function

Sometimes you may want to see what functions are called in a function in multiple level. Below command in my extension may help. crash> edis -c irq_exit {irq_exit} -+- {rcu_irq_exit} -+- {warn_slowpath_null} |- {idle_cpu} |- {tick_nohz_stop_sched_tick} -+- {ktime_get} | |- {update_ts_time_stats} | |- {sched_clock_idle_sleep_event} | |- {rcu_needs_cpu} | |- {select_nohz_load_balancer} | |- {rcu_enter_nohz} | |-Continue reading “Print callgraph of a function”

crash extension ‘pstree’

‘crash’ is useful tool to analyse system crashes or debugging in Linux system. It has many useful commands, but sometimes I wanted to get full picture of process list that was running at the time of crash. You can get process list with ‘ps’, but if you want to get hierarchical view, only ‘ps -p’Continue reading “crash extension ‘pstree’”

How to setup debugging environment for RHEL packages

It’s hard to set the debugging environment in a different machine as it has different packages installed in each systems. It’s really hard when we are talking about distributed packages as it requires lots of other libraries with separate debug info packages. So, I made a small application that you can use to set theContinue reading “How to setup debugging environment for RHEL packages”

Tracing a function with jprobes

One problem with kprobes is that you can’t check validity of the arguments passed to the function you are monitoring. For that matter, jprobes comes in. It’s basically make a wrapper for the existing function and will be called instead without make any changes to the existing function. jprobes is an extention to the kprobesContinue reading “Tracing a function with jprobes”

Tracing an instruction or a function with kprobes

As the kernel is running on top of all other services, it’s hard to debug it in a live system. You can use ‘gdb’ on a live system, but you only can check the current values of some exported symbols. You can’t use breakpoint on a running kernel. If you set a breakpoint, it’ll stopContinue reading “Tracing an instruction or a function with kprobes”

Extending SysRq

Basics about SysRq During the kernel debugging, you can use SysRq to get some details about the system status at some point or to execute some commands without typing the command. We can use one of the below method to trigger the operation. Method 1. $ echo 1 > /proc/sys/kernel/sysrq Press ‘Alt-SysRq-[key]’ combination to triggerContinue reading “Extending SysRq”