Sungju's Slow Life

If you are dealing with a vmcore (Linux memory dump), you must be familiar with ‘crash’. It is a powerful tool, but it doesn’t cover all the data you can find in Linux kernel. So, there comes ‘mpykdump’ which is a crash extension which understands python code.

mpykdump comes with many prebuilt commands that you can check by run ‘extend’, but, the biggest benefit of using ‘mpykdump‘ is that you can write your own extension using python. So, I am going to cover the very basic about writing python extension to reduce the burden you may have while dealing vmcore.

note: I am not covering how to install mpykdump here.

Basic steps to make your extension available

1. Set ‘PYKDUMPPATH’ environment variable to point your extension file location
2. Write registration code
3. Write actual extension code
4. Put ‘epython <registration code file name>’ in ${HOME}/.crashrc

Set ‘PYKDUMPPATH’

Best place to put this will be ~/.bash_profile or ~/.bashrc if you are using bash.

export PYKDUMPPATH=/root/myext:$PYKDUMPPATH

If you miss the path or put incorrectly, the command will be failed with the below error message.

crash> commandline
 Cannot find the program <commandline>

Registration code

This file’s main purpose is to register commands to make the mpykdump understands the existence of the command.

$ cat regext.py 
from crash import register_epython_prog as rprog
from pykdump.API import *


help = '''
Show kernel boot options.
It has the similar output as 'p saved_command_line'
'''

rprog("commandline",
      "Show commandline",
      "-h   - list available options",
      help)

Actual extension code

Now it is time to write an actual extension code in python. Here’s an example code that is printing kernel command line parameters by getting it from ‘saved_command_line’.

$ cat commandline.py 
from pykdump.API import *

command_line = readSymbol("saved_command_line")

print(command_line)

It is really simple one, but will show you the command line.

Register the extension

To register the extension, you should run ‘epython regext.py’. As it is better to be registered when you launch the crash, usually put this in ~/.crashrc.

$ cat ~/.crashrc
extend /root/myext/mpykdumpx86_64.so
epython /root/myext/regext.py

Start ‘crash’ and see if it is registered.

crash> extend
SHARED OBJECT                           COMMANDS
/cores/crashext/mpykdumpx86_64.so       epython <...> commandline 
crash> man commandline

NAME
  commandline - Show commandline

SYNOPSIS
  commandline -h   - list available options

DESCRIPTION

Show kernel boot options.
It has the similar output as 'p saved_command_line'

crash> commandline
ro root=/dev/mapper/vglocal-root crashkernel=512M rd_NO_LUKS rd_LVM_LV=vglocal/root rd_NO_MD  KEYTABLE=us SYSFONT=latarcyrheb-sun16 rd_NO_DM LANG=en_US.UTF-8 rhgb quiet transparent_hugepage=never elevator=noop printk.time=1

 ** Execution took   0.00s (real)   0.00s (CPU)

It is almost done, but if you try ‘command -h’, it doesn’t show options, but just the same parameter value will be printed.

To avoid this and to get an options, you can use ‘OptionParser’ class.

$ cat commandline.py 
from pykdump.API import *


def commandinfo():
    op = OptionParser()
    op.add_option("-d", "--details", dest="show_details", default=0,
                  action="store_true",
                  help="Show details")

    (o, args) = op.parse_args()

    command_line = readSymbol("saved_command_line")

    print(command_line)
    if o.show_details:
        print("0x%x : saved_command_line" % (Addr(command_line)))


if __name__ == '__main__':
    commandinfo()

Some useful APIs

mpykdump provides many useful APIs which is the reason it was named ‘Python/CRASH API’. Here are some APIs that you can use in your extension code.

• exec_crash_command(“command with arguments”) : Run crash command and return string
• readSymbol(“symbol”) : Read symbol using symbol name that can be accessed in python
• addr2sym(address) : Read symbol from address
• sym2addr(“symbol”) : Get address of symbol using symbol string
• Addr(symbol) : Get address for the symbol variable
• member_offset(“struct name”, “entry_name”) : Get distance of ‘entry_name’ in ‘struct name’
• symbol_exists(“symbol”) : Check if symbol exists

Below is an example that is added to the above.

$ cat commandline.py 
from pykdump.API import *


def commandinfo():
    op = OptionParser()
    op.add_option("-d", "--details", dest="show_details", default=0,
                  action="store_true",
                  help="Show details")

    (o, args) = op.parse_args()

    command_line = readSymbol("saved_command_line")

    print(command_line)
    if o.show_details:
        print("0x%x : saved_command_line\n" % (Addr(command_line)))
        sys_i = exec_crash_command("sys -i")
        print(sys_i)


if __name__ == '__main__':
    commandinfo()