Sungju's Slow Life

Personal journal

  • Where’s PageSlab() macro???

    If you are having hard time to find the definition of PageSlab() in linux kernel, here’s the answer. In include/linux/page-flags.h: Above becomes like below during compile time. Continue reading

  • What is ‘page_cache’, how it is managed and how ‘drop_caches’ dropping this pages?

    – The “buffers/cache” values reported by free include the page cache, but not the dentry cache which is saved in slab ‘dentry_cache’. – page cache is increased and decreased based on the disk access activities and managed by each super block (it means each disk). – ‘echo 1 > /proc/sys/vm/drop_caches’ frees page caches by calling… Continue reading

  • How to calculate available memory for mem_cgroup.

    mem_cgroup is checking available amount of memory the group can charge by calling the below function. /** * mem_cgroup_margin – calculate chargeable space of a memory cgroup * @memcg: the memory cgroup * * Returns the maximum amount of memory @mem can be charged with, in * pages. */ static unsigned long mem_cgroup_margin(struct mem_cgroup *memcg)… Continue reading

  • What happens if you try two commands ‘ethtool -p ‘ and ‘ethtool ‘ in parallel.

    If you start ‘ethtool -p ‘ and also start ‘ethtool ‘ after that, you may see the delays in ‘ethtool ‘ command. It is because any ethtool commands start by taking ‘rtnl_lock’ and ‘ethtool -p’ is keep running for LED on/off. In the below, bnx2x’s identity function just turns on or off the led. get_settings()… Continue reading

  • Print callgraph of a function

    Sometimes you may want to see what functions are called in a function in multiple level. Below command in my extension may help. crash> edis -c irq_exit {irq_exit} -+- {rcu_irq_exit} -+- {warn_slowpath_null} |- {idle_cpu} |- {tick_nohz_stop_sched_tick} -+- {ktime_get} | |- {update_ts_time_stats} | |- {sched_clock_idle_sleep_event} | |- {rcu_needs_cpu} | |- {select_nohz_load_balancer} | |- {rcu_enter_nohz} | |-… Continue reading

  • Why error message not goes into pipe nor redirected path in ‘crash’?

    In the below example, the error always shows in the console. crash> sym ffffffffa02ef86 > /dev/null sym: invalid address: ffffffffa02ef86 This ‘sym’ command is implemented in ‘void cmd_sym(void)’ function in crash. /* * This command may be used to: * * 1. Translate a symbol to its value. * 2. Translate a value to it… Continue reading

  • What happens if numa=off is provided in kernel parameter?

    If “numa=off” is in kernel boot parameter, it will mark ‘numa_off’ global variable which will be checked during initialization function which is ‘x86_numa_init()’ in x86_64. This will make it not call ‘numa_init’ if numa_off is 1. static __init int numa_setup(char *opt) { if (!opt) return -EINVAL; if (!strncmp(opt, “off”, 3)) numa_off = 1; #ifdef CONFIG_NUMA_EMU… Continue reading

  • An example case with some of my commands

    System got high load average and it wasn’t responding for long which is a typical hang situation. It shows total 56 tasks in D (Uninterruptible) state and 5 tasks were in D state longer than 120 seconds which is considered as a hung task. Let’s see what it was waiting for in this process. Alright,… Continue reading

  • ‘edis’ command now shows stack values

    One of the repeating job is finding the argument values passed from the caller. Now by just run ‘edis’, you can see some stack values after the operation code as a comment in yellow color. Continue reading

  • Clocksource tsc unstable

    The reason of the below message 2013-07-16T05:00:05.181538-04:00 xxxxxx kernel: Clocksource tsc unstable (delta = -95170507948 ns). Enable clocksource failover by adding clocksource_failover kernel parameter. clocksource structure /** * struct clocksource – hardware abstraction for a free running counter * Provides mostly state-free accessors to the underlying hardware. * This is the structure used for system… Continue reading

  • pycrashext – A rich python extension

    Based on Pykdump, I wrote a set of plugins named ‘pycrashext’ which is basically trying to help to reduce the troubleshooting time.  My favorite command in this set is ‘edis’ which can display source code in between disassembled lines. This requires an additional source code server with source codes, but once you have it, it… Continue reading

  • Python/CRASH API aka pkydump

    I am dealing with vmcore analysis for the most of my daily work. To speed up the analysis, I needed some extra command set on top of the commands ‘crash’ is providing. Luckily there is a tool names ‘pkydump’ which is a crash extension and also provides a way to implement extensions using python. I… Continue reading