Linux kernel 2.6.38 and 2.6.39 power regression workaround

Recently I've been referencing Linux 2.6.38 and 2.6.39 kernel power drain regression in many of my articles. Fortunately Phoronix has found the root cause of this issue and has suggested a workaround for the affected systems. Also because of the Linux kernel 3.0 merge window closed this power regression will not be properly fixed until Linux kernel 3.1 release. In this article I'm bringing you two ways to work around this problem on your laptop or desktop system running Linux kernel 2.6.38 or newer.

Workaround using Sysfs (temporary, use for testing)

Notice:It seems that in the most recent kernel revisions ASPM is disabled on pre PCIe 1.1 devices and must be manually enabled using pcie_aspm=force kernel argument (the second method in this article). So if it happens that when you try Sysfs method you receive Operation not permitted error, then you can skip to the "Workaround by editing GRUB" method.

Sysfs is virtual file system used amongst other things to configure Linux hardware options from userspace. In plain words you can control your hardware options in real time by writing into the simple text files. First lets check the state of things by running following command:

cat /sys/module/pcie_aspm/parameters/policy

The output of this command will probably be something like this:

[default] performance powersave

This means that the default PCIe ASPM (Active State Power Management) profile is selected. This "default" is where the problem lies. To work around Linux kernel 2.6.38 power regression we must force PCIe ASPM to be enabled. For ASPM to be enabled we must make sure that it stays off the "default" and "performance" profiles. This is how you can do it using Sysfs on Ubuntu based Linux distributions:

sudo -i
echo powersave > /sys/module/pcie_aspm/parameters/policy
exit

For other non-Ubuntu based distros you would use su instead of sudo -i. Controlled this way, this setting remains until you change it again using sysfs or until you reboot. This is actually good thing because before enabling this setting at boot time, users should test is PCIe ASPM working fine on their configuration. This is necessary because there are reports of PCIe ASPM causing lockups when enabled on systems with buggy ASMP BIOS implementation. If this option isn't causing problems on your laptop or desktop system you can proceed to the configuring ASPM by editing GRUB or you can use sysfs to enable this option using init script like I've described here.

Workaround by editing GRUB (permanent)

After you've make sure that PCIe ASPM isn't causing problems on your configuration you can apply this workaround for good by editing GRUB configuration like this:

gksudo gedit /etc/default/grub

Now you find the GRUB_CMDLINE_LINUX_DEFAULT line that might look like this:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"

You should edit this line to look like this:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash pcie_aspm=force"

This way you are passing pcie_aspm=force kernel argument to force PCIe ASPM. This settings will be applied every time you boot your laptop or desktop. At the and you should do following to update GRUB configuration using this command:

sudo update-grub

After reboot you're all set. Your laptop should now spend a lot less juice than without this workaround. Lets all thanks Phoronix for finding the root cause of this Linux kernel 2.6.38 power regression. And feel free to share results for your configuration. Cheers!

DevGenii

A quality focused Magento specialized web development agency. Get in touch!

25 thoughts on “Linux kernel 2.6.38 and 2.6.39 power regression workaround

  1. Paresh Mathur

    Hey,

    I was just trying out your work around (I get 24.4W consumption on ubuntu while I get 16ish Watt on windows). Here is the problem

    rick@Abigail:~$ cat /sys/module/pcie_aspm/parameters/policy
    default performance [powersave]

    It marks powersave as the profile. But That does not make any sense. Can you help?

    Reply
  2. Marko Author

    @Paresh Mathur

    Hi! If I’ve understood correctly, your system isn’t affected by the Linux kernel 2.6.38 kernel bug if your PCIe ASMP isn’t on performance or default. The thing is that Linux has troubles reaching Windows power usage because of tweaks hardware manufacturers take to caterer Windows so Ubuntu power usage is always a bit higher.

    To be sure are you still affected you could do some testing with solid kernel like 2.6.32 and using 2.6.38 and than you will know for sure. Here you can find my article on how to compile kernel from kernel.org using something like 10 steps it is really easy and you can easily test any kernel you want:

    http://www.techytalk.info/compile-upstream-kernel-org-ubuntu-debian/

    Cheers!

    Reply
  3. Rahmadi

    Hi,
    There was error when I typed “echo powersave > /sys/module/pcie_aspm/parameters/policy”. The error was “bash: /sys/module/pcie_aspm/parameters/policy: Permission denied”. I tried adding “sudo” at the beginning of the command, but the same exact error showed up.

    Can you help? Thanks.

    Reply
  4. guyr

    @Rahmadi
    the sudo command apply only to echo powersave
    So i suggest these 3 lines :
    sudo -s
    echo powersave > /sys/module/pcie_aspm/parameters/policy
    exit

    Reply
  5. Rahmadi

    Hi guyr,
    When I typed line number 2 from your suggestion above, another error showed up: “bash: echo: write error: Operation not permitted”.
    I could load the file using sudo gedit, however I could not save it, the error was: “Could not save the file /sys/module/pcie_aspm/parameters/policy. You do not have the permissions necessary to save the file. Please check that you typed the location correctly and try again.”

    I am using Ubuntu 11.04 (natty), kernel 2.6.39 from ppa.

    Thanks for your answer.

    Reply
  6. Marko Author

    @Rahmadi

    Gedit isn’t necessary because I’ve changed the article using quyr pointers (thanks guyr) so it shouldn’t give you any errors. If you really want to do this using gedit, you should use gksudo gedit /sys/module/pcie_aspm/parameters/policy and then drop powersave inside and save. Hope it works 🙂

    Reply
  7. Rahmadi

    Hi Marko,
    I did tried the newest method above (using sudo -i), however I still encountered error: “-bash: echo: write error: Operation not permitted”.

    Thanks.

    Reply
  8. Marko Author

    @Rahmadi

    That is really strange because after you enter sudo -i on Ubuntu operating system you are practically root and there is nothing that can answer “Operation not permitted” to your command. The only thing could be that /sys/module/pcie_aspm node doesn’t exist on your system. The reason could be that you are running pre 2.6.35 kernel that doesn’t support pcie_aspm or that for some reason pcie_aspm isn’t supported on your machine. There is also possibility that you’ve messed things with sudoers file (man sudoers).

    Here’s copy/paste for sysfs procedure from the terminal on my Ubuntu Maverick system with custom 2.6.38.8 kernel:


    marko@marko-pc:~$ cat /sys/module/pcie_aspm/parameters/policy
    [default] performance powersave
    marko@marko-pc:~$ sudo -i
    [sudo] password for marko:
    root@marko-pc:~# echo powersave > /sys/module/pcie_aspm/parameters/policy
    root@marko-pc:~# exit
    logout
    marko@marko-pc:~$ cat /sys/module/pcie_aspm/parameters/policy
    default performance [powersave]
    marko@marko-pc:~$

    Reply
  9. nukleus

    @Marko
    Hi Marko,
    I’m using a gentoo system using 2.6.39 at the moment, and I do get the “Operation not permitted” error when logged in as root or after becoming root using sudo -i, just like Rahmadi does.
    The file does definitely exist and is readable, perms are 0644 (-rw-r–r–). In my kernel config, I have
    CONFIG_PCIEASPM=y
    #CONFIG_PCIEASPM_DEBUG is not set

    Any idea?

    Reply
  10. Rahmadi

    @Marko,

    I am confused myself.
    The file /sys/module/pcie_aspm/parameters/policy does exist. I can open it using sudo gedit (the content is: “[default] performance powersave”), but cannot save it.
    I am using kernel 2.6.39 from ppa.
    I never touch sudoers file.

    Really strange.
    Anyway, thank you very much for your efforts and answers Marko.

    Reply
  11. Marko Author

    @nukleus
    @Rahmadi
    @Tobbs

    Thanks Tobbs, this could be it. It sounds that on some hardware you must first force aspm by editing GRUB and then you can control aspm by using sysfs. This is really dumb and I don’t consider this as a feature, this is a bug. If someone else confirms this because on my hardware it works even with the 2.6.39 kernel, i will add it to the article as notice. Again, Tobbs thanks for this.

    Reply
  12. vitto

    @Marko

    echo powersave > /sys/module/pcie_aspm/parameters/policy
    -bash: echo: write error: Operation not permitted

    also on my acer travelmate

    Reply
  13. Marko Author

    @vitto

    Hi, thanks for reporting with your results. For more info you can check this link:

    https://bugs.launchpad.net/ubuntu/+source/linux/+bug/760131/comments/99

    It looks like in recent kernel revisions ASPM is disabled on PCIe 1.1 devices and must be manually enabled using pcie_aspm=force kernel argument (the second method in this article).

    Edit: I’ve noticed a lot of ASPM related commits in Linux 2.6.39 changelog where kernel devs complain on ASPM on certain hardware:

    http://www.kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.39

    That could explain this kind of behavior.

    Reply
  14. Rahmadi

    I found a way to set “pcie_aspm=force” temporarily:
    -Restart your computer.
    -When the computer restarts, immediately hold Shift, and the boot menu will appear.
    -Press “e” to edit the first entry.
    -Find text “quite splash”, add “pcie_aspm=force” (without quotes) after that text.
    -Press F10 to continue boot with the new setting.
    -To check, type “cat /sys/module/pcie_aspm/parameters/policy” at terminal.
    -In my experience, when you charge the battery (by pluging in the cable), the setting resets to default.

    I am sorry I cannot provide you the source of the information above, I forgot the link.

    Reply
  15. Marko Author

    @Rahmadi

    Hi Rahmadi! The same procedure can be used when GRUB method is used without testing if something goes wrong. Using the procedure you wrote in this situation way we can remove “pcie_aspm=force” from kernel line to be able to boot and then fix GRUB permanently. Thanks for the procedure and great job 🙂

    Reply
  16. bug?

    Has the kernel team confirmed this to be a bug? When I read about it the kernel team said they were aware of it, and that it wasn’t a bug at all.

    Enabling this can cause some machines to crash without warning, if it is off leave it off.

    Reply
  17. Marko Author

    @bug?

    If I’ve understood correctly, this isn’t a bug. The problem is that pcie aspm can’t be reliable enabled on all configurations due to buggy aspm implementations. But I see no reason for disabling sysfs control?

    Reply
  18. Virus

    Hi guys,

    The problem lies deep in the Kernel and it can’t be fixed even with this fix. It is better to just wait for the Kernel 3.1. I think it is not worth fixing or spending time on it to find a work around. Not like it is going to stay the same for next 5 years like windows. Linux CHANGES fast.

    And you might wanna fix this line “At the and you should do following to update GRUB configuration using this command:” “:D

    Reply
  19. Todd

    Hi i have Toshiba L300 – PSLB8E (1A6) model, with H2O Bios inside v2.20, i think i have overheating problems and Fn keys dont work :SS and on battery power i have 30min, on Windows 7 1h :S what is whrong why Windows is better than Linux ?

    Reply
        1. Florin

          Maybe you should follow the instructions in this article. In my
          Toshiba (C650, Intel i3) it quite fixed my battery consumption.

          I had first to set the Grub and then the powersave settings:
          STEP1:
          sudo gedit /etc/default/grub
          Insert in the line “RUB_CMDLINE_LINUX_DEFAULT” pcie_aspm=force, it
          should get like this:
          GRUB_CMDLINE_LINUX_DEFAULT=”quiet splash pcie_aspm=force”
          Update your grub:
          sudo update-grub
          Restart to apply the settings. In my Oneric, STEP2 did not work
          without STEP1.

          STEP2, test for temporary powersave:
          sudo -i
          echo powersave _ /sys/module/pcie_aspm/parameters/policy
          exit
          It should not echo any error.
          STEP2, permanent powersave (it will reset on restart/suspend):
          sudo gedit /etc/rc.local
          echo powersave _ /sys/module/pcie_aspm/parameters/policy
          Restart.

          Check the powersave settings:
          cat /sys/module/pcie_aspm/parameters/policy
          It should print something like this:
          default performance [powersave]

          Reply

Leave a Reply to Rahmadi Cancel reply

Your email address will not be published. Required fields are marked *