Cross-compiling Linux Kernel for Raspberry Pi on Ubuntu 12.04

About

Why recompile the kernel?

Because few drivers might be missing. In my case, the eGalax USB touchscreen driver had to be enabled.

What is this cross compilation thing?

It simply means compiling on a different architecture (eg, Intel processor). You want this as your laptop is most probably 5-10x faster than the Raspberry.

What is your setup?
In this tutorial I assume a separately running Raspberry Pi connected to the network, with the hostname raspberrypi.

Pre-requisites

Install the compiler (for arm) and the tools necessary:

ubuntu$ sudo apt-get install git-core gcc-arm-linux-gnueabi make ncurses-dev

I also created a symlink for the cross-compiler. This is needed as the CROSS_COMPILE kernel flag takes the prefix "arm-linux-gnueabi-", and appends "gcc" to it, without the version number.

ubuntu$ sudo ln -s /usr/bin/arm-linux-gnueabi-gcc-4.6 /usr/bin/arm-linux-gnueabi-gcc

Obtain and uncompress the kernel source:

ubuntu$ wget --no-check-certificate http://github.com/raspberrypi/linux/archive/rpi-3.10.y.tar.gz
ubuntu$ tar xvfz rpi-3.10.y.tar.gz
ubuntu$ cd linux-rpi-3.10.y

Note: 3.10.y is the current stable branch at the time of the writing (November 2013), you might want to change this.

Configuration

Copy and uncompress the current Raspberry kernel configuration (your scp usage might vary, I assume default hostname and username):

ubuntu$ scp pi@raspberrypi:/proc/config.gz .
ubuntu$ gunzip config
ubuntu$ mv config .config

Now comes the tricky part. Since the kernel we are compiling is (probably) more recent than the one we have on the Raspberry, more configuration choices have been added. The make the kernel ask for the new config options only, issue:

ubuntu$ make oldconfig

Follow the instructions, you should be safe to accept the default answers offered. When done, you can optionally run the shiny menu-based configuration to change your needs -- eg, to enable the touchscreen driver:

ubuntu$ make menuconfig

Compilation

This can take several hours. If you have multiple CPU cores, to use all of them, make sure to specify -jN, where N is number of cores + 1. Eg, a dual-core machine should use -j3.

ubuntu$ make ARCH=arm CROSS_COMPILE=/usr/bin/arm-linux-gnueabi- -j3

When done, we should have the kernel and all the modules built. The module binaries must be installed to separate them from the source files:

ubuntu$ mkdir ../modules
ubuntu$ make ARCH=arm CROSS_COMPILE=/usr/bin/arm-linux-gnueabi- INSTALL_MOD_PATH=../modules modules_install

Upgrade the Raspberry Pi Binary Firmware

On the Raspberry Pi, back up the /boot directory containing the boot loader and the current kernel abd the /opt directory containing the VideoCore firmware:

raspberrypi$ sudo cp -R /boot /boot.bak
raspberrypi$ sudo mv /opt /opt.bak

Warning: without these files your Raspberry will not be bootable.

Obtain the most recent Raspberry Pi firmware (takes long, huge repository -- you might choose to clone this to Ubuntu and scp over the required files only):

raspberrypi$ git clone git://github.com/raspberrypi/firmware.git

...then copy over the most recent boot loader files:

raspberrypi$ cp firmware/boot/bootcode.bin /boot
raspberrypi$ cp firmware/boot/fixup.dat /boot
raspberrypi$ cp firmware/boot/start.elf /boot

...and the VideoCore firmware (assuming you are using hard float; use command  gcc -v 2>&1 |grep with-float=hard to find out; if it prints a line, you have hard-float configured):

raspberrypi$ cp -R firmware/hardfp/opt /

Note: this "firmware" is different to the module-loaded firmware that is included in the Linux kernel.

Upload the Kernel to the Raspberry

First, let's transfer the necessary files:

ubuntu$ scp arch/arm/boot/zImage raspberrypi:kernel.img
ubuntu$ scp -r ../modules/lib/modules raspberrypi:
ubuntu$ scp -r ../modules/lib/firmware raspberrypi:

Then let's move them to their place:

raspberrypi$ sudo cp kernel.img /boot
raspberrypi$ sudo cp -R modules /lib
raspberrypi$ sudo cp -R firmware /lib

You should be able to reboot now.

Alternative Choices

Kernel source can be obtained through git as well; this takes much longer though:

mkdir linux-rpi-3.10.y
cd linux-rpi-3.10.y
git init
git fetch git://github.com/raspberrypi/linux.git rpi-3.6.y:refs/remotes/origin/rpi-3.10.y
git checkout rpi-3.10.y

Also, instead of scp'ing to a running Raspberry, you might want to insert the SD card into your Ubuntu machine and copy the files over. (You must obviously do so if the Raspberry does not boot.)

Resources

• http://mitchtech.net/raspberry-pi-kernel-compile/
• http://elinux.org/RPi_Kernel_Compilation

Using a 800x480 LCD with Raspberry Pi

Is simple. Into /boot/config.txt, put:

#increase HDMI signal strength (just a black screen if not set!)

config_hdmi_boost=4

#remove black borders

disable_overscan=1

#set specific CVT mode

hdmi_cvt 800 480 60 6 0 0 0

#set CVT as default

hdmi_group=2

hdmi_mode=87

My LCD is the Lilliput 669GL -- works like a charm.

RealTek RTL-8029 under Linux 3.x

A short note with a solution of a 1.5-hour long struggle.

If you have an older Realtek 8029 network interface card, you need to use the ne2k-pci kernel module. With stock kernels shipped with most distributions today this should be no problem, they usually have this module readily precompiled.

However, if you are compiling the kernel yourself, it can be tricky to find the driver among the Linux Kernel configuration options. So here it is:

Device Drivers => Network Device Support => Ethernet driver support => National Semi-conductor devices => National Semi-conductor 8390 devices => PCI NE2000 and clones support

Note: as the help of this option says, this kernel module should be used for all of the following network cards: RealTek RTL-8029, Winbond 89C940, Compex RL2000, KTI ET32P2, NetVin NV5000SC, Via 86C926, SureCom NE34, Winbond, Holtek HT80232, Holtek HT80229.

C/C++: Async TCP Connection Failure Detection on OSX / BSD

As the hard-core C programmers know, the connect(2) function can operate in to modes:

• blocking: the program will stop running until connection succeeds/fails/times out (latter can be as long as 30 seconds!)
• non-blocking: connection is initiated only, and the socket's file descriptor can be checked if connection has completed or not (as a standard, check for writability with select(2)).

Whenever select(2) returns, we must be able to check if our connection attempt was the triggering event, that is, if the connection attempt has been finished or not.

One might believe the manuals and tutorials, which basically tell following:

To check the state of an asynchronous connection attempt, use getsockopt(2) to retreive the SO_ERROR value. If this is 0 (ESUCCESS), the connection has succeeded. If EINPROGRESS, the attempt has not been finished yet. Otherwise, the attempt has failed.

Problem: this is only true for Linux. On BSD (and yes, this includes OSX), when a connection is refused, getsockopt(2) will still return ESUCCESS. Even more strange, a write attempt to this socket simply makes the program exit - without a core dump.

What then?

According to this excellent summary, we are best off double-checking the state of the socket. The best option listed is issuing a getpeername(2) call, which either succeeds (connection established!) or fails (connection failed).

In code:

  // assuming connection is already in progress, select(2) has returned,
  // socket file descriptor is fd_:

  int err = -1;
  int len = sizeof(err);
  if (getsockopt(fd_, SOL_SOCKET, SO_ERROR, (int*)&err,
                 (socklen_t*)&len) != 0) {
    handleConnectionError("getsockopt error");
    return;
  }   
    
  struct sockaddr sa;
  socklen_t saLen = sizeof(sa);
  int gpnErr = getpeername(fd_, &sa, &saLen);
  
  if (err == 0 && gpnErr == 0) {
    handleConnected();
    return;
  }

  if (err == EINPROGRESS) {
    // do one more select(2) run on the file descriptor fd_
  }

Speeding Up Motorola Droid 2 / Milestone 2

I expect my phone to run smooth and fast. I mean, it _has_ the hardware to do so! My old server has a Pentium III processor ticking at 1GHz, it has 512M RAM. I even used Eclipse + Tomcat to code a web application for my thesis program in java!

The Milestone 2 has similar specs. Okay, different architecture, but still, I expect it to be pretty fast. And since I installed Viber, it became unusable and slow, sometimes having to wait ~10 seconds just to see the dialpad appear!

Buy a new phone? Galaxy S4 or Xperia Z, for ~$1000?? Naaaay.

Jelly Bean: a New Phone for Free

Thanks to the guys at xda-developers.com, Jelly Bean, the shiny new operating system is available for the Milestone 2 as well, via the CyanogenMod custom ROM.

Installing CyanogenMod in X Steps

• Root the phone; see http://wiki.rootzwiki.com/Motorola_Milestone_2 for details. If you're paranoid, download the rooter from here and compile and run yourself.
• Install Droid 2 Bootstrap Recovery; download for free from here (I did this manually, ie just gaining root on the phone but not installing the "su" command; just checking the error messages the installer gave me and typing the commands myself. Call me paranoid, but I like to see what's being done.)
• Download CyanogenMod onto the sdcard - from here: http://code.google.com/p/cm-milestone2/downloads/list
• Do the same with the official Google Apps - from here: http://goo.im/gapps

Speed Issue

While very happy with the slick UI, all the new features of Jelly Bean, the phone continued to crawl after Viber got reinstalled. What now?

Fortunately, CyanogenMod has a great built-in option: you can change the CPU governor. If a CPU supports it, Linux can decrease the clock speed if there's no demand, to save battery. Go to System Settings => Performance => CPU Governor, and select "Performance". Also, tick the Set on boot option, to make this change permanent.

While the phone got much snappier, I did not notice considerably less battery time. I think this is due to the fact that when the screen is off (most of the time), the phone sleeps (ie, CPU speed has no effect), and when it's on, the CPU usage is high anyway - due to all the effects, etc.

Another tip: turn off background data (maybe enable it selectively for a handful of applications). This will save bandwidth and battery.
Update: recently I installed LINE, but I had to realize it drains my battery extremely fast. Switching back to "Ondemand" governor solved this issue.

CyanogenMod too Risky?

In this case, "just" root the phone, and install an application that is able to set the CPU governor. (If you're a Linux guru, edit the boot scripts yourself from the terminal to issue "echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor" on every bootup.) 

Just Another HOWTO on Subversion-to-Git Conversion

I found many blogs detailing how to do this -- each tailored for different circumstances, some working, some producing strange error messages. Here is the workflow I used -- bare in mind, my SVN repositories are _very_ simple, contain no branches, no tags, and only I committed to them.

Set Up authors-transform.txt

This file is used to map SVN users to Git committers. The authors-transform.txt file should be plain text, containing lines like:

joe = Joe Smith <joe@smith.com>
jane = Jane Smith <jane@smith.com>

Clone the Subversion Repository

git svn clone <Subversion_URL> --no-metadata -A ~/authors-transform.txt --stdlayout

 Note: the URL should not contain "trunk".

Manual Fixes

The git svn clone creates a bidirectional repository, ie you can commit your changes back to SVN, and also make further updates. This is not what we want here - we want to migrate away from SVN for good.

The fixes I like to do manually:

• create .gitignore file (use svn propget svn:ignore on a checked out svn rep)
• check unhandled stuff in .git/svn/refs/remotes/trunk/unhandled.log -- e.g. keywords ($Id$, etc) are not supported by git
• remove links to the Subversion repository (more might be needed, check your .git directory sturcture):
• rm .git/refs/remotes/trunk
• rm -r .git/svn
• edit .git/config -- remove Subversion specific parts

Push to a Remote Repository

This part is less related to the topic, just for reference. Assuming you have a different host storing repositories:

[on the remote host] mkdir <git_repo_dir>; git init --bare <git_repo_dir>
[on the local host] git remote add origin <user>@<host>:<git_repo_dir>; git push -u origin master

Want a Lean and Fast Linux Desktop?

At work I used to have an AMD desktop with an Athlon 4450B dual-core processor and 3G mem. Not the most recent config, but should not be bad either.

Software being run:

• Ubuntu Unity as desktop
• Firefox
• Thunderbird
• occasionally a terminal

The system was still dead slow, even with regular Firefox restarts (FF being the main memory hog).

The Change

I wanted change, and after trying and reading about LXDE and XFCE, I settled with WindowMaker as the least resource hungry desktop. I remember having used it - conveniently - on a 300MHz Pentium in the 90's (an old Linux intro I wrote in Hungarian is still available online and shows my old desktop). Why not give it a try again? Guess what, it works like a charm, it's fast, easy-to-use and to configure:

The Details

- use Google Chrome instead of Firefox
- PCManFM (part of LXDE)
- install WM
- in ~/GNUstep/Library/WindowMaker/autostart add:

xset m 20/10 4
wmsetbg -u YOUR_BEAUTIFUL_BACKGROUND.png &
xscreensaver -nosplash&
sleep 0.2;
gnome-settings-daemon &

PHP (and WordPress): Command-line 'gettext' Workflow from Scratch

Let's assume you are creating brand new code (for now say a WordPress plugin in PHP) intended for multi-language (internationalized) usage.

Writing the Code

Make sure to use the _() gettext function (WordPress: __(), _e(), and _x() functions) wherever translatable string literals are used, properly specifying the text domain as needed.
When using WordPress, make sure to call load_plugin_textdomain() from the 'init' action callback.
(Note that WordPress does not use the PHP gettext built-in functions, it has its own .mo file reader code in wp-includes/pomo directory.)

Files Involved

• .po (Portable Object) files: ascii files containing translations (strings from the source code paired with translated strings
• .mo (Machine Object) files: binary files containing the same information, can be read faster by programs
• .pot (PO Template) file: contains untranslated strings only; usually completely overwritten after any code change (think of it as a temporary file used for empty .po generation)

The Initial Template

When the initial code is completed, a template ".pot" file needs to be created:

find . -iname \*.php | xargs xgettext --from-code=utf-8 -k__ -k_e -k_n:1,2 -k_x:1,2c -k_ex:1,2c -k_nx:1,2,3c -kesc_attr__: -kesc_attr_e -kesc_attr_x:1,2c -kesc_html__ -kesc_html_e -kesc_html_x:1,2c -k_n_noop -k_nx_noop:1,2c  -ctranslators: -o lang/messages.pot

Starting a new translation

An empty .po file needs to be initialized before we can start translating. Assuming we are translating into German:

cd lang
msginit -l de -o de_DE.po

For some reason, msginit creates a "de.po" file by default.
"locale -a" command to list all the locales installed on your system.
msginit sets encoding to ASCII, needs to be updated.

Translation Memories, Compendia

TODO: read pinfo about compendia & try tmx files.

- create translation memory: po2tmx -l xx -i pofiles -o xx.tmx
- apply it pot2po --tm=xx.tmx -i wordpress.pot -o wordpress_xx.po

Translating the Strings

Work can finally begin! Being a command-line and terminal enthusiast, I use VIm to edit the .po file; it has a great plugin helping in common tasks: po.vim.

Wrapping Up: Creating .mo files

...is simple:

msgfmt de_DE.po -o de_DE.mo

When using WordPress, make sure to prepend the "domain" to the .mo file name, e.g. myplugin-de_DE.mo

When the Code Changes

• Re-create .pot file - see above
• issue msgmerge -U de_DE.mo messages.pot

Further Read

http://codex.wordpress.org/Translating_WordPress
HTML-ized form of "info gettext": http://www.gnu.org/software/gettext/manual/gettext.html
PHP gettext docs: http://php.net/manual/en/book.gettext.php

Go!

I am a programmer. I like helping others. Back in 1999, the Linux Absolute Beginner's HOWTO I wrote (in Hungarian) even got published in GameStar.

There are many problems we find solutions for during the course of our life. It really helps to share the solution when found -- and a blog is the best place to do this.

If there is only one man finding relief here, creating it has already been worth it.