Installing Newer Kernels in Ubuntu

As part of the ongoing TV streaming project i’m working on, I tried a new DVB-T tuner (the August DVB-T210). Whilst it is supported by the Linux TV project, support is only available in kernel version 3.14 and newer.

At the time of writing, Kernel 3.14 is so new that it only received a stable release a week ago. Naturally, no Linux distribution is providing this kernel as a default option yet as it is so new.

But what if you’re dependent on a new bit of hardware support or a new feature in a newer kernel? In the past, you would have needed  to download the new source and compile the new kernel from scratch – not an easy process. Luckily though, Ubuntu provides pre-compiled, packaged kernels which can be easily installed! These are known as mainline kernels, instead of the default stable kernel which is part of the current version of Ubuntu.

These kernels are located at http://kernel.ubuntu.com/~kernel-ppa/mainline/. Then, find the kernel version you wish to install, in this case v3.14. The word after the version denotes which version of Ubuntu this kernel is meant for – at the moment 3.14 is only available for Trusty (Ubuntu 14.04). If you’re not running this version of Ubuntu, you can try installing the kernel, but things will likely break!

Inside the folder for the version you wish to install, you’ll find a multitude of files. First, decide on the variant of kernel you want – in most cases this will be generic. At this point, you should also know what platform you’re running on, most systems now are 64-bit, so you’ll likely want the amd64 platform.

Now, download the following files in to an empty directory;

  • linux-headers-version-variant-version-platform.deb
  • linux-headers-version-version-all.deb
  • linux-image-version-variant-version-platform.deb

For example, if I wanted the 3.14 generic 64-bit kernel, I would have downloaded;

Once downloaded, we need to install the new kernel packages. To do this, use the dpkg -i command. If you’ve downloaded the packages to an empty directory, you can simply run dpkg -i * to install all of them, otherwise you’ll need to specify each file after the dpkg -i command. Installing packages requires root privileges, so prefix the dpkg command with sudo if you’re not already root.

dpkg will now install the kernel for you and update your GRUB configuration so that next time you reboot, the new kernel is loaded instead of your current one. Once the installation is complete, just issue a reboot command and wait for your computer to reboot. While your computer is rebooting, pay careful attention to the GRUB screen, it should show your new kernel in the list of available boot options and it should also be the default.

Once the reboot is completed, check that the new kernel is loaded by running uname -a. This will output the current kernel version. For example, after installing the above packages, uname -a outputs;

Linux hostname 3.14.0-031400-generic #201403310035 SMP Mon Mar 31 04:36:23 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

This is all the confirmation we need that the kernel installation was successful. If the output does not match the kernel version, variant and platform you installed before, then the installation likely went wrong or another kernel version has taken precedence.

Of course, running a non-stable kernel does come with inherent risks. You’re now running a kernel which hasn’t been certified against the packages for that version of Ubuntu and you’re also now responsible for updating the kernel to receive bug and security fixes. Switching to the mainline kernel isn’t a decision that should be taken lightly and you should only do it if there’s a specific feature you need in a newer kernel.

rpi

Streaming Live TV using Tvheadend and XBMC on Raspberry Pis

I recently had an interesting request from some relatives – they wanted to be able to watch TV in the kitchen. A simple task I thought, I grabbed an old analogue TV, digital set-top box and internal aerial. Unfortunately, despite the post-switchover boost in transmission power, the signal received was just slightly too weak to be useful, resulting in only a handful of channels reliably working, not to mention how untidy the jumble of equipment looked.

This lead to a crazy idea – what if a computer attached to the main aerial, which has perfect reception, received live TV and retransmitted it over the internal network to be received by another computer attached to the TV? It was crazy enough that it might just work.

Since the launch of the Raspberry Pi almost 2 years ago, it has found its way in to a number of scenarios that were never anticipated. It’s been used for everything from controlling advertising boards, small micro-servers, distributed computing and, most importantly for this project, media centres.

Whilst the processing power of the Raspberry Pi is low, the GPU is able to fully decode H.264 video at up to 40Mbit/sec – no easy feat. Due to this, the Raspberry Pi can play back most media flawlessly. The attractive price point ($35), small size, low power requirements and video processing power make it particularly suited to this project.

Software Selection

I then needed to identify two bits of software that would run on the Raspberry Pi – one on the back end to receive and re-transmit the TV signal and one on the front end to receive and display the re-transmitted signal. For receiving, the long-running XBMC project makes for a fantastic media centre and, most importantly for this project, supports reception from a wide range of PVR hosts. Thus, selecting XBMC for the front end display wouldn’t affect my choice of back end. As XBMC is just a Linux application, I could theoretically choose any distribution to run it on. However, I instead chose to go with OpenELEC - a stripped down distribution that’s designed only to run XBMC. It works like an appliance, and so handles updates both to itself and the underlying Linux system automatically. A downside for tinkers, but an upside for simplicity. As this system was for a relative, simplicity is key – it reduces the number of calls I get about things not working.

On the back end, there are a number of potential options, however only a few offer support for the Raspberry Pi. These are Tvheadend and VDR. I went with Tvheadend as the setup looked relatively simple. There’s no appliance version of Tvheadend as there was with XBMC and OpenELEC, so I opted to run it on Raspbian, the ‘official’ distribution of Linux for the Raspberry Pi.

Hardware Architecture

Next up, hardware. I had decided that I was going to run this project using Raspberry Pis, but other hardware was also required. A wired connection was available next to an aerial point for the back end Raspberry Pi, but the front end Raspberry Pi (in the kitchen) wasn’t anywhere near a wired network connection, and running a new cable wasn’t an option. Instead, I used a spare Edimax EW-7811Un USB adaptor to add WiFi capabilities to the front end Raspberry Pi. With WiFi devices on the Raspberry Pi, you’re recommended to run them through a powered USB hub due to their power draw requirements, but I found that it worked well connected to just the on board USB ports. There are no other USB peripherals connected (e.g. Keyboard/Mouse) so this reduced the overall power requirements.

As for receiving the signal, I had an old Freecom DVB-T USB stick, which has Linux support. Unfortunately, the power draw of this was too much for the Raspberry Pi and so I had to run it through a powered hub. Being old, it also does not support the newer DVB-T2 standard, which is used for Freeview HD transmissions. This isn’t a major issue however considering that the front end is an analogue TV and so can’t display HD anyway!

Setup

Front End Installation

Next up was setting up everything. OpenELEC was relatively painless, I just needed to write the provided image to an SD card and then go through the initial setup. Easy!

Back End Installation

Tvheadend was more involved however. After writing the Raspbian image to the SD card and going through the initial setup, I needed to add the repository for Tvheadend and install it. Once installed it runs automatically as a background service. Tvheadend is administered entirely through a web interface which runs on port 9981.  To get my channels into Tvheadend I had to do the following;

  1. Browse to Configuration > DVB Inputs > TV Adapters. Select the connected adapter from the list. My Freecom DVB-T USB stick appeared as “Wideview USB DVB-T”.
  2. Configure the adapter. Due to compatibility issues, I had to make sure that ”Close device handle when idle” was checked and “Disable PMT monitoring” was unchecked. If I didn’t do this, I would quickly hit the file handle limit for the USB stick and need to reboot to use it again. Once you’ve configured the right settings for your adapter click Save.
  3. Browse to the Multiplexes tab. You can click “Add DVB Network by location” on the General tab to automatically add the Multiplexes for your area however I found that they were out of date.
  4. Add the multiplexes for your area one by one by clicking “Add mux(es) manually”. For Freeview in the UK, you can find the values you need to enter at ukfree.tv and entering your postcode to find your nearest Freeview transmitter. Enter the frequency, bandwidth (normally 8Mhz), constellation and transmission mode (normally 8k), leave all other values at Auto. Click add to finish adding the multiplex. Repeat for as many multiplexes as you wish to add. When you add a multiplex, Tvheadend will automatically scan the multiplex for available channels.
  5. Browse to the General tab. On the information and capabilities section, wait for the “Muxes awaiting initial scan” count to drop to 0. Now, click “Map DVB services to channels” and Tvheadend will map the discovered channels.
  6. Done! Tvheadend is now advertising the channels.

Connecting the Front End to the Back End

Now that both the Front End and the Back End are configured and hopefully functional, we can tell the Front End (XBMC) to use the Back End (Tvheadend) to receive TV. To configure XBMC;

  1. Browse to System > Settings > Live TV > General.
  2. Click “Enabled” to Enable Live TV reception.
  3. You will be warned that there is currently no configured add on for Live TV. Click OK and you will be taken to the list of available Live TV back ends.
  4. Find “Tvheadend HTSP Client” and select Configure. Enter the IP address or hostname of your back end and then select OK.
  5. Select Enable. XBMC will now try and connect to the Tvheadend back end and retrieve channel listings etc.

Testing

At this point, you should be able to receive Live TV on your XBMC front end. Browse to Live TV > TV channels and select a channel to start watching. If you have no channels listed or can’t view any channels, check on Tvheadend that you’ve performed the mapping and check the debug log by clicking the little arrow on the bottom right of the web interface.

However, at this point, you may find that while you have audio, you don’t have any video. Earlier, it was noted that part of the power of the Raspberry Pi is that it has a built in GPU that can decode video, doing tasks which would be impossible on the weak CPU.  However, as a cost-saving measure to help the Raspberry Pi meet its $35 price point, not all of the video formats that the GPU can decode are enabled. The majority of video content today is encoded in MPEG4 H.264, which the Raspberry Pi’s GPU can decode by default. However, UK Freeview DVB-T transmissions use the older MPEG2 format – simply because MPEG4 didn’t exist when Freeview originally launched. Unfortunately, while the Raspberry Pi’s GPU is able to decode MPEG2, it is one of the codecs which was not enabled by default in order to save costs. Thankfully, you can buy a licence key to enable it for the cheap price of £2.40 from the Raspberry Pi web store. Buy it, await your key, and then install it on your Raspberry Pi. It is very important that you give the serial number of the front end Raspberry Pi – that’s the device that does the decoding and display! The back end Raspberry Pi just receives the MPEG2 stream from the aerial and re-transmits it over the network – it does not alter the stream.

Control

Once you’ve got this all setup, you’ll probably wonder how you’re going to control it. Having a keyboard and mouse attached to the front end probably isn’t practical or convenient for control. You could likely buy an Infrared receiver to plug in to the Raspberry Pi and use a remote. However, since XBMC is connected to your network, why not control it from another device that’s also on your network, say a phone or tablet.

Indeed, XBMC supports remote control by other devices. A web interface is available, or even better an API. This can be used by other applications to control XBMC and receive information.

I use Yatse on my Android devices to control XBMC, solutions for other devices also likely exist but I wouldn’t know about them as i’m purely an Android user. The great thing about Yatse is that it can act as a normal remote (e.g. going left, right. up, down etc in menu interfaces) or it can also be used as a “second screen” to select content which will immediately start playing back on XBMC – easier than navigating through several menus. For Live TV, just tap the PVR button then tap the channel you want to watch. It’s that simple.

Performance Tweaks

The Raspberry Pi is just slightly too underpowered to run the XBMC interface smoothly (but as video playing is done by the GPU, there’s no issues there). If the lack of a smooth interface annoys you though, you can either make tweaks to the XBMC interface, such as disabling the RSS feed, or you can overclock your Raspberry Pi. Overclocking runs your Raspberry Pi past its rated performance and may cause instability. Overclocking will not void the warranty on your Raspberry Pi, so long as you overclock it in a way which is endorsed by the Raspberry Pi foundation.

I run the the front end Raspberry Pi using the pre-set Turbo setting.  The supported method of overclocking is using the raspi-config utility, but this isn’t present on OpenELEC. Instead, if you want to overclock you’ll need to edit some files as described in this forum thread.  It is very important that you stick to the pre-set values (Modest, Medium, High, Turbo) and do not set force_turbo=1 as this will definitely invalidate your warranty. Without force_tubro=1, the Raspberry Pi will automatically overclock itself subject to the requirements of the current workload (i.e. it won’t overclock if the extra processing power isn’t needed) and the current temperature of the processor (it won’t be overclocked if it is 85 centigrade or higher, but you’re unlikely to hit this. Mine in a case runs at around 65 centigrade with a Turbo overclock and full load.

Price List

  • 2x Raspberry Pis (Model B, 512MB RAM): £58.14 via Amazon.
  • 2x Cyntech Case for Raspberry Pi: £11.50 via Amazon.
  • 2x 8GB SanDisk Ultra Micro SD Card with SD Adapter: £17.70 via Amazon.
  • 1x Freecom DVB-T USB Stick: Discontinued. You may wish to buy the PCTV 290e instead, with the added bonus of supporting Freeview HD on DVB-T2. However, ensure you don’t buy the newer 292e as it is not compatible with Linux yet.
  • Edimax EW-7811Un USB WiFi Adapter: £6.78 via Amazon.

Total cost: £94.12, excluding DVB-T receiver. For me, the total cost is lower than this as I already owned some of these components. Still, not a bad price at all, and certainly much cheaper than having a new aerial feed run to the kitchen.

Summary

The system that i’ve built works quite well. Setup wasn’t exactly an easy process, but once it’s working it is very stable. This certainly made for a fun weekend project and the reception from my relatives has been positive, they have no issues using Yatse as a remote control from their phones either.

As for the next steps on this project, i’m looking in to the DVR and Timeshifting options offered by Tvheadend as well as buying a more recent Freeview receiver, such as the previously mentioned PCTV 290e in order to also receive HD Freeview transmissions.

Resetting the IPMI Password on the ASRock E3C224D2I

Note: This will likely work for other similar ASRock boards too (e.g. E3C226D2I), however as I’ve only got the E3C224D2I I cannot verify if it does or not. Due to the method, it may even work on other boards too. Proceed at your own risk!

I recently bought the ASRock E3C224D2I motherboard for a new Home NAS build. It has an integrated IPMI controller, so regardless of the state of the system it is possible to connect over the network to view the video output and use the keyboard and mouse. You can even mount images and directories from your computer making it possible to carry out an installation fully remotely. Perfect for troubleshooting when things go really wrong!

Unfortunately, after experimenting with the firmware upgrade option on the IPMI controller I locked myself out. No passwords would work, not the default admin/admin login nor admin and the password I had set before. I couldn’t find any way to reset the password either – the IPMI’s password reset interface required a working SMTP server to be configured, which I hadn’t done. Additionally, there seemed to be no option in the BIOS to reset the password. I really didn’t want to lose access to the IPMI as it’s one of the main reasons I chose this board.

After much Googling, I came across a forum post where someone had the same problem. A reply from an ASRock Rack representative said to contact them for a tool that would reset the password. I did so, but while waiting I thought it was worth trying another mechanism.

Supermicro, another motherboard vendor that often features IPMI on their motherboards, provides a download for ipmicfg. This DOS tool is intended for performing operations on the built-in IPMI chip without having to go through the IPMI interface – perfect for password resets. Despite being from another vendor, I thought I’d give it a go, and what do you know, it works!

So, to reset your password using ipmicfg;

  1. Prepare a bootable DOS USB stick via your preferred means. I used rmprepusb to create a bootable FreeDOS USB stick.
  2. Download and place the ipmicfg files on to your newly created USB stick.
  3. Boot from your USB stick on the machine whose IPMI password you are trying to reset.
  4. Run ipmicfg -m to verify communication with the IPMI chip is working. If the command succeeds, you should see the IP address and MAC address of the IPMI displayed.
  5. Run ipmicfg -user list. This will display a list of users that can log in to the IPMI. Note down the User ID for the account whose password you wish to reset.
  6. Run ipmicfg -user setpw userid password, replacing userid with the User ID you found with the previous command and password with the new password you wish to set.
  7. Done! Try logging in to the IPMI again with your new passwords. If things still don’t work, try running ipmicfg -fd to reset the IPMI to its factory defaults.

As to why this worked with a non-Supermicro board, I theorise that it’s because the IPMI chip on the ASRock board complies the IPMI standards. Therefore, any tool which is compliant to the standards, such as the Supermicro ipmicfg tool, should be able to interact with the chip.

It’s also worth noting that ASRock Rack did get back to me about a day later with a tool to reset the password, but I didn’t use it having found a way to do it with ipmicfg. If you don’t feel like trying out a method using another vendor’s tool, contact them and wait for their reply.

Good luck!

No Text Displayed on Windows 7

I often end up doing various bits of technical support for my relatives. Recently however, I came across an interesting issue – a relative said that their Windows 7 laptop was displaying no text whatsoever – no text on the desktop icons, no text in most applications and if you clicked the start button, nothing would appear.

Connecting remotely, I confirmed the issue. Of course, diagnosing the issue was hampered by the fact that I couldn’t read any event log messages or such. Anything that used the System Default font was simply not rendered.

Reading around, this is sometimes caused by a corrupt font, and is easily fixable. Booting into Safe Mode, I was at least able to get a working command prompt up and ran sfc /scannow. This command checks the integrity of all core Windows files and repairs any that are corrupt or missing.

After scanning, the command confirmed that some files were found to be corrupt and had been successfully repaired. After rebooting back into the normal Windows environment, all text was rendered as before. Problem solved!

The sfc /scannow command does generate a log at C:\Windows\Logs\CBS\CBS.log which I took for further investigation.  Reviewing later, this log did confirm that several font related files such as segoeui.ttf, seguisym.ttf and segoeuib.ttf did not match known-good ones and were replaced.

So if you ever come across this problem yourself;

  1. Reboot in to Safe Mode.
  2. Open a Command Prompt and run the command sfc /scannow.
  3. Wait for the command to complete – depending on the speed of your system drive this could take tens of minutes.
  4. Review the output of the command to see if any files were found to be corrupt or missing and replaced.
  5. Reboot.

IPv6 with Online.net and Dibbler

After getting a bit fed up with OVH, I recently moved my personal server to Online.net. Like OVH, they are a French-based provider of low price, but high performance servers – great for my needs of a powerful box for all my experiments and personal web, voice etc hosting.

Unfortunately, getting IPv6 working with Online.net is somewhat more involved than with OVH. With OVH, you just enter in the static IPv6 addressing you’ve been assigned and you’re done. In comparison, Online.net requires that you use a DHCPv6 client to request an address before it will permit traffic to flow. If you just configure your interface with the IPv6 addressing you’ve been assigned, you won’t get anywhere. In addition, there is no default gateway to define, the link-local IPv6 gateway is used. If this is a good design architecture, I can’t comment, but it certainly makes the setup a bit more involved.

My server runs Windows Server 2012 R2 (running plenty of Linux-based HyperV VMs however), so this guide will focus more on the setup of Dibbler under Windows. However, considering how sparse Online.net’s documentation is, Linux users may also find this useful. The documentation Online.net provides for using Dibbler can be found here, albeit in French only.

Windows does support DHCPv6, but the setup process for it is also somewhat convoluted, hence the suggested Online.net DHCPv6 client was used – Dibbler, which can be downloaded from SourceForge.

Before installing and configuring Dibbler, you will need to configure your IPv6 address on Windows. In Network Connections, double-click on your Internet-connected Ethernet adapter, click Properties and then double-click on Internet Protocol Version 6 (TCP/IPv6). In the IPv6 address box, enter your assigned IPv6 address, followed by ::1. E.g, if you have been assigned 2001:bc8:8234, enter 2001:bc8:8234::1. In the subnet prefix length, enter 48 (if you have delegated your prefix, you will need to enter different values for the address and prefix length, but you should know what these are if you are delegating). Counterintuitively, leave the Default gateway box blank. Click OK twice. The default gateway is left blank because as mentioned earlier, Online.net’s setup uses the link-local default gateway rather than one that can be routed to.

Now install Dibbler, once installed you will need to configure it. Browse to the installation directory (the default is C:\Program Files (x86)\dibbler) and edit the client.conf file. My configuration is as below;

log-level 8
 iface "vEthernet (HP Ethernet 1Gb 2-port 332i Adapter - Virtual Switch)" {
 pd
 option dns-server
 option domain
 }

The log-level statement defines how verbose Dibbler is in the log. I’ve left it at the default of 8 as this appears to be verbose enough. The iface must be the full name of the interface as it appears in Network Connections – if you’re not using HyperV like me then it is likely just “Ethernet” or similar. Make sure you put the right interface here as otherwise it won’t work. Most Online.net servers come with two network connections – the main internet connection and the private network (RPN), so check the address configuration on the interface to be sure. pd (prefix delegation) tells Dibbler to ask for a prefix (address). The option dns-server and option domain statements tell Dibbler to ask for DNS information.

Next, edit the client-duid file. In here, only place your assigned DUID which can be found on the IPv6 page of your server’s configuration on the Online.net console. The DUID is a long hexadecimal number seperated by colons that is a unique identifier for your server. Dibbler sends this out when asking for address information so that Online.net’s DHCPv6 servers assign the correct address.

Now Dibbler should be configured and ready to go. Open up a Command Line prompt and navigate to the Dibbler directory (hint: Shift-Right Click in the directory in Windows Explorer and select the Open command window here option). Next, run the command dibbler-client.exe run. This runs Dibbler in a console so that you can see the log messages, rather than in the background. You should see Dibbler sending out a request and then receiving the response. Don’t worry that it assigns the prefix to the wrong interface, since we’ve already configured our interface with our IPv6 address Windows knows which prefix to use.

Now that Dibbler has requested and received a prefix, IPv6 connectivity should be working. Verify this by using an IPv6 tester, such as test-ipv6.com. If everything is configured correctly, then you should pass all the tests.

Of course, running Dibbler in console mode is less than ideal – it means that you need to remained logged on to your Windows server and make sure that you run Dibbler when restarting your server. Ideally, you want to run Dibbler as a service and it does support this. Unfortunately, i’ve yet to be able to get it to do this, as running dibber-client.exe start results in the error “Service DHCPv6Client startup failed”. I’m going to keep having a look at it however and will update this post when I do get it working successfully in service mode.