Neural Networks and Deep Learning

A little while ago i started reading the free online book “Neural Networks and Deep Learning” by Michael Nielsen.  If you’re looking for an introduction to the subject then this is a really great place to start.

In the first two chapters Michael explains how to use stochastic gradient descent to train a feed forward neural network to recognise handwritten digits. The highlight for me is the solid explanation of the theoretical side, and the in situ application of this to a real world problem. The result is 74 lines of Python code capable of recognising  handwritten digits with an error rate of less than 4%.

As an exercise I decided to re-implement Michael’s Python example in C99, using the GNU Scientific Library (GSL)  for matrix / vector operations and Catch for some (limited) unit tests.

To allow direct comparison with the Python example i’ve set it up to read in the first 50,000 numbers from the MNIST database of handwritten digits, and use the final 10,000 to test the network on each epoch, just like in the example.

As you’d expect the C port behaves just like the Python example, with some minor variation likely due to differences in the generation of random numbers and floating point inaccuracies. Perhaps the one nice thing is that despite neither version being optimised for speed, the C implementation runs about 15 times faster on my machine, making it less time consuming to play with the various parameters.

Finally a big thank you to Michael for taking the time to write this book, and for making it freely available.

The code for the C port can be found here.

Simulating the SSD1306 OLED display in simavr

As part of a larger project I’ve started building some virtual parts for simavr. It’s turned out to be a real pleasure to use and i’d highly recommend it. Great job BusError!

Here you see an identical firmware running on both the physical device and in simavr. Aside from the fact that simavr runs about 10 times faster than the actual hardware the results are identical. The demo goes through the routine of testing the driver functionality. This includes things like setting the contrast, inverting the display, writing to the buffer and changing the orientation.

Currently the virtual part has only been tested with this driver. If you’re using some of the more obscure hardware functionality you may need to extend it. The same goes if you’re using the parallel or I2C interfaces.

Mapping speed, resistance and power on an ergotrainer and calibrating the Tacx Bushido

For any turbo trainer a relationship exists between the wheel speed and the resistance of the brake. For a standard turbo trainer this relationship is typically a somewhat arbitrary approximation to the resistance experienced when cycling outdoors. However, for a cycling simulator, the relationship should attempt to closely mimic the behaviour observed out on the road. Clearly the extent to which the actual behaviour is modelled determines the quality of the simulation.

In a standard turbo trainer there is no automated feedback between the speed of the wheel and the resistance of the brake. Ignoring any manual resistance controls, a fixed relationship exists between wheel speed and brake resistance, whilst the inertia of the cyclist and bike is often simulated with a flywheel. The key difference between this and an ergo trainer is that in an ergo trainer there exists a feedback loop between brake resistance and wheel speed. The feedback loop can be used to match the brake resistance to that expected under specific conditions. Essentially this is achieved by matching the total power expended by the brake with the estimated required power for cycling under the simulated conditions.

In Cyclismo we calculate power using the model derived by Martin et al. which has been validated against real world tests. This model takes into account the following:

  • Speed
  • Gradient
  • Rider and bike mass
  • Drag area
  • Local wind speed and direction
  • Yaw angle
  • Bearing friction
  • Rolling resistance

These quantities are then used to estimate the required cycling power, whilst inertia can be simulating by keeping track of the system energy and clamping the speed at a level suitable for the present reserve. We note that whilst some terms such as bearing friction are negligible, the model has been implemented in its entirety for completeness. In the future we will add corrections for the brake and flywheel inertia.

Mapping speed, power and brake resistance

To implement the power model described in the preceding section it is necessary to know how the power produced as function of wheel speed varies with the brake resistance. By obtaining a calibration map linking these three quantities one can adjust the brake resistance according to the current cycling conditions.

There are various ways of ensuring that cycling power and speed are consistent, however we consider it advantageous to calculate a ‘virtual’ cycling speed from the actual power produced by the rider. This way one can easily utilise power measurements from external devices such as hub or crank based sensors.

Once the virtual speed has been computed, an appropriate brake resistance can be calculated to force the wheel to the virtual speed This is achieved using the calibration map, and effectively forces the actual wheel speed to be equivalent to the predicted wheel speed for the current cycling conditions.

Calibrating the Tacx Bushido

We performed a trial calibration by recording power output as a function of actual wheels speed at fixed brake resistances. The results are shown in the figure below.


In the top figure we plot the raw calibration data which has been measured for fixed brake resistance intervals of 50, ranging from -300 to 1000. We note that the Bushido has a dead zone below a resistance of about 250 where any further reductions in resistance have little or no effect. Resistances above 1000 were not tested as the wheel starts slipping on the brake even when pedalling smoothly with a low gear and high wheel-brake compression. Above resistances of ~250 we observe steady increases in the amount of power produced for a fixed wheel speed, and that power is approximately a linear function of wheel speed below about ~70km/h.

Taking these observations into consideration we fit a surface to the calibration data by first fitting linear regression lines to each fixed resistance series and extrapolating within reasonable bounds. This helps to ensure a reasonable surface fit at powers in excess of what we were able to produce during calibration.

The surface fit is shown in the background of the top figure where it is compared to the observations, and in 3D in the bottom figure. In both cases it has been bounded such that the brake resistance is kept between 250 and 1000 which in our opinion are reasonable limits.

We have implemented the speed, power, resistance map described by the bounded surface fit in Cyclismo alongside two other methods; one which utilises a PID controller, and another which is based on the OEM brake speed map. We hope that the former two models will help to create a more realistic cycling experience and have the additional advantage that the Bushido head unit is no longer required!

The fitting tool used to generate these plots and the raw calibration data is available here.

Bushido news


The Big D and I have been working hard at coding an android app with Tacx Bushido support. The current state is that we have a working app (albeit with a few rough edges) that can simulate GPX courses. So it is with great pleasure that we release a preview (source code now up):

You will need:
– A Tacx Bushido Turbo Trainer and Headunit
– The app:
– ANT+ Radio Service:
– ANT+ USB Service and a USB host OTG cable (if you want to use a usb ant+ stick and your device isn’t listed on the ANT+ Radio Service page) :

It is worth pointing out that you can run this in VirtualBox on windows/max/linux using android x86 ( I use android-x86-4.0-RC2-eeepc.iso for testing.

You will need to enable:

– Settings -> Location services -> allow GPS satellites
– Settings -> Developer options -> Allow mock locations (this was the easiest way of plotting your position on the google map)

– Upload a gpx you want to simulate some where on your android device
– Start the app
– Press Record
– Choose simulation mode
– Choose the Course (GPX import will appear)
– Select the GPX you uploaded earlier (takes you back to course setup screen)
– Turn on the bushido headunit
– Press Go
– After a few seconds ‘start cycling’ will appear on the headunit display
– Start cycling!


– Connects to the Bushido Head unit only
– Exceptions not really handled at the moment (this means the app will force close (‘stop responding’). This will happen if you don’t have the required packages (see above) or some other incompatability.
– Only tested on Android 4.03 (so your milage may vary on earlier versions)

– Shows power,cadence, speed and heart rate (in the chart view)
– Route is shown on a google map
– Export ride as tcx (to process in other bits of software)

Feature requests welcome.

In other news, BerryS has reversed a few more nuggets of the Bushido Headunit protocol (see the wiki). We now know about long/medium button presses, the device serial and the user’s weight. Cheers BerryS!



Accelerometer controlled RGB Lamp

 This is an RGB Lamp assembly that I constructed based around the following parts:

The idea is that the accelerometer is used to calculate the lamp position in spherical polar coordinates, which are then used to control the colour of the lamp. The LED is driven by 3 switching converters coupled to the  PWM outputs of the ATmega8 through opto-isolators.

Anyway, enough details, here’s a video of the lamp in action:


And for those interested in the circuitry (picture taken prior to adding optos):

The blue board on right is the ADXL345 break out board, which also features a selectable 3.3V regulator and TWI pull up resistors. In this case the regulator is used because the ATmega8 I had lying around requires a minimum of 4.5V.  Fortunately the SDA and SCL lines of the TWI bus are open drain so there’s no need to worry about level conversion in this case. I was actually surprised at just how good the ADXL345 is. Not only does it have some cool stuff like tap detection, with a little bit of software filtering I can comfortably get an angular resolution of ~0.25 degrees after the conversion to spherical polars.

Also seen on the above picture are three little green boards which are the LED drivers, hooked up directly to the DC input in the bottom left hand corner which is 16V.  Originally I built my own switching converter, but it actually worked out cheaper (and faster!) to modify modules from DX by removing the rectifier and soldering a wire directly to PT4115 dimming pin for brightness control. Obviously, it’s a good idea to use an opto-isolater between the dimming pin and the MCU, not only in case the PT4115 fails, but because the dim pin registers ~5.3V and is about 1mm from the 16V supply!

Other things that you see are an ISP header in the lower middle, 16MHz crystal in the centre, an LED in the top left, and switch connection / UART headers in the top right. The UART header was used for debugging with a CP2102 USB to UART bridge. These are awesome little modules which are well worth investing in, especially if you don’t have a debugger.

The code for the project is now available here:

Comparison of Tacx Bushido vs Powertap SL+

Here’s one for cyclists: A quick comparison of the power measurement from a Tacx Bushido and a PowerTap SL+ hub. The data was acquired over a 10 mile time trial simulated by Tacx Trainer software 2. Data from the PowerTap was recorded with a Garmin 705.

The procedure:

  • Calibrated Bushido with the coast down test. Obtained 10.9
  • 5 minute warm up at ~250 Watts
  • Calibrated Bushido again. Obtained 10.2
  • Calibrated PowerTap via the Garmin 705
  • Road the 10 mile TT (took 20 minutes 49 seconds)

I then wrote a quick Python script to plot the data exported from Tacx Trainer and tcx file from the Garmin. Here are the results:

Some observations:

  • Cadence which is estimated from variations in torque by both devices is only an approximation to the real cadence.
  • Heart rate which in both cases is measured by a Garmin Ant+ monitor is (almost!) the same as would be expected.
  • Average power output from the Bushido is recorded ~6% higher at 372W than the PowerTap which averages 352W.
  • The power output from the Bushido drifts upwards with time, whilst the output from the PowerTap does not. However the output from the PowerTap was used for pacing so it is impossible to determine which one is drifting.
  • The power output from the Bushido is less noisy than that from the PowerTap.
  • Tacx Trainer 2 is easily the worst piece of software i have ever used.
  • The Bushido on the other hand is well engineered, that is apart from the screen on the head unit which is terrible.

In conclusion I was very pleased to see a good agreement between both devices. If the correction factor on the Bushido was adjusted, it could be made to agree closely with the PowerTap. However, most importantly the Bushido measures consistently and is therefore a very useful training device on its own.

I’ll upload the script i wrote when i get a chance.

Building Angstrom for the mini2440 with Fedora 15

Here’s a quick update for anyone struggling with building Angstrom for the mini2440 on Fedora 15 (probably also relevant for F14).

If you’re using BusError’s original repo then follow the OE instructions as normal, then you can use this guide if you want. As mentioned you’ll need to downgrade patch. I also had to downgrade make due to some incompatibility.

yum downgrade patch --releasever=12

yum downgrade make --releasever=13

At first i had some error about the GPG key so i downloaded them for F13 and F12 from here. You can install them like this, for example:

rpm --import E8E40FDE.txt

Then, you’re good to bake.

If you want something bleeding edge you’ll need to get your hands dirty, OE has changed a lot recently!

CAT3626 RGB LED Driver + Qt Embedded 4.6 on the Mini2440

The CAT3626 is a nice little I2C device from ON Semiconductor. It comes in a 14 pin TQFN package which is probably the most difficult thing i’ve ever soldered! Each pad is separated from it’s neighbour by about 100 micrometers! To make matters even more difficult there’s a ground pad right in the middle for sinking heat away. Really you should get some sort of breakout board or reflow it but that would be boring! Here’s a picture of the device before soldering:

…and here’s the finished product! The LED is 10mm diffuse with a common anode and a maximum current rating of 20mA per channel.  As soon as i’d soldered the wires to the CAT3626 i glued it to the board with epoxy, threaded the wires through the holes and then glued those down with more epoxy. After that i realised i’d forgotten where pin 1 had gone so i had to scrape the glue off the top of the chip!

The wire coming through the hole in the middle of this picture is soldered to the CAT3626 and acts as a crude heat sink. In practice it seems to work well enough although i probably should have used copper. I pulled the ceramics from an old projector i had lying around. Not the neatest job but it works fine!

After I’d built that i tested it quickly using I2C tools and then wrote a kernel driver.  The CAT3626 allows a maximum of 32mA per channel regulated in steps of 0.5mA. Unfortunately because my LED can take a maximum of 20mA per channel i can only generate ~64,000 colours. To make matters worse the light intensity is not in general a linear function of the input current so the achievable fade sequences are not as nice as those done with PWM but they still look pretty good. Here’s a video!


Update: The kernel driver for the CAT3626 is now available for free on github!

Chameleon 135 for the Mini2440

Thanks to both Will and I now have the Chameleon 135 enclosure for the Mini2440.  I’ve been working on a li-ion charger and once that’s finished we should be ready for some field testing. It takes a little while to get the holes drilled in the right place but once it’s screwed together the case is extremely sturdy. There’s loads of room inside for additional wiring 🙂

You can get the Chameleon from

New hardware!

After breaking the z-axis connection on my first device I ordered a new KXPS5 accelerometer from crodnet on ebay. I really recommend this seller, he’s very cheap and sends things rapidly.

I quickly knocked together a circuit on veroboard and hooked up the base of a transistor to the freefall / motion interrupt pin to check it was working. It is remarkably sensitive, the LED flicks on if i drop my pen at one end of my desk when the device is at the other! I was quite pleased that everything was working well so i hooked it up to the I2C bus using the cable kit kindly supplied by Using I2C tools i was able to set registers and read acceleration vectors without any trouble so i’m pretty sure everything is working well. I was a bit worried about exceeding the bus capacitance at first.



It turns out that i can still use the old device if i rely on the internal low pass filter. I was thinking of doing this anyway and using a Kalman filter to process the output. First i need to write a device driver so i’ve rapidly being trying to learn C. So far i’ve got a very basic kernel module up and running on the mini2440 🙂