– This project is now resuming after a record heat wave for the summer of 2018 –
I received my call sign OH2BNF on April 19th 2018, having passed General Class examination at the end of March in Helsinki. I am a member of SRAL (Suomen Radioamatööriliitto ry) – The Finnish Radio Amateurs League, and of course the OH2AAV Vantaan Radioamatöörit ry – The Radio Amateurs League of Vantaa, and finally Suomen DX-liitto. Links to each are provided immediately below
In addition to the ICOM IC-7200 transceiver, various hand held devices and a Tecsun S-2000 receiver, I have numerous SDR devices and a remote QTH project involving 40 Raspberry Pi 3 systems. In fact this system will be the main content for this page as the project moves forward.
Bitscope in Australia sells a 19″ rack enclosure allowing for installation of 40 Raspberry Pi 3 / 3+ single board computers, via Quattro Pi boards (10 of them) in a setup that supports up to 160 USB devices and most important, sufficient and simple power management for them.
In the rack I have a 48V/25A power supply made by Suomen Muuntolaite Oy (thanks to Bror OH2BNN). Those 40 Raspberry Pi’s only require about 2A when PXE-booted and idle. Same power supply is hence used also for various other devices and converted voltages.
As the x86_64 hardware I use a HPE Proliant DL20 Gen9. Hypervisor is ESXi 6.5, virtual machines include Centos 7 (OpenNMS and Ganglia, likely also SLURM for cluster/grid job scheduling), Raspbian Pixel for x86_64 as the PXE server and cluster central database server, and a Windows 10 Pro for those few ham radio applications only available on Windows platform.
Raspberry Pi boards run a fairly modified version of the Raspbian. While basic features are as delivered, I have had to install quite a bit of additional software into the image. This deals primarily with the cluster type of applications, which range from frameworks such as SLURM to specific items such as MPI libraries. Raspbian being Debian based, this has been trivial to implement.
Network switch is an old 3Com 48-port 1Gb solution which I repaired by replacing 2 fans that had worn out. Above picture shows the rack during an early setup phase. Remote power management is handled with a Ratio ELECTRIC IP Power Switch, which I acquired at a paltry sum of 20 euro’s from a surplus sale. This was quite a deal, as those used to sell for several hundred euro’s.
Rack itself is a 21RU unit from Thomann. It was originally intended for audio devices, was quite cheap and offers a proper cabinet for the system. The height of the rack is about 1.5 meters and depth is less than ordinary IT racks. This was intentional but has posed a few limiting factors into choosing equipment for the solution. Most rack equipment such as servers tend to have a much larger installed depth than for example the Proliant DL20 Gen9 I specifically chose for the purpose.
There are basically two phases in the buildup. First, the whole setup must be made to function as a HPC (High Performance Computing) type of a cluster, and with full remote management capabilities for every aspect of the system.
Final phase is to convert the system into ham radio specific tasks.
I anticipate 20, 30… or so SDR receivers, some of them operating individually – others as a coherent array, using a common external oscillator. There may also be SDR transceivers in addition to a “real” radio such as the ICOM IC-7200. There are also numerous other devices such as antenna switches, NATO qualified Stridsberg antenna multiplicators, upconverters, LNA’s etc.
At the end of the final phase a proper remote QTH will be selected, and system located there. As I live in a high RFI environment with a particularly challenging situation (apartment building with no permission for sufficient antennas) the remote operation is really the only chance. Also I like to incorporate my 20 years of professional Unix/Linux systems administration into the project.
Primary objectives are to incorporate automated adaptivity to the system at large – for example leveraging on band condition information, WSPR (Weak Signal Propagation Report) & friends, automated signal detection and decoding, great flexibility in terms of individual cluster nodes being able to fast respond to various needs and tasks, strong emphasis in parallel processing where applicable depending on the problem type and dataset, support for multiple end users benefiting from the computing and reception capacity of the cluster – to name the most significant.
Question One: “Why?”
Because it is a lot of fun and I like to merge Unix/Linux with a new hobby, ham radio. I also quickly noticed that with all the various digital and other “modes”, not to mention frequency bands and other variables, there seemed to be so much to adjust, tune, configure when moving from one reception type or band to another. I thought that perhaps I might look into automating most of it…
Question Two: “Why Raspberries – aren’t they weak in terms of computing power?”
I am 48 years old with 20 years in Unix/Linux systems administration, including HPC computing sites such as Nokia Research Center and current employer Qualcomm. I have learned a long time ago that for any given task, sufficient performance is needed. The Raspberry Pi is a small wonderful device that is right at home with moderately heavy computing tasks, if used appropriately.
Primary reason for having not just few but 40 of these deals with USB I/O, which I soon observed was not that great even with a fairly powerful desktop computer. Try running 4, 8 or more SDR receivers at full speed, and you’ll quickly see what I mean. Hence as we say in Finnish “lisää rautaa rajalle”, that is – I now have capacity for up to 160 USB devices which should be more than enough.
A single Raspberry will do fine with 1 or at most 2 relatively narrow bandwidth SDR receivers such as the famous RTL-SDR Blog v3 (which is the type I am primarily using). Beyond that the I/Q data eats up more networking bandwidth than is available. So just put more of them pesky Raspberries, right?
Question Three: “Are there similar setups around the world?”
Sure, many of them. I haven’t specifically seen a Bitscope enclosure with 40 Raspberries dedicated for ham radio activities, but the Raspberry and ham radio software are widely used together. There are many hams who deserve credits for inspiring this project – I have spent endless hours of reading other people’s blogs.
Someone I do want to mention specifically is Oona Räisänen (OH2EIQ). Go and check out her inspiring blog “Absorptions” at http://windytan.com – that blog got me interested into all this. Be prepared to spend hours reading…
Question Four: “Why so many Raspberries and why not a broad bandwidth XYZ receiver?”
The bandwidth available from devices other than RTL-SDR v3 is not at all the issue. In fact this project has dimensions and objectives that extend far beyond just the bandwidth question; and perhaps it needs to be clarified that the intention is specifically to use 40 Raspberry devices, not to limit the number of them.
As stated we are still in the first – infrastructural – phase, and we’ll get to the interesting stuff later on. Certainly as has been indicated there are some interesting possibilities in the forecast. Setting up a complete, remote manageable and robust HPC cluster – despite the admittedly low computing performance of the Raspberries – nevertheless takes time, in particular the PXE booting is nowhere near robust enough at the moment.
UPDATE: one faulty new Cat6 cable discovered and certain issues with a consumer grade ADSL router built-in DHCP function. Will likely move cluster itself into a separate network and provide DHCP in that segment, using a real self contained solution.
Also, port level saturation was discovered on the network switch – during PXE booting – and was resolved initially by staggering the power-on of various functionalities in the overall cluster and its controller layer. Further investigation needs to be carried out, including possibilities offered by (switch) port configurations including QoS (Quality of Service) and other features.
Initially the whole cluster seems to boot at once now with the reduced network congestion on the Raspberry connected ports on the switch. One specific node is however still having odd issues, which I will investigate in the following days.
Broadening the bandwidth of (SDR) reception seems to turn up frequently in discussions, but is really not near the top requirements for the operation. Stuff of interest rather revolves around the cluster at large, its adaptive capabilities to varying conditions, pattern detection and automation.
Parallel processing when applicable and supported by the “problem” and dataset also is one of the priorities. For the vast number of radio related software, examining the possibilities and testing, testing, testing 🙂 is what I am most looking forward to. And finally – for those with an appetite for the heavy stuff, I recommend to keep an eye on Tim O’Shea and company DeepSig Inc.
– 73 de OH2BNF