Handling Power Loss - Cost vs Benefit

As part of my day-to-day life, I'm currently holding the president position in WiredSoc (the student computer science society in St. Andrews). It's an interesting and challenging position to hold.

One of the responsibilities I have (and delegate to some extent) is the reliable operation of WiredSoc's servers. Specifically it's me or the systems administrator that get complained to if something goes wrong.

This brings me onto a recent outage we had. At approximately 1am on Monday morning, services disappeared due to a brief power interruption/surge (I've got conflicting reports on that one). 2 of out 3 severs dropped out and came back online. Our public facing server came back online at about 9am and only after I manually kicked it into action.

Now an 8 hour outage on a voluntary service - is it acceptable? I would say yes. There is no way I'm going to force someone to go onsite at 2am to fix a problem when there is no compensation available. It's just beyond the bounds of acceptability! Really, would you volunteer to do it?

Also, a UPS (uninterrupted power supply, basically a battery used to keep computers ticking over in power loss) is out of the question - too costly and of little benefit. Realistically, power outages tend to take out our connectivity as well. So, we could remain powered up but of no use to anyone!

This leads me onto a similar problem I also saw with STAR's systems when I was in charge of those. Would it have been worth installing a UPS if we had the budget? Nope - power outages again took out the network and server. So, even if the studio was still live, we could not get the broadcast out (we were an online station).

In conclusion, it is my opinion that voluntary projects should strive for high availability. However, it should also be taken into consideration that it is not the end of the world if your service disappears temporarily.

A Lick Of Paint

There's many benefits to developing AllDay DJ in Java. These include the usual things like cross-platform support on a whim (I'm developing and testing on both Windows and Linux).

Also, the Swing toolkit (the library I use to build the user interface) comes with support for "Look and Feel". This leads me to one of the more frivolous benefits - the ability to change how AllDay DJ looks on a whim.

After a bit of mucking about on the internet I found the NIMROD look and feel. After a bit of tweaking, this is what AllDay DJ 3 now looks like:

I'm sure you'll agree, it's much easier on the eye.

Radio Station In A (Linux) Box

It's an interesting concept - is it possible to go from source audio file (mp3, wav, ogg) to a complete streaming station on just one computer system. The answer is yes and with only one soundcard too! (Though the software I developed and use can punt out to many soundcards for live use).

This past week or so, I've put together an internal test stream for AllDay DJ 3. This is mainly to prove it's up to the challenge. Also, it means I can run this on a headless (no monitor) system in a rack that I don't need for things like coursework. I can also get the music on any computer in the house then - very useful!

So, let's talk through the components, starting with the system itself. Well, it's your off the shelf Linux box. The one I'm using currently runs Gentoo (not my choice) but I have also done this with a stable Debian system.

Basic system in place, let's get our audio playing. A silent radio station is useless!

I installed PostgreSQL, a VNC server and Sun Java 6. This is enough to get AllDay DJ 3 ticking over. My music library (now categorised and rather broad spectrum) and some jingles I knocked together were loaded. At this point, I have a playout system pushing audio out to the sound card and sounding a bit like a radio station. The music choice got a bit of a mixed response from friends helping with the project. :P

Now, we've got our audio - how do we get it to the audience of one (me!). Well, that requires streaming. There's two components to this - the streaming source and the streaming server.

If you're not up with the terminology: the streaming source (as I've called it), is the piece of software that captures the audio then sends it to the streaming server. The streaming server then supplies copies of this stream to the listener.

Being a Linux box, I went open source. Initially I used Darkice as the source software and Icecast as the server software. Simply tell darkice to use the "Stereo Mix" as a source. It worked and I got an MP3 stream direct to my desktop.

However, it sounded a bit lifeless, especially compared to what you hear on the dial. Time to add some audio processing.

Now, I don't own a compressor and whatnot that I could plug into the audio stream. Also, it kills the idea of using the "Stereo Mix" source if I have to use external kit. To top it off, most audio processing kit can't be adjusted remotely.

What I needed was a software solution. Thankfully one exists. It means I have to move from an MP3 stream to an OGG stream. That's not a huge issue.

The solution is to make use of ecasound and ices. Both are tools that I had not heard of before until recently but due to the stdin/stdout support, they can be tied together without the need for another sound card - useful!

Now the command line I used to launch the internal stream is:

ecasound -i alsahw,0,0 -o stdout -eca:10,0.01,1,1 -ea:500 | ices /etc/ices2/ices.xml

This tells ecasound to take take audio from the first alsa sound card (set to record from "Stereo Mix" using alsamixer). The "-o stdout" bit tells ecasound to spew out to the ices streaming source software.

The next parts of the ecasound command are the inteseting parts.

"-eca:10,0.01,1,1" is the compressor settings. It means knee at 10%, attack of 0.01s then use a 1:1 ratio (I just want everything level / heavily compressed to start with, I'll adjust later). "-ea:500" just means amplify 500% (or 5x).

The final part of the command tells "ices" to launch with the correct config file. This has details of my server and to take an audio stream from stdin.

The suprising result of all this is that it possible to run an radio station from a Linux box. All be it one that streams on a closed network.

The Work Keeps Going

So, when I should really be working on my coursework project, I've been putting a bit more time into AllDay DJ. In the short time I have been doing this, I've managed to pull a fair bit off. Below is the list of features I've added this week:

Sweep Point

In previous version of AllDay DJ sweepers and voice tracks were dumbly played over the intro for songs. Now it's possible to have such a sweeper fully produced then sweep on the dry bit (like those "we play music like this... Donkey FM!" jingles). How? Just set the sweep point.

Networking

A large number of stations are doing it - taking programming from another station. The command cart system has been built in AllDay DJ to allow simple networking between a "server" station and a number of clients. The result: you get a nifty program from down the road with your own adverts and jingles.

Even better - it's a plain text protocol. This makes building extras for the system easy. All command start with ADDJ 3.0 to indicate the protocol (ADDJ) and version (3.0) and then consist of single words plus parameters. Proof, I suppose, that I listen to my lecturers about designing network protocols.

Command Cart System

It's not only networking that has benefited from work in this area. It's now possible to take an IRN feed and have it go in any sound card on your system and out any other. Very useful! The only issue now is endurance testing it - there's only so many times I can put up listening to a dummy news feed (really some music).

Keyboard Shortcuts

No need to fumble with that mouse any more! Some sensible shortcuts have now been build into AllDay DJ and work no matter what AllDay DJ window is in focus. Not bad for a Java application. The plan is to work on an extra mini-keyboard at some point with will have all the major keys in one place.

Scheduling

This is mostly done - music and jingle scheduling works on the well loved "deck" model and repetition rules can be enforced. As for adverts - erm... I've never had to schedule any in my life! Ideas for what would be required here are always considered good.

So... wrapping up: Remember - this is all cross-platform. I've been developing on a Windows XP system and testing on a headless Linux server (well, running vnc4server - there's no monitor plugged in). Also, it's all sitting on top of the reliable and easily backed up PostgreSQL database system. No bespoke binary rubbish here!

Voice Tracking in 4 Easy Steps

It's a part of modern radio that you can't get away from - voice tracking. If your response is "voice what?", it's the recording of presenter links in advance to be played out by the computer later. Often you won't know this practice is being used.

Anyway, its a feature missing from the older versions of AllDay DJ. So, I've just added it to version 3 as demonstrated in the following screen capture:

The left screen is the voice tracking panel and consists of the following steps:

1. Record or choose the audio to use.
2. Review the audio.
3. Add details (so other users can tell what it is).
4. Save then drag-and-drop into the log.

The right-hand screen is the log screen. Normally, it's showing the current log. However, as shown by the banner, it's actually a "future" log for 24/01/2010 at 11:00. You can edit and save any log without affecting what's going to air. Neat, eh? What's more, you can drag and drop audio from the future log into empty players to preview with. Even neater!

Technology - now working and on the blink

After finishing university for Christmas I've been busy. 1 live and another VT show. But that's normal.

I've spent some of the past couple of days setting up a test server for AllDay DJ (I'm now operating 2). The system is streaming fine from an old Duron 1.6GHz box I've got kicking around the house. This is all complete with in-house "Donkey FM" jingles. Well, you've got to test it with realistic audio. :P

Although I now have a few test rigs, development has come to a standstill at the moment due to university commitments. Specifically, I've got a sizable solo software development project to complete by April. It eats up a fair amount of time.

On another topic - we've had some technology fail. I'm currently shacking up with family for Christmas where they have a Sky HD box. All well and good, except it's throwing up an error. Specifically moaning about a lack of HDCP (encrypted content protection) support on the TV. Odd for something that's worked fine for a number of months. I've got a funny feeling that it's the set top box on the blink. However, the usual sky box fixes don't work. Go figure...

Pronouncing Local Names On Air

Local radio is made up of a number of different local aspects - stories, views, geographical knowledge, etc. Well, at least that's what it's supposed to be and most stations do it.

However, one issue that comes up from time to time is pronouncing place names. It can make you stick out like a sore thumb if you get a name obviously wrong. However, with a lot of names there's even some disagreement between the locals.

Take the example of Crossford, a village near Dunfermline. Is it crossf-OAR-d or crossf-OR-d? I'm more likely to pronounce it OAR, but that's due to going to school in the Dunfermline area.

My personal point of view is local place names should be pronounced properly if possible (be prepared to see "flem" on pronunciation sheets in Scotland!) otherwise phonetically.

As an exercise - why not try to figure out these Scottish place names that are occasionally mispronounced on air:

Kirkcaldy

East Wemyss

Scone

Falkirk

Broughty Ferry

And the answers (from my personal knowledge, may be wrong, etc.):

Kirkcaldy - Kirk-aw-dy

East Wemyss - East Weems

Auchtermuchty - As spelt but "ch" is as per the Scottish loch

Scone - Skoon

Falkirk - Fall-kirk

Broughty Ferry - Br-aw-ty Ferry

Feel free to add your viewpoint or pronunciations you see wrong all the time in the comments section.