HTML5 Audio on desktop browsers is a mess. But you haven't experienced true pain and suffering until you try to get HTML5 audio done on mobile devices. This article is a follow-up to my talk at the Barcamp in Salzburg in March and was inspired by Chris Heilmans tweets on that topic a few days ago.

First of all: We all know Modernizr, we all use it and we all love it. With Modernizr we made a huge step away from applying features by browser detection towards feature detection. This is not only the recommended way of building websites and web-apps, but even more a mindset or way of thinking: Ask your browser what it can do (do a test), and you know what you can use, absolutely independent of browser vendors and versions.

For such a complex feature like HTML5 Audio, it's great that with areweplayingyet there is a community out there which supplies such tests for HTML5 Audio subfeatures, API calls and best practices like hot swapping audio sources.

If you run through some tests and view the browser compatibilty list, you will see that many tests fail on mobile browsers like early Android implementations and especially on iOS devices. But why? Is HTML5 Audio compatibility really that bad on iPad and iPhone, especially when it's the only way to include sound?

Well. Yes and no. HTML5 Audio is pretty f***ed up on iOS too, but it works. And if you know the rules, you know why all tests fail and how you can deal with that.

#1 - One does not simply play sound on JS event

The most important rule of all: Do not think that you just can play sound by using JS events, like you're used to when working with Desktop browsers. Each sound playing requires a user interaction, their "OK" or their "Go for it" by touching an element on your page. This can be the built-in Audio player controls (which seem to be not compliant to Apples user interface standards by being flipping small), or any click or touch eventhandler bound to your elements. Once an interaction is done, the file is downloaded to your device. After that, you can do all the cool audio things you want to do.

But why do you need user interaction to play audio (and video)? Isn't that unusual? Might be, but imagine yourself with your iPhone in a roaming region, having limited bandwidth, trying to reach your favourite website which just came up with some nifty 10MB background song not only trying to ruin your nerves, but also adding some more bucks on your phone bill for next month. User interaction seems legit now.

Possible solution for that: Have a splash screen in your app, that the user has to remove, and that triggers the download of your soundfile. Quirky and awful, but so far the only way at the moment.

#2 - There can be only one!

Classy old Immortal rule: You can't play more than one sound file at once. So no background music and laser buzzes in your star shooter. Personally I'm okay with that. But how about making use of that? Think of "There should be only one!". Enter sound sprites.

Image sprites are well known: Having one image with all icons and stuff, and accessing them by putting the right background-position in your CSS file. Sound sprites work similar: Put all sound you want to play in one audio file, and define start and ending indices. Now the workflow is as follows:

• User removes splash screen, downloads file
• A few seconds of silence are played, right at the start and right after interaction
• After that, we should have direct access to all positions in the whole file. Just jump to your positions as you like

Below is some sprite demo I came up with in my talk, but also take a look at Chris' more sophisticated solution.

<audio src="sprite.mp3" controls="none" id="myaudio" />

var maudio = document.getElementById('myaudio');

var soundSprite = [
  {start: 0, end: 3000},
  {start: 3500, end: 6789}
];

element.addEventListener('touchstart', function(ev) {
  maudio.play();
  playSoundFile(0);
})

function playSoundFile(idx) {
  maudio.currentPosition = soundSprite[idx].start;
  var x = setInterval(function() {
    if(maudio.currentPosition >= soundSprite[idx].end) {
      maudio.pause(); // There is no stop() in HTML5
      clearInterval(x);
    }
  }, 50);
}

Soundsprites are pretty common now, and well supported by libraries like SoundManager2.

Think back to Modernizr: We now know that iOS devices support HTML5 Audio, we know that we can do a lot with that even if the tests fail. Can you adapt your tests? Is it okay to have the user interact with your app before testing? Is it okay to download test files? Or do you just go back to user agent sniffing to wrap up your target platforms?

Internet Explorer 9 Mobile

The things learned above apply to most mobile Webkit implementations, but we have to deal with another browser as well: IE9 on mobile devices

IE9 doesn't support a lot HTML5/CSS3 features, but the features that it supports are usually really well implemented and robust. Same goes for HTML5 Audio, which also has been stated by the developers of Cut the rope in a blog post some months ago.

Same goes for the mobile version of Internet Explorer 9. And usually, you can expect everything that works in IE9 will also work in IE9 mobile. Same goes for sound, we even don't have to wait for user interaction when playing and downloading files! Although I don't know if that's a good thing or a bad one.

However, there's one thing that doesn't work on IE9 Mobile, and that is changing your sound "position" when playing, so sound sprites won't work there. Now imagine a web-app exclusively covering IE9 mobile and iOS, I wonder how tests are going to look like there, or if you just go back to user agent sniffing.

Bottom line: HTML5 Audio is a wreck at the moment, and don't get fooled by "support". Just because your devices supports a certain feature does not mean that it's usable in the way you are used to.

Related Articles