HTML5 Audio Data API (Videos)
Well seeing as our work has been posted all over the internet but not my own blog and because It has been a long time since I have posted anything I thought I would take the opportunity to summarize what has been happening lately.
The Audio Data API has come a long way and we have build many new demos on top of it. A patch for Mozilla Firefox is currently in the review process and will hopefully be accepted and released as part of Firefox 4 (the next major release).
There have been some great developments, in march at the Boston Processing-js workshop I was lucky enough to meet Doug Schepers of the W3C. After showing him some of the demos I was working on we started talking about drafting an official W3C Audio spec. After the workshop he really got the ball rolling and started an Audio Incubator Group which I am happy to report now includes members from Mozilla, Yahoo, Google, and Apple. The discussion on the Audio XG mailing list has been ramping up as members from Google and Apple have put together their own draft spec. It will be exciting to see how things come together now that we have all the major players on board. It is very strange to think about how this all began and where it is going.
The audio ticket on bugzilla that started it all shows most of the discussion that informed our thought process when defining the API. There has been a lot of back and forth on what should and should not be included in the spec, specifically what should be done natively in browser C++ and what should be offloaded to Javascript. I initially believed Javascript would be way to slow for things like realtime FFT and filtering but after testing that theory we discovered that algorithms like FFT can be computed very quickly in JS. After optimizing we are down to about 0.25ms on a 1024 sample buffer.
I started a project called DSP.js which is a collection of digital signal processing functions. It currently includes a Wavetable oscillator, ADSR envelopes, Biquad filters (lowpass, highpass, bandpass, notch, shelfing), Window functions, FFT , DFT and a Sampler. The next version will include feedback delay and reverb fx. Pretty impressive to see this all done in js and operating in real time on a 44.1KHz signal.
The new typed arrays which are currently staged on minefield trunk have given us a huge speed increase for doing array operations. These are the same arrays which are being used in WebGL, Float32Array, UInt32Array, etc. These typed arrays are available in webkit nightlies as well.
Synthesizer
This demo uses DSP.js to generate a polyphonic dual-osc signal and apply envelopes and filters. The sequencer is rendered with processing-js and the UI controls are JQuery-UI. You can automate the filter controls by drawing a curve in the sequencer window.
HTML5 and Javascript Synthesizer from Corban Brook on Vimeo.
Low pass filter
This example takes the data played from a muted HTML5 audio tag and pipes it through a Low pass filter and then writes the data out to a second audio tag.
HTML5 Audio with Realtime Javascript Low Pass Filter from Corban Brook on Vimeo.
Sampler
The sampler is my latest demo. It buffers a sample wave table from a HTML5 audio tag and then allows you to specify the loop region and loop modes for playback. Allowing you to play back peices at any pitch. Loop modes include: none, forward, forward-backwards, backwards.
Web Audio Data API – Audio Sampler from Corban Brook on Vimeo.