Metaworker javascript library released (alpha)
With the HTML5 Web Workers standard, browsers now have support for Javascript worker threads, which offer the potential for performing heavy computations in the background without causing CPU usage warnings.
metaworker is a library which wraps around the HTML5 Web Workers standard. It provides an API which is very loosely based on Google's MapReduce algorithm. It partitions input, computes sub-solutions and combines sub-solutions into a solution. It's now posted on github in a very alpha form, and includes two motivating examples on usage. An up-to-date version of Firefox and Firebug are required to use the alpha.
Javascript programs that perform very heavy processing will typically trigger a warning from the browser which stops execution and asks the user whether they want to stop the offending script. Over the last couple years, I've been toying around with using Javascript code to perform more involved computations which often trigger these types of CPU usage warnings. This happens because the browser expects Javascript code to yield to the CPU fairly often; somewhere between fractions of a second to several seconds.
My early attempts to work around this problem involved partitioning the computation's input into slices and then placing setTimeout() delays between the processing of each slice. By placing delays between the chunks of processing, the executing Javascript code would yield to the CPU, however momentarily, and allow the browser to update state and reset the warning timer.
This worked well at first, but there are two problems with this solution: First, all of Javascript is executed in a single thread, which means you don't get the advantage of multiple cores common in hardware these days. Second, depending on system performance, it is very difficult to tune the timing and chunk size of such algorithms, which means you could still end up with CPU usage warnings.
metaworker aims to solve not only this problem (by using Web Workers) but also to abstract away some of the details of partitioning, dispatching, tracking, and so on, and allows the programmer to simply focus on mapping and reducing. Mapping, the main part of computation, happens inside of web workers. Reduction is still performed on the foreground thread of the browser, but in upcoming updates to the library should happen in workers as well (with optional transmission of intermediate results to the foreground for purposes of updating the user on what's going on).
About this entry
- Author:
- Maciek
- Published:
- Category:
- Javascript, Processing
- Tags:
- javascript, mapreduce, multithreading, webworkers
