NetWorkSpaces: a framework to coordinate distributed programs


NetWorkSpaces (NWS) is one way to write parallel programs. It allows you to take advantage of a multi-core machine; multiple virtual machines, as well as cloud-based clusters, using languages such as Python, R, Java, and Matlab. With NetWorkSpaces for Python, you can execute Python functions and programs in parallel using methods very much like the standard Python map function. In some cases, you may be able to parallelize your program in minutes, rather than months.

For example, here's a simple Python NWS script:

          from math import sqrt
          from nws.sleigh import Sleigh
          s = Sleigh()
          for x in s.imap(sqrt, xrange(10)):
              print x
          

It looks pretty simple, but you'll need to be familiar with the imap function in the standard itertools module.