Thespian Python Actors


Thespian is a Python library providing a framework for developing concurrent, distributed, fault tolerant applications.
Thespian is built on the Actor Model which allows applications to be written as a group of independently executing but cooperating "Actors" which communicate via messages. These Actors run within the Actor System provided by the Thespian library.

Concurrent
All Actors run independently within the Actor System. The Actor System may run the Actors as threads, processes, or even sequential operations within the current process—all with no change to the Actors themselves.

Distributed
Actors run independently…anywhere. Multiple servers can each be running Thespian and an Actor can be run on any of these systems—all with no change to the Actors themselves. Thespian handles the communication between the Actors and the management process of distributing the Actors across the systems.

Location Independent
Because Actors run independently anywhere, they run independently of their actual location. A distributed Actor application may have part of it running on a local server, part running on a server in Amsterdam, and part running on a server in Singapore… or not, with no change or awareness of this by the Actors themselves.

Fault Tolerant
Individual Actors can fail and be restarted—automatically—without impact to the rest of the system.

Scalable
The number of Actors in the system can be dynamically extended based on factors such as work volume, and systems added to the Distributed Actor System environment are automatically utilized.

One of the key aspects of the Actor Model is that it represents a higher level of abstraction than is provided by most frameworks. When writing an Actor-based application, the concurrency and transport layers are completely abstracted, which both simplifies the design and allows the concurrency or transport to be changed in the future without requiring changes in the Actor-based application.
The above qualities of Actor programming make it ideally suited for Cloud-based applications as well, where compute nodes are added and removed from the environment dynamically.

History
-------

Thespian was developed at GoDaddy as part of the support for GoDaddy's
dedicate and virtual server product line. At one point, Thespian was
helping to manage 10,000 physical servers via a combination of
on-server and remote actors.

The primary Thespian author (Kevin Quick) is no longer working at
GoDaddy, but through GoDaddy's willingness to open-source the Thespian
code and transfer the rights to this author, this repository is still
being maintained and developed. With heartfelt thanks to the Vertigo
team at GoDaddy, this Python Actor model has been developed and
refined to provide a highly functional library for general use.

Getting Parallel with PyClips

This module embeds a fully functional CLIPS engine into Python, and gives the developer a more Python-compliant interface to CLIPS without cutting down on functionalities. In fact CLIPS is compiled into the module in its entirety, and most API functions are bound to Python methods. However the direct bindings to the CLIPS library (implemented as the _clips submodule) are not described here: each function is described by an appropriate documentation string, and accessible by means of the help() function or through the pydoc tool. Each direct binding maps to an API provided function. For a detailed reference for these functions see Clips Reference Guide Vol. II: Advanced Programming Guide, available for download at the CLIPS website.
PyCLIPS is also capable of generating CLIPS text and binary files: this allows the user to interact with sessions of the CLIPS system itself. An important thing to know, is that PyCLIPS implements CLIPS as a separated engine: in the CLIPS module implementation, CLIPS ``lives'' in its own memory space, allocates its own objects. The module only provides a way to send information and commands to this engine and to retrieve results from it.

Crowd Programming Engine (CPE)

"A Distributed Agent-Based system with PyClips, RandomForests and, ThespianPy Actors over a shared Tuple-Space." - Charles Kosta, Sc.D.

Getting Started with NetWorkSpaces

We are ultimately trying to use NetWorkSpaces (NWS) as a way for Python, R, and Java agents running on different machines to
communicate and coordinate with one another.

To get started with Python NWS, we'll first have to install the NWS server, and the Python NWS client.



pip install nwsserver (https://pypi.org/project/nwsserver/)

pip install Twisted (https://pypi.org/project/Twisted/)


Next, an NWS server must be started. This can be done from a shell
using the nws command, as follows:

% twistd -ny .local/nws.tac

From another window, start an interactive Python session, and import
the NetWorkSpace class:

% python
>>> from nws.client import NetWorkSpace

Next, create a work space called 'cpe':

>>> ws = NetWorkSpace('cpe')

This is using the NWS server on the local machine. Additional arguments
can be used to specify the hostname and port used by the NWS server if
necessary.

Once we have a work space, we can write data into a variable using the
store method:

>>> ws.store('joe', 17)

The variable 'joe' now has a value of 17 in our work space.

To read that variable we use the find method:

>>> age = ws.find('joe')

which sets 'age' to 17.

Note that the find method will block until the variable 'joe' has a
value. That is important when we're trying to read that variable from a
different machine. If it didn't block, you might have to repeatedly try
to read the variable until it succeeded.

There are times when you don't want to block, but just want to see if
some variable has a value. That is done with the findTry method.

--------------------------------------------------------------------------------------

From another window, start an interactive Python session, and import
the NetWorkSpace class:

% python
>>> from nws.client import NetWorkSpace

Next, connect to a work space called 'cpe':

>>> ws = NetWorkSpace('cpe')

This won't create the 'cpe' work space, since it already exists. Now you
can execute a operation such as:

>>> x = ws.fetch('joe')
>>> x
17

you should see that you got the value 17 from the previous w.store() function;
only this time you fetched it permanently. Now try to execute the the fetch again.

>>> x = ws.fetch('joe')

Your program will now be blocked! in this session, watch it block for a minute, and then execute a 'store' in
the other session:

>>> ws.store('joe', 18)

You should see the previous session return from the fetch command and x should now be 18.


That's a basic sample of this technology, which of course can become much more complex. Keep in mind
that these programs could have been on different machines; or hosted on Amazon.

More importantly, an R program and a python program
and even a Java program could be working cooperatively.




NetWorkSpaces for Python

NetWorkSpaces: a framework to coordinate distributed programs


NetWorkSpaces (NWS) is one way to write parallel programs. It allows you to take advantage of a multi-core machine; multiple virtual machines, as well as cloud-based clusters, using languages such as Python, R, Java, and Matlab. With NetWorkSpaces for Python, you can execute Python functions and programs in parallel using methods very much like the standard Python map function. In some cases, you may be able to parallelize your program in minutes, rather than months.

For example, here's a simple Python NWS script:

          from math import sqrt
          from nws.sleigh import Sleigh
          s = Sleigh()
          for x in s.imap(sqrt, xrange(10)):
              print x
          

It looks pretty simple, but you'll need to be familiar with the imap function in the standard itertools module.