pycos Releases

News about dispy / pycos. Forum members will be notified of releases.

Moderator: admin

Post Reply
Giri
Site Admin
Posts: 58
Joined: Sun Dec 27, 2020 5:35 pm

pycos Releases

Post by Giri »

Below is list of pycos release accouncements / short summary of changes.
  • 4.12.2 (2022-10-02)
    • Fixed closing of SSL sockets.
    • Fixed '_timed_out' method,
    • 'Task' constructor now supports 'varargs'.
  • 4.12.1 (2021-05-16)
    • Replaced named tuples with classes in dispycos so messages
      from scheduler are not confused with messages from retmote tasks
      that may be instances of tuples.
    • If 'host' parameter is not given, 'hostname' is used to find
      appropriate IP address to use for client, scheduler and node.
    • Fixed installation with new pip version.
  • 4.12.0 (2021-03-31)
    • dispycos servers now raise KeyboardInterrupt exception in
      tasks scheduled with dispycos before such tasks are
      terminated. This feature can be used by such tasks can take
      appropriate action such as sending appropriate response to client,
      save and transfer current state of computation (perhaps to resume
      computation from that state later) etc.
    • Added 'dispycos_kbd_int.py' in examples to show a simple use
      case to process KeyboardInterrupt raised in dispycos tasks.
    • Remote dispycos scheduler location should now be specified
      with 'schedule' method instead of in 'Client' constructor.
    • Changed 'ext_ip_addr' parameter of Pycos constructor to
      'ext_host' as it can be host name or IP address.
    • Fixed pycos to work with Python 3.6. This fix is necessary for
      dispy to work with Python 3.6.
  • 4.11.0 (2021-02-04)
    • If local or remote tasks fail, they will now have MonitorStatus instances as their result
      (so the result of using 'finish' or '()' method on a failed task will be a MonitorStatus object).
      This is useful to detect and diagnose faults.
  • 4.10.1 (2020-01-20)
    • Changed RTI to RPS (for Remote Pico Service or Remote Pycos Service).
    • Remote tasks created with RPS can now be waited with '()' or 'finish' method (similar to local
      tasks or remote tasks created with dispycos).
    • Added 'rps_log_watch' example to show how RPS can be used to monitor logs of remote servers(s).
    • Added 'dispycos_live_analytics' example to show how dispycos can be used for live/real-time
      analytics; this example is a slightly modified version of 'dispycos_client6', which is now
      removed.
    • Fixed '*node' methods of Client of dispycos in resolving node names.
  • 4.10.0 (2020-11-28)
    • Chagend dispycos Client constructor to take scheduler location as optional parameter. Default value for this parameter causes constructor to start private/local scheduler automatically so no need for user programs to start one for such cases.
    • Replaced MonitorException with MonitorStatus class with 3 attributes that can be used for different use cases.
    • Errors when running node_setup or server_setup or tasks with dispycos are sent back to client and logged. The error messages include traceback information so users can see, understand and fix issues with functions running on remote servers. These error messages are also shown at client so by default users can see errors.
    • If dispycos scheduler is private (i.e., not in shared environment), there is no need to start scheduler - it will be started automatically when Client is scheduled.
    • Changed 'node' parameter of Pycos constructor to 'host'.
    • Added 'dispycos_mem_servers.py' example that shows how dispycos can be used to load data in each server's memory and query computations on this data repeatedly. This allows client to view each CPU of cluster as separate computer (with each dispycos server that runs on a CPU having its own network address, memory and CPU).
    • User task names are no longer prefixed with '!' to avoid confusion.
    • Changed 'run' methods of dispycos's Client to 'rtask'.
    • Tasks (including remote tasks) can be waited with '()', which is easier than 'finish' method ('()' is a shorthand for 'finish' method).
    • dispycos Client's status_task gets 'TaskTerminated' message for any tasks terminated by dispycos servers.
    • Fixed interoperability with Windows serves so that a Linux / OS X client receiving instances of user classes from Windows dispycos server are received at client.
    • Changed semantics of 'zombie_period' parameter to dispycosnode. If zombie_period is given and abandon_zombie_nodes is True, then servers are closed if no tasks run for zombie_period seconds and client is closed if no tasks run at any server for 2*zombie_period seconds.
    • Client's methods that operate on nodes (e.g., suspend_node) can now be called with host name instead of IP address.
  • 4.9.1 (2020-10-13)
    • Added 'dispycos_close_node' function that computations may call to close the node where that computation is executing.
    • Added pico_client*.py and pico_service*.py examples that implement tiny (pico size) services.
    • dispycos servers now can recover even if computation tasks execute 'exit' or 'os.exit' either directly or indirectly (e.g., a library function that may exit).
    • Fixed dispycos to wait for pending tasks to finish if computation closes.
    • Fixed dispycosnode to wait for servers to finish pending tasks if computation closes or "quit" command is given at the command line.
    • Replaced 'dispycos_client_restart_servers.py' with 'dispycos_client10.py' to show how to use 'restart_servers' and 'dispycos_close_server'.
  • 4.9.0 (2020-09-09)
    • Value of tasks now is the value with 'return(value)' (in Python 3.7+) or with 'raise StopIteration(value)' in versions of Python prior to 3.7. It is no longer last value used with 'yield value', as done in Pycos versions up to 4.8. Thus, now tasks must use 'return' or 'raise StopIteration' to finish with value; otherwise, it is treated as 'return None' or 'raise StopIteration(None)'.
    • Renamed 'Computation' in dispycos to 'Client'. For now 'Computation' is an alias to 'Client' for backward compatability, but 'Computation' will be removed in a future release.
    • Added 'dispycos_close_server(terminate=False, restart=False)' function for dispycos computations (tasks) to quit the server the computation is executing at. Optional parameters 'terminate' and 'restart' can be used to control if all running tasks must be terminated right away and if the server must be restarted after quitting current instance.
    • Added 'restart_servers' optional parameter to 'Client'. This option can be used to always restart servers that are closed (by client either with 'dispycos_close_server' or 'close_server' method of 'Client').
    • Added 'restart_node' and 'restart_server' methods to 'Client'.
    • dispycos computations can now discover other servers with 'discover_peers' method.
    • Added 'dispycos_client_restart_servers.py' and 'dispycos_mas.py' to examples to illustrate new features.
    • Added 'rti_node_server.py' and 'rti_node_client.py' to examples. These can be used to administer (e.g., to periodically run tasks on all servers) using authentication for safety.
    • Fixed AsyncPipe under Windows and latest version of pywin32.
    • dispycosnode program under Windows now doesn't force daemon mode; with latest version of pywin32 multiprocessing works as expected while reading input.
    • 'peer_status' method of pycos can now be used more than once to install multiple tasks to be notified of peer status. In addition, 'None' can also be passed as task to remove all status tasks.
    • Added 'PickleProtocolVersion' parameter to config.py. This can be used work with different versions of Python that may use different versions of default pickle protocol versions. Setting this option also works with dispy, e.g., node using Python 3.7 and client using Python 3.8.
    • Fixed 'is_alive' method of Task.
  • 4.8.15
    pycos version 4.8.15 has been released. In this version
    • Added support for abandoning zombie dispycos nodes. This can be controlled with abandon_zombie_nodes parameter with Compute and abandon_zombie method.
    • Added tasks and close_node methods to Compute.
  • 4.8.14 (2020-02-18)

    pycos version 4.8.14 has been released. In this version
    • pycos now works with Python 3.8 under Windows.
    • Fixed dispycos scheduler to set scheduler port as configured in config.py.
  • 4.8.13 (2019-11-18)
    pycos version 4.8.13 has been released. In this version
    • Configuration parameters, such as MsgTimeout, MaxConnectionErrors etc. have been moved to config.py file in the package. This file can be modified for site-specific configuration instead of having to modify them in each program.
    • If zombie (dispycos) nodes come back, connection is re-established so such nodes can be used for distributed computing.
    • Fixed docker build script for Python 2.7.
    • Version 4.8.12 was also updated a few minutes ago, but it was missing config.py file so 4.8.13 has been uploaded fixing that issue.
  • 4.8.11 (2019-02-25)
    pycos version 4.8.11 has been released. In this version
    • Added checks to make sure files installed for Python versions up to 3.6 are not used with Python 3.7 and vice versa. For Python 3.7, the only way to install pycos is through pip and only those files can be used (and not, for example, files downloaded from github).
  • 4.8.10 (2019-01-07)
    pycos version 4.8.10 has been released. In this version
    • Changed node_allocations parameter to dispycos' Computation to nodes, which works similar to nodes of dispy project: Only nodes that match given allocations are used to run computation.
    • Sub-classing DispycosNodeAllocate (elements of nodes) now works even if remote dispycos scheduler is used.
  • 4.8.9 (2018-12-21)
    pycos version 4.8.9 has been released. In this version
    • Fixed installation with Python 3.7. Version 4.8.8 had appropriate install files but installation itself was broken.
  • 4.8.8 (2018-12-20)
    pycos version 4.8.8 has been released. In this release
    • Added support for Python 3.7
    • Fixed dispycos so clients automatically wait for computation to finish and terminate (broken in 4.8.7 release).
  • 4.8.7 (2018-12-17)
    pycos version 4.8.7 has been released. In this version
    • Added 'dispycos_cloud_computing.py' example to illustrate using dispycos for cloud computing.
    • Fixed a few issues / added improvements in dispycos.
    • Thanks to Sergey Kozlov for his contributions in this release.
  • 4.8.6 (2018-11-26)
    pycos version 4.8.6 has been released. In this release
    • Fixed dispycosnode for Python3 so servers don't close connection to node process (issue in 4.8.5 release).
    • Fixed docker file for Python 3 to build images for Python 3 (instead of Python 2).
  • 4.8.5 (2018-11-19)
    pycos version 4.8.5 has been released. In this release
    • Fixed dispycos module for Python 2.7 to import 'stat' module.
  • 4.8.4 (2018-10-29)
    pycos version 4.8.4 has been released. In this release
    • If Python interpreter has 'suid' and 'sgid' bits set, switch to that user and group when running computation. This protects files used by dispycosnode (e.g., files that store PID of processes) from side effects of computation.
  • 4.8.3 (2018-10-02)
    pycos version 4.8.3 has been released. In this version
    • Improved process termination in dispycosnode when servers close and when clean option is used to terminate processes in prior run. If psutil module is available (which is strongly recommended), additional checks are made to check that processes being terminated (from previous run) are actually dispycosnode servers.
  • 4.8.2 (2018-09-12)
    pycos version 4.8.2 has been released. In this release
    • Added support for *BSD systems (FreeBSD, NetBSD etc).
  • 4.8.1 (2018-08-26)
    pycos version 4.8.1 has been released. In this release
    • Monitors for a task receive GeneratorExit (with argument "terminated") message when that task is terminated.
      Fixed service start / stop / end times in dispycosnode.
  • 4.8.0 (2018-08-15)
    pycos version 4.8.0 has been released. In this release
    • DispycosNodeAllocate can now be used to add nodes to dispycos scheduler (in case node are not detected automatically in local network, or if nodes are in remote network). Subclassed DispycosNodeAllocate objects can also be used with remote scheduler.
    • Remote tasks and channel references are protected from ID reuse clashes (i.e., interpretter may reuse an ID if object goes away so remote references may not refer to object originally sent).
    • dispycosnode is more robust in closing servers when computation is closed or if it is deemed zombie computation.
  • 4.7.7 (2018-07-26)
    pycos version 4.7.7 has been released. In this version
    • Fixed host name resolution when netifaces is not available and resolving name with IPv6 may fail.
    • Changed peer method's broadcast parameter to relay.
  • 4.7.6 (2018-07-25)
    pycos version 4.7.6 has been released. In this version
    • Fixed host name resolution with Windows when netifaces module is not available.
    • Fixed dispycosnode to allocate only CPUs reserved by the scheduler (instead of all availbale even if reserved number is smaller).
    • Any nodes given with DispycosNodeAllocate will be automatically located by the scheduler (so the client doesn't need to use peer method do discover).
    • Added port parameter to DispycosNodeAllocate so clients can indicate which port is used by given node if different from default port.
    • If dispycosnode has more CPUs than given ports with tcp_ports option(s), use any necessary ports for remaining CPUs starting from last tcp_ports option.
  • 4.7.5 (2018-07-16)
    pycos version 4.7.5 has been released. In this release
    • Fixed exception (crash) when server and client programs are run on same computer (EADDRINUSE from errno should be imported in netpycos instead of pycos).
  • 4.7.4 (2018-07-15)
    pycos version 4.7.4 has been released. The changes since last release are:
    • Fixed potential crash during shutdown while SSL sockets are used.
    • Fixed potential issue with binding to link-local IPv6 address with OS X.
  • 4.7.3 (2018-07-05)
    pycos version 4.7.3 has been released. In this version
    • Fixed crash when AsyncFile/AsyncPipe is used (broken in version 4.7.1).
    • Updated pipe_csum.py and pipe_grep.py example files to use pycos (these were left behind when convertnig from asyncoro).
    • Fixed UDP broadcasting with OS X.
    • Fixed dispycosnode with Python 2 under Windows.
  • 4.7.2 (2018-07-02)
    pycos version 4.7.2 has been released. In this version
    • Changed default value of ipv4_udp_multicast in py3 branch (for Python 3) to be False as documented (this was incorrectly True in version 4.7.1).
    • Changed multicast group address for IPv6 to a distinct form (from a generic simple form) so pycos doesn't interfere with any other devices that use generic multicast address.
  • 4.7.1 (2018-06-25)
    pycos version 4.7.1 has been released. Changes since last release are:
    • Fixed networking (netpycos module) under Windows (broken in 4.7.0)
    • Added optional parameter ipv4_udp_multicast to use multicasting (instead of default broadcasting) for UDP with IPv4.
  • 4.7.0 (2018-06-17)
    pycos version 4.7.0 has been released. Changes since last release are
    • pycos now uses 9705 port by default (instead of 51350). dispycosnode program uses ports 9706 onwards by default. If using firewall, please update your settings to match these ports.
    • By default tcp_port used is 9705, not a random port as done in earlier releases.
    • finish method of pycos (either from pycos module or netpycos module) now cleans up all setup so after this method is done, another pycos scheduler can be created if necessary; i.e., in the same program pycos schedulers can be created and destroyed repeatedly.
  • 4.6.6 (2018-06-05)
    pycos version 4.6.6 has been released. The changes since last release are:
    • Fixed IPv6 network configuration when netifaces module is available.
    • Added 'suspend' and 'resume' methods for computation to suspend / resume nodes or servers. Web interface supports these features as well.
    • Fixed shutdown methods so after one instance of Pycos is shutdown, another instance can be started in the same program (if necessary).
    • Now only one release file is available from sourceforge. This file can be directly used to install with pip install pycos-4.6.6.tar.gz to install 4.6.6 release.
  • 4.6.5 (2017-11-14)
    pycos version 4.6.5 has been released. In this version
    • Added monitor method to RTI (Remote Task Invocation). This can be used to set or reset monitor task for future tasks created with an RTI instance (instead of having to set monitor method for each task created).
    • Fixed a race condition with closing IOCP sockets when pycos is terminating.
  • 4.6.4 (2017-09-13)
    pycos version 4.6.4 has been released. In this version
    • Fixed send_file method of Pycos to use appropriate socket family for the remote peer. This was broken since 4.6.0 release when support for multiple socket families was added to netpycos (but send_file was not updated to use modified structures).
    • Fixed name property (method) of Task and Channel. Since 4.6.0 release the name returned missed first character.
  • 4.6.3 (2017-09-06)
    pycos version 4.6.3 has been released. In this version,
    • Added zombie_period option to Computation. This represents maximum time in seconds allowed for a node to have no communication before scheduler deems a node to be dead. When a node is deemed zombie, scheduler informs status_task of Computation with abandoned status for tasks, servers and that node.
    • zombie_period and pulse_interval parameters set in Computation do not influence setting of zombie_period option for dispycosnode program - dispycosnode only uses the value of zombie_period option to determine if a client is dead.
  • 4.6.2 (2017-08-30)
    pycos version 4.6.2 has been released. In this release dispycosnode has been fixed so server processes and node task communicate.
  • 4.6.1 (2017-06-19)
    pycos version 4.6.1 has been released. This release changes licence to Apache 2.0 (from MIT). In this release, UDP server task has been fixed to handle multiple network interfaces having same broadcast address (in the case of multihome networking).
  • 4.6.0 (2017-06-06)
    pycos version 4.6.0 has been released. In this version support for multihome networking, i.e., host connected to multiple networks, has been added. To use this feature, node parameter to netpycos.Pycos can be given as list of host names or IP addresses.
Top
Post Reply

Return to “News”