Project Week 2008 Special topic breakout: GWE

Back to 2008 Summer Project Week Agenda

Agenda

GWE Overview

• Distributed enterprise system, which leverages on clusters of computers, in order to provide a grid computing platform for end users to easily and effectively parallelize the execution of inter-independent processes.
• GWE Client (End User - You) <=== Secure RPC Network ===> GWE Daemon(s) (Cluster Head Node)
• Detailed information can be found on the GWE Project Site.
• Overview Presentation

GWE Setup

Installation

• Download bundle, unbundle & set appropriate environment variables.
• Detailed information can be found on the 'GWE Installation Guide'

Configuration

• Define your grid and provide your authentication information to access "grid resources".
• Detailed information can be found on the 'GWE Configuration Guide'

Daemon Setup

• ONLY IF YOU ARE GOING TO ENHANCED A NEW CLUSTER WITH GWE CAPABILITIES (GWE Enabled).
• "Standalone GWE" will be available in the next release. GWE will run on your machine. Parallelization at the OS level, not distributed across cluster(s).
• From the machine where you installed GWE, run the utility "gwe-util-daemon-fullsetup.sh". That's it!
• After that a "GWE Daemon" will be running in your cluster having your cluster user as this "GWE Daemon" administrator (different than roles in the cluster OS!).
• A "GWE Daemon" is just another application in a machine, therefore "GWE Daemon"s can run in a single machine, although it is not recommended.
• Detailed information can be found on the 'GWE Administrator Guide'

GWE Clients

• GWE architecture provides multiple extensible modules, one of which is the "GWE Client API".
• Using the "GWE Client API", developers can create their own "GWE Client" applications.
• "GWE Client" applications are those; which allow end users to submit their "grid computing" requests to the GWE backend system.
• Currently, the GWE project team has created two "GWE Client" applications:

GWE Terminal

• Console based GWE client.
• Uses P2EL proprietary language to describe set of commands as permutation of templated commands with an associated parameter space.
• Many other monitoring/control commands available.
• Detailed information can be found on the 'End Users Guide'
• Video Demo

GSlicer3

• Slicer3 based GWE client ("GWE Powered" Slicer3).
• Basically a Slicer3 product enhanced with "GWE Client API" and custom development to provide "grid computing" capabilities to Slicer3.
• Setup: Download GSlicer3 bundle, unbundle it, set appropriate environmental variables.
• Config: Same as regular GWE.
• Detailed information can be found on the 'GSlicer3 Guide'
• Overview Presentation
• Video Demo

Future Work

• Usual fixes, improvements, user support, documentation and project management.
• GWE Core
  • "Standalone GWE" (requests parallelized as processes running in your machine - no cluster(s) needed).
  • Resolve "Windows" issues (cygwin does not work as expected).
  • Remote files directory staging (currently only single files).
  • Users application registry database and auto-deployment module to clusters.
  • Support for array type of iteration variables.
  • Daemon admin utilities (shutdown, restart, download DB, etc.)
  • Order behavioral parameters (max concurrent jobs running, job execution timeout, launch mode, virtual file system size, etc).
  • Alert notification module (email, pager, etc).
  • Scheduling module.
  • Requests control: abort, pause, resume (under construction).
  • Descriptive request result parsing. P2EL semantics + parser framework.
  • Multi-cluster functionality. Currently only single cluster mode (not true grid computing yet). Pending: module to daisy chain "GWE Daemon"s.
  • Single daemon services registry (Single IP Port/RMI registry) for multiple daemons.
  • More file system drivers: GridFTP, SRB, etc.
  • X.509 certificates support for authentication.
  • "GWE Powered" Portlets, FreeSurfer, FIPS, etc.
  • MPI support.
  • Order chaining (revamping prototyped framework for chaining Slicer3 modules).
  • More resource manager drivers: Torque, GWE, Cloud, etc.
  • JDSL support (standard submission protocol).
  • JMX support (runtime diagnostics).
  • Full feature statistical module.
  • etc...
• GSlicer3:
  • Centralized configuration.
  • Usage of currently available specialized widgets (and maybe new GWE specialized widgets?).
  • API to build specialized "GWE Powered" Slicer3 modules.
  • Auto-deployment of Slicer3 in running clusters (as an extension of the users application auto-deployment module).
  • GWE requests monitoring module to replace progress report in the status bar. UI with a realtime status of all GWE requests submitted using GSlicer3.