Difference between revisions of "Project Week 2008 Special topic breakout: GWE"

From NAMIC Wiki
Jump to: navigation, search
 
(6 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
  [[2008_Summer_Project_Week#Agenda|Back to 2008 Summer Project Week Agenda]]
 
  [[2008_Summer_Project_Week#Agenda|Back to 2008 Summer Project Week Agenda]]
−
Requests for Agenda:
+
 
−
*tutorial format on how to use GWE
+
== GWE Overview ==
−
*include materials so that attendees can go back to their labs and step through the process and run things through the GWE.  
+
* Distributed enterprise system, which leverages on clusters of computers, in order to provide a grid computing platform for end users to easily and effectively parallelize the execution of inter-independent processes.
−
*Does one need own grid to do this? If yes, the tutorial could discuss how to set up a grid as well. What does one need to download? What architectures can it support? What access rights does one need to set it up (does one need local sys admins do something)?
+
* GWE Client (End User - You) <=== Secure RPC Network ===> GWE Daemon(s) (Cluster Head Node)
−
*How to apply GWE to pecific NA-MIC tasks such as:
+
* Detailed information can be found on the [http://www.gridwizardenterprise.org GWE Project Site].
−
**parameter space exploration of BSplines
+
 
 +
== GWE Setup ==
 +
 
 +
=== Installation ===
 +
* Download bundle, unbundle & set appropriate environment variables.
 +
* Detailed information can be found on the [http://www.gridwizardenterprise.org/guides/install.html GWE Installation Guide]
 +
 
 +
=== Configuration ===
 +
* Define your grid and provide your authentication information to access "grid resources".
 +
* Detailed information can be found on the [http://www.gridwizardenterprise.org/guides/config.html GWE Configuration Guide]
 +
 
 +
=== Daemon Setup ===
 +
* ONLY IF YOU ARE GOING TO ENHANCE A NEW CLUSTER WITH GWE CAPABILITIES (GWE Enabled).
 +
* "Standalone GWE" will be available in the next release (To run GWE on your machine. Parallelization at the OS level, not distributed across cluster(s)).
 +
* From the machine where you installed GWE, run the utility "gwe-util-daemon-fullsetup.sh". That's it!
 +
* After that a "GWE Daemon" will be running in your cluster having your cluster user as this "GWE Daemon" administrator (different than roles in the cluster OS!).
 +
* A "GWE Daemon" is just another application in a machine, therefore "GWE Daemon"s can run in a single machine, although it is not recommended.
 +
* Detailed information can be found on the [http://www.gridwizardenterprise.org/guides/admin.html GWE Administrator Guide]
 +
 
 +
== GWE Clients ==
 +
* GWE architecture provides multiple extensible modules, one of which is the "GWE Client API".
 +
* Using the "GWE Client API", developers can create their own "GWE Client" applications.
 +
* "GWE Client" applications are those; which allow end users to submit their "grid computing" requests to the GWE backend system.
 +
* Currently, the GWE project team has created two "GWE Client" applications:
 +
 
 +
=== GWE Terminal ===
 +
* Console based GWE client.
 +
* Uses P2EL proprietary language to describe set of commands as permutation of templated commands with an associated parameter space.
 +
* Many other monitoring/control commands available.
 +
* Detailed information can be found on the [http://www.gridwizardenterprise.org/guides/user.html End Users Guide]
 +
 
 +
=== GSlicer3 ===
 +
* Slicer3 based GWE client ("GWE Powered" Slicer3).
 +
* Basically a Slicer3 product enhanced with "GWE Client API" and custom development to provide "grid computing" capabilities to Slicer3.
 +
* Setup: Download GSlicer3 bundle, unbundle it, set appropriate environmental variables.
 +
* Config: Same as regular GWE.
 +
* Detailed information can be found on the [http://www.gridwizardenterprise.org/integration/slicer.html GSlicer3 Guide]
 +
 
 +
== Future Work ==
 +
* Usual fixes, improvements, user support, documentation and project management.
 +
* GWE Core
 +
** "Standalone GWE" (requests parallelized as processes running in your machine - no cluster(s) needed).
 +
** Resolve "Windows" issues (cygwin does not work as expected).
 +
** Support for array type of iteration variables.
 +
** Users application registry database and auto-deployment module to clusters.
 +
** Daemon admin utilities (shutdown, restart, download DB, etc.).
 +
** Order behavioral parameters (max concurrent jobs running, job execution timeout, launch mode, vfs clean up policy, pre-install applications, etc).
 +
** Alert notification module (email, pager, etc).
 +
** Scheduling module.
 +
** Requests control: abort, pause, resume (under construction).
 +
** Descriptive request result parsing. P2EL semantics + parser framework.
 +
** Multi-cluster functionality. Currently only single cluster mode (not true grid computing yet). Pending: module to daisy chain "GWE Daemon"s.
 +
** Single daemon services registry (Single IP Port/RMI registry) for multiple daemons.
 +
** More file system drivers: GridFTP, SRB, XNAT/XCAT?, etc.
 +
** More resource manager drivers: Torque, GWE, Cloud, etc.
 +
** X.509 certificates support for authentication.
 +
** Order chaining (revamping prototyped framework for chaining Slicer3 modules).
 +
** MPI support.
 +
** "GWE Powered" Portlets, FreeSurfer, FIPS, etc.
 +
** JDSL support (standard submission protocol).
 +
** JMX support (runtime diagnostics).
 +
** Full feature statistical module.
 +
** etc...
 +
* GSlicer3:
 +
** Centralized configuration.
 +
** Usage of currently available specialized widgets (and maybe new GWE specialized widgets?).
 +
** API to build specialized "GWE Powered" Slicer3 modules.
 +
** Auto-deployment of Slicer3 in running clusters (as an extension of the users application auto-deployment module).
 +
** GWE requests monitoring module to replace progress report in the status bar. UI with a realtime status of all GWE requests submitted using GSlicer3.
 +
 
 +
== Links ==
 +
* GWE Core - [[Media:GWE-Core.ppt|Overview Presentation]] / [[Media:GWECore-Demo.mov|Video Demo]]
 +
* GSlicer  - [[Media:GWE-GSlicer3.ppt|Overview Presentation]] / [[Media:GSlicer3-Demo.mov|Video Demo]]

Latest revision as of 14:52, 26 June 2008

Home < Project Week 2008 Special topic breakout: GWE
Back to 2008 Summer Project Week Agenda

GWE Overview

  • Distributed enterprise system, which leverages on clusters of computers, in order to provide a grid computing platform for end users to easily and effectively parallelize the execution of inter-independent processes.
  • GWE Client (End User - You) <=== Secure RPC Network ===> GWE Daemon(s) (Cluster Head Node)
  • Detailed information can be found on the GWE Project Site.

GWE Setup

Installation

  • Download bundle, unbundle & set appropriate environment variables.
  • Detailed information can be found on the GWE Installation Guide

Configuration

  • Define your grid and provide your authentication information to access "grid resources".
  • Detailed information can be found on the GWE Configuration Guide

Daemon Setup

  • ONLY IF YOU ARE GOING TO ENHANCE A NEW CLUSTER WITH GWE CAPABILITIES (GWE Enabled).
  • "Standalone GWE" will be available in the next release (To run GWE on your machine. Parallelization at the OS level, not distributed across cluster(s)).
  • From the machine where you installed GWE, run the utility "gwe-util-daemon-fullsetup.sh". That's it!
  • After that a "GWE Daemon" will be running in your cluster having your cluster user as this "GWE Daemon" administrator (different than roles in the cluster OS!).
  • A "GWE Daemon" is just another application in a machine, therefore "GWE Daemon"s can run in a single machine, although it is not recommended.
  • Detailed information can be found on the GWE Administrator Guide

GWE Clients

  • GWE architecture provides multiple extensible modules, one of which is the "GWE Client API".
  • Using the "GWE Client API", developers can create their own "GWE Client" applications.
  • "GWE Client" applications are those; which allow end users to submit their "grid computing" requests to the GWE backend system.
  • Currently, the GWE project team has created two "GWE Client" applications:

GWE Terminal

  • Console based GWE client.
  • Uses P2EL proprietary language to describe set of commands as permutation of templated commands with an associated parameter space.
  • Many other monitoring/control commands available.
  • Detailed information can be found on the End Users Guide

GSlicer3

  • Slicer3 based GWE client ("GWE Powered" Slicer3).
  • Basically a Slicer3 product enhanced with "GWE Client API" and custom development to provide "grid computing" capabilities to Slicer3.
  • Setup: Download GSlicer3 bundle, unbundle it, set appropriate environmental variables.
  • Config: Same as regular GWE.
  • Detailed information can be found on the GSlicer3 Guide

Future Work

  • Usual fixes, improvements, user support, documentation and project management.
  • GWE Core
    • "Standalone GWE" (requests parallelized as processes running in your machine - no cluster(s) needed).
    • Resolve "Windows" issues (cygwin does not work as expected).
    • Support for array type of iteration variables.
    • Users application registry database and auto-deployment module to clusters.
    • Daemon admin utilities (shutdown, restart, download DB, etc.).
    • Order behavioral parameters (max concurrent jobs running, job execution timeout, launch mode, vfs clean up policy, pre-install applications, etc).
    • Alert notification module (email, pager, etc).
    • Scheduling module.
    • Requests control: abort, pause, resume (under construction).
    • Descriptive request result parsing. P2EL semantics + parser framework.
    • Multi-cluster functionality. Currently only single cluster mode (not true grid computing yet). Pending: module to daisy chain "GWE Daemon"s.
    • Single daemon services registry (Single IP Port/RMI registry) for multiple daemons.
    • More file system drivers: GridFTP, SRB, XNAT/XCAT?, etc.
    • More resource manager drivers: Torque, GWE, Cloud, etc.
    • X.509 certificates support for authentication.
    • Order chaining (revamping prototyped framework for chaining Slicer3 modules).
    • MPI support.
    • "GWE Powered" Portlets, FreeSurfer, FIPS, etc.
    • JDSL support (standard submission protocol).
    • JMX support (runtime diagnostics).
    • Full feature statistical module.
    • etc...
  • GSlicer3:
    • Centralized configuration.
    • Usage of currently available specialized widgets (and maybe new GWE specialized widgets?).
    • API to build specialized "GWE Powered" Slicer3 modules.
    • Auto-deployment of Slicer3 in running clusters (as an extension of the users application auto-deployment module).
    • GWE requests monitoring module to replace progress report in the status bar. UI with a realtime status of all GWE requests submitted using GSlicer3.

Links

Retrieved from "https://www.na-mic.org/w/index.php?title=Project_Week_2008_Special_topic_breakout:_GWE&oldid=27578"