Difference between revisions of "Project Week 2008 Special topic breakout: GWE"
From NAMIC Wiki
| (2 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
[[2008_Summer_Project_Week#Agenda|Back to 2008 Summer Project Week Agenda]] | [[2008_Summer_Project_Week#Agenda|Back to 2008 Summer Project Week Agenda]] | ||
| − | |||
| − | |||
== GWE Overview == | == GWE Overview == | ||
| Line 7: | Line 5: | ||
* GWE Client (End User - You) <=== Secure RPC Network ===> GWE Daemon(s) (Cluster Head Node) | * GWE Client (End User - You) <=== Secure RPC Network ===> GWE Daemon(s) (Cluster Head Node) | ||
* Detailed information can be found on the [http://www.gridwizardenterprise.org GWE Project Site]. | * Detailed information can be found on the [http://www.gridwizardenterprise.org GWE Project Site]. | ||
| − | |||
== GWE Setup == | == GWE Setup == | ||
| Line 13: | Line 10: | ||
=== Installation === | === Installation === | ||
* Download bundle, unbundle & set appropriate environment variables. | * Download bundle, unbundle & set appropriate environment variables. | ||
| − | * Detailed information can be found on the | + | * Detailed information can be found on the [http://www.gridwizardenterprise.org/guides/install.html GWE Installation Guide] |
=== Configuration === | === Configuration === | ||
* Define your grid and provide your authentication information to access "grid resources". | * Define your grid and provide your authentication information to access "grid resources". | ||
| − | * Detailed information can be found on the | + | * Detailed information can be found on the [http://www.gridwizardenterprise.org/guides/config.html GWE Configuration Guide] |
=== Daemon Setup === | === Daemon Setup === | ||
| − | * ONLY IF YOU ARE GOING TO | + | * ONLY IF YOU ARE GOING TO ENHANCE A NEW CLUSTER WITH GWE CAPABILITIES (GWE Enabled). |
| − | * "Standalone GWE" will be available in the next release | + | * "Standalone GWE" will be available in the next release (To run GWE on your machine. Parallelization at the OS level, not distributed across cluster(s)). |
* From the machine where you installed GWE, run the utility "gwe-util-daemon-fullsetup.sh". That's it! | * From the machine where you installed GWE, run the utility "gwe-util-daemon-fullsetup.sh". That's it! | ||
* After that a "GWE Daemon" will be running in your cluster having your cluster user as this "GWE Daemon" administrator (different than roles in the cluster OS!). | * After that a "GWE Daemon" will be running in your cluster having your cluster user as this "GWE Daemon" administrator (different than roles in the cluster OS!). | ||
* A "GWE Daemon" is just another application in a machine, therefore "GWE Daemon"s can run in a single machine, although it is not recommended. | * A "GWE Daemon" is just another application in a machine, therefore "GWE Daemon"s can run in a single machine, although it is not recommended. | ||
| − | * Detailed information can be found on the | + | * Detailed information can be found on the [http://www.gridwizardenterprise.org/guides/admin.html GWE Administrator Guide] |
== GWE Clients == | == GWE Clients == | ||
| Line 37: | Line 34: | ||
* Uses P2EL proprietary language to describe set of commands as permutation of templated commands with an associated parameter space. | * Uses P2EL proprietary language to describe set of commands as permutation of templated commands with an associated parameter space. | ||
* Many other monitoring/control commands available. | * Many other monitoring/control commands available. | ||
| − | * Detailed information can be found on the | + | * Detailed information can be found on the [http://www.gridwizardenterprise.org/guides/user.html End Users Guide] |
| − | |||
=== GSlicer3 === | === GSlicer3 === | ||
| Line 45: | Line 41: | ||
* Setup: Download GSlicer3 bundle, unbundle it, set appropriate environmental variables. | * Setup: Download GSlicer3 bundle, unbundle it, set appropriate environmental variables. | ||
* Config: Same as regular GWE. | * Config: Same as regular GWE. | ||
| − | * Detailed information can be found on the | + | * Detailed information can be found on the [http://www.gridwizardenterprise.org/integration/slicer.html GSlicer3 Guide] |
| − | |||
| − | |||
== Future Work == | == Future Work == | ||
| Line 54: | Line 48: | ||
** "Standalone GWE" (requests parallelized as processes running in your machine - no cluster(s) needed). | ** "Standalone GWE" (requests parallelized as processes running in your machine - no cluster(s) needed). | ||
** Resolve "Windows" issues (cygwin does not work as expected). | ** Resolve "Windows" issues (cygwin does not work as expected). | ||
| − | ** | + | ** Support for array type of iteration variables. |
** Users application registry database and auto-deployment module to clusters. | ** Users application registry database and auto-deployment module to clusters. | ||
| − | ** Order behavioral parameters (max concurrent jobs running, job execution timeout, launch mode, | + | ** Daemon admin utilities (shutdown, restart, download DB, etc.). |
| + | ** Order behavioral parameters (max concurrent jobs running, job execution timeout, launch mode, vfs clean up policy, pre-install applications, etc). | ||
** Alert notification module (email, pager, etc). | ** Alert notification module (email, pager, etc). | ||
** Scheduling module. | ** Scheduling module. | ||
** Requests control: abort, pause, resume (under construction). | ** Requests control: abort, pause, resume (under construction). | ||
** Descriptive request result parsing. P2EL semantics + parser framework. | ** Descriptive request result parsing. P2EL semantics + parser framework. | ||
| − | ** Multi-cluster functionality. Currently only single cluster mode (not true grid computing yet). | + | ** Multi-cluster functionality. Currently only single cluster mode (not true grid computing yet). Pending: module to daisy chain "GWE Daemon"s. |
** Single daemon services registry (Single IP Port/RMI registry) for multiple daemons. | ** Single daemon services registry (Single IP Port/RMI registry) for multiple daemons. | ||
| − | ** More file system drivers: GridFTP, SRB, etc. | + | ** More file system drivers: GridFTP, SRB, XNAT/XCAT?, etc. |
| + | ** More resource manager drivers: Torque, GWE, Cloud, etc. | ||
** X.509 certificates support for authentication. | ** X.509 certificates support for authentication. | ||
| + | ** Order chaining (revamping prototyped framework for chaining Slicer3 modules). | ||
| + | ** MPI support. | ||
** "GWE Powered" Portlets, FreeSurfer, FIPS, etc. | ** "GWE Powered" Portlets, FreeSurfer, FIPS, etc. | ||
| − | |||
| − | |||
| − | |||
** JDSL support (standard submission protocol). | ** JDSL support (standard submission protocol). | ||
** JMX support (runtime diagnostics). | ** JMX support (runtime diagnostics). | ||
| Line 79: | Line 74: | ||
** Auto-deployment of Slicer3 in running clusters (as an extension of the users application auto-deployment module). | ** Auto-deployment of Slicer3 in running clusters (as an extension of the users application auto-deployment module). | ||
** GWE requests monitoring module to replace progress report in the status bar. UI with a realtime status of all GWE requests submitted using GSlicer3. | ** GWE requests monitoring module to replace progress report in the status bar. UI with a realtime status of all GWE requests submitted using GSlicer3. | ||
| + | |||
| + | == Links == | ||
| + | * GWE Core - [[Media:GWE-Core.ppt|Overview Presentation]] / [[Media:GWECore-Demo.mov|Video Demo]] | ||
| + | * GSlicer - [[Media:GWE-GSlicer3.ppt|Overview Presentation]] / [[Media:GSlicer3-Demo.mov|Video Demo]] | ||
Latest revision as of 14:52, 26 June 2008
Home < Project Week 2008 Special topic breakout: GWEBack to 2008 Summer Project Week Agenda
Contents
GWE Overview
- Distributed enterprise system, which leverages on clusters of computers, in order to provide a grid computing platform for end users to easily and effectively parallelize the execution of inter-independent processes.
- GWE Client (End User - You) <=== Secure RPC Network ===> GWE Daemon(s) (Cluster Head Node)
- Detailed information can be found on the GWE Project Site.
GWE Setup
Installation
- Download bundle, unbundle & set appropriate environment variables.
- Detailed information can be found on the GWE Installation Guide
Configuration
- Define your grid and provide your authentication information to access "grid resources".
- Detailed information can be found on the GWE Configuration Guide
Daemon Setup
- ONLY IF YOU ARE GOING TO ENHANCE A NEW CLUSTER WITH GWE CAPABILITIES (GWE Enabled).
- "Standalone GWE" will be available in the next release (To run GWE on your machine. Parallelization at the OS level, not distributed across cluster(s)).
- From the machine where you installed GWE, run the utility "gwe-util-daemon-fullsetup.sh". That's it!
- After that a "GWE Daemon" will be running in your cluster having your cluster user as this "GWE Daemon" administrator (different than roles in the cluster OS!).
- A "GWE Daemon" is just another application in a machine, therefore "GWE Daemon"s can run in a single machine, although it is not recommended.
- Detailed information can be found on the GWE Administrator Guide
GWE Clients
- GWE architecture provides multiple extensible modules, one of which is the "GWE Client API".
- Using the "GWE Client API", developers can create their own "GWE Client" applications.
- "GWE Client" applications are those; which allow end users to submit their "grid computing" requests to the GWE backend system.
- Currently, the GWE project team has created two "GWE Client" applications:
GWE Terminal
- Console based GWE client.
- Uses P2EL proprietary language to describe set of commands as permutation of templated commands with an associated parameter space.
- Many other monitoring/control commands available.
- Detailed information can be found on the End Users Guide
GSlicer3
- Slicer3 based GWE client ("GWE Powered" Slicer3).
- Basically a Slicer3 product enhanced with "GWE Client API" and custom development to provide "grid computing" capabilities to Slicer3.
- Setup: Download GSlicer3 bundle, unbundle it, set appropriate environmental variables.
- Config: Same as regular GWE.
- Detailed information can be found on the GSlicer3 Guide
Future Work
- Usual fixes, improvements, user support, documentation and project management.
- GWE Core
- "Standalone GWE" (requests parallelized as processes running in your machine - no cluster(s) needed).
- Resolve "Windows" issues (cygwin does not work as expected).
- Support for array type of iteration variables.
- Users application registry database and auto-deployment module to clusters.
- Daemon admin utilities (shutdown, restart, download DB, etc.).
- Order behavioral parameters (max concurrent jobs running, job execution timeout, launch mode, vfs clean up policy, pre-install applications, etc).
- Alert notification module (email, pager, etc).
- Scheduling module.
- Requests control: abort, pause, resume (under construction).
- Descriptive request result parsing. P2EL semantics + parser framework.
- Multi-cluster functionality. Currently only single cluster mode (not true grid computing yet). Pending: module to daisy chain "GWE Daemon"s.
- Single daemon services registry (Single IP Port/RMI registry) for multiple daemons.
- More file system drivers: GridFTP, SRB, XNAT/XCAT?, etc.
- More resource manager drivers: Torque, GWE, Cloud, etc.
- X.509 certificates support for authentication.
- Order chaining (revamping prototyped framework for chaining Slicer3 modules).
- MPI support.
- "GWE Powered" Portlets, FreeSurfer, FIPS, etc.
- JDSL support (standard submission protocol).
- JMX support (runtime diagnostics).
- Full feature statistical module.
- etc...
- GSlicer3:
- Centralized configuration.
- Usage of currently available specialized widgets (and maybe new GWE specialized widgets?).
- API to build specialized "GWE Powered" Slicer3 modules.
- Auto-deployment of Slicer3 in running clusters (as an extension of the users application auto-deployment module).
- GWE requests monitoring module to replace progress report in the status bar. UI with a realtime status of all GWE requests submitted using GSlicer3.
Links
- GWE Core - Overview Presentation / Video Demo
- GSlicer - Overview Presentation / Video Demo
