Engineering:UCSD
Contents
UCSD Progress (in preparation for SLC AHM)
Overview
GWE Background
The field of high performance computing (HPC) has provided a wide array of strategies for supplying additional computing power to the goal of reducing the total “clock time” required to complete large scale analyses. These strategies range from the development of higher performance hardware to the assembly of large networks of commodity computers. However, for the non-computational scientist wishing to utilize these services, usable software remains elusive. Here we present a software design and implementation of a tool, Grid Wizard Enterprise (GWE; http://www.gridwizardenterprise.org/), aimed at providing a solution to the particular problem of the adoption of advanced grid technologies by biomedical researchers. GWE provides an intuitive environment and tools that bridge this gulf between the researcher and current grid technologies allowing them to run inter-independent computational processes faster by brokering their execution across a virtual grid of computational resources with a minimum of user intervention. The GWE architecture has been designed in close collaboration with NA-MIC researchers and supports the majority of every-day tasks performed by computational scientists in the fields of computational biology and medical image analysis.
Monthly Progress (since July 2007)
July - September 2007
- Inception, analysis, architecture, design and implementation of Grid Wizard Enterprise (GWE).
October 2007
- Presentation and live demo of first GWE prototype at BIRN AHM. (PowerPoint / PDF)
December 2007
- Launch of GWE's:
January 2008
- Unit tests of GWE's first release candidate components.
- Internal pre-release of first version of GWE.
- Release of GWE guides, technical details and collaboration tools in the GWE project site.
February 2008
- First release of GWE (version 0.6.alpha). See its 'features' page for details.
March 2008
- GWE version 0.6.1.alpha released. See its release notes for details.
- GSlicer3 integration effort released. (Overview Presentation / Video Demo / Documentation @ GWE Project Site / Documentation @ NAMIC Wiki Site).
- Presentation of GWE and GSlicer3 at the 2008 Engineering review at Utah.
April 2008
- GWE version 0.6.2.alpha released. See its release notes for details.
- Wrote paper "Simplifying the Utilization of Grid Computation using Grid Wizard Enterprise" to be submitted to the 'MICCAI Grid Workshop'.
May 2008
- Inception, design and implementation of the 'Chain Modules Engine' (a project different from GWE and which will be worked in parallel):
June 2008
- GWE version 0.6.3.alpha released. See its release notes for details.
- NAMIC summer week presentations and projects:
July 2008
- Two extra rounds of edition of the paper submitted to the 'MICCAI Grid Workshop'. Paper was accepted and scheduled for a long presentation in September.
August 2008
- GWE version 0.6.4.alpha released. See its release notes for details.
September 2008
- Presented the paper "Simplifying the Utilization of Grid Computation using Grid Wizard Enterprise" to the 'MICCAI Grid Workshop' in New York.
October 2008
- Collected user requests at the BIRN AHM.
November 2008
- GWE version 0.7.0.alpha released. See its release notes for details.
- Major re-architecture to allow running GWE without clusters and with daemons per user.
- End to end study case to query and process images from remote OASIS database.
- Step by step tutorials:
December 2008
- GWE version 0.7.1.alpha released. See its release notes for details.
- New web client application (GWE Web Control Panel).
- Infrastructure to support integration of custom XML based catalogs to define experiments in preparation for the "GWE integration with catalog files" project (NAMIC Winter Project Week 2009)
- Macro definitions to ease the description of medical imaging processes and Slicer3 usage.
- Step by step tutorials:
January 2009
- NAMIC Winter Project Week and AHM 2009.
- "GWE integration with catalog files" project at the NAMIC Winter Project Week 2009.
Miscellaneous
- Weekly NAMIC engineering teleconferences.
- GWE testbed and user's support.
Dissemination Activities Prior to Monthly Progress Outlined Above
- Introductory meeting and demonstration with Tina Kapur & Birn-CC.
- Hosted NAMIC dissemination event (February 17-18).
- Taught Data Grid course at UCSD dissemination event.
- Attended NAMIC dissemination event.
- Attended SLC AHM.
- Instructions available for deploying a "tunneled" SRB server.
Infrastructure Prior to Monthly Progress Outlined Above
- Researched, tested and deployed a newly configured SRB server for NAMIC that allows for the tunneling of all SRB commands via SSH. This tunneling has been tested with the command line (SCommands), Java (JARGON) and Windows (InQ) clients. The current staging server is running at UCSD and is available for testing.
- A server co-located at BWH will be discussed at the AHM.
- Leveraged BWH BIRN Rack to provide gigabit connection for na-mic.org.
- Researching and developing a parallel system for backend parallel processing of Slicer3 algorithms
- Discussed the use of BatchMake for submitting grid-like jobs to a Condor pool
Data Sharing Prior to Monthly Progress Outlined Above
- Hosting NAMIC data on data grid accessible to all NAMIC participants.
- Providing data grid and Portal support to all NAMIC participants.
- Provided custom project space for NAMIC in BIRN Portal.
- Provided account generation for first batch of NAMIC users. New users can now utilize the new account request feature in the Portal.
- Working with Isomics to assist core 3 sites in their data uploads.
- Provided template data hierarchy constructs and integration of hierarchy in data grid.
- Provided statistics to Tina Kapur on account creation, number of uploaded data sets, and audit information.
