2010-Slicer-Factory

Back to 2010 09 Leadership Brainstorming

Introduction

• A new compile and debug setup will be created for Slicer under the URL factory.slicer.org
• OS: Virtual machines for OS X, win 32, win 64, linux 32, linux 64
• A variety of compilers
• Backups

Setup

• Located on the BIRN network
• Managed by the NA-MIC service core
  • OS, Virtual Machines, Compilers, Backups, Monitoring

Initial Plan

(1) get a big mac pro (12 core, 32G, SSD RAID, external backup) and put it on the network at 1249 Boylston, outside the firewall so any of us can access it as factory.slicer.org via ssh and/or apple desktop sharing / vnc. This machine will need to be monitored about weekly for OS patches etc. This host would primarily just be a server for virtual machines, not a workstation per se.

(2) install a vm server (vmware?, parallels? virtual box?) to host all the OS versions we want to build on and test on: mac osx, windows 32 and 64, linux 32 and 64. Exact OS versions to use open to discussion. Each of these virtual machines will need to be monitored about weekly for OS patches, etc.

(3) set up continuous and nightly tests for each part of the NA-MIC Kit on each OS (cmake, vtk, itk, teem, ctk, slicer...). Some will need to be the 'owner' of these test machines to diagnose dashboard issues. We may also want multiple versions of each toolkit. I think if we have a beefy enough mac, we should be able to do all this on one machine but if needed we could have more than one factory machine.

(4) create slicer binary installers for supported platforms and copy them to the appropriate spot on slicer.org for download. Someone will need to be ready to troubleshoot if builds are not available.

I think this is all fairly straightforward -- but it will be a fair amount of work. Hopefully it will be pretty automatic once it is set up, but it will need to be carefully documented and experts will need to be available for troubleshooting.

Extensions

• We should also have a plan for the extensions and the extension server.
• Proposal:
  • New e-journal is created to host Slicer modules, data, and tutorials
  • Each submission can consist of one or more of the following:
    • Slides, dlls, python code, executables, C++ code, etc.
  • MIDAS performs nightly builds and tests appropriate submissions and returns results as dashboards
  • Server is indexed by Google Scholar, etc using OAI standards
  • Server provides revision control for all uploaded items
  • Server allows the author to specify the build environment for the uploaded items
    • e.g., version of Slicer, ITK, ... to be used
  • API is provided for querying server from within Slicer
  • Slicer modules provide one-click download of tutorials, testing data, etc
  • Module testing data is downloaded if a module is tested, as part of the testing process (developers need not search for testing data).

Policies

• Limited number of root users who communicate and record all modifications to the system
• Dashboard scripts are stored in a repository (svn?) and fetched from that repo every night
• No machine-specific configurations outside of dashboard scripts are done beyond standard package installs
• Dashboard logins do not have root access
• Wiki pages are used to track modifications to machines, publicize who is responsible for each dashboard script, etc.
• Machine systems admin is not necessarily the person responsible for any dashboard script on the machine
• Dashboard script maintainers must subscribe to get emails on all dashboard failures for the corresponding project