Data:DownloadInstructions
Contents
Search for NAMIC Data
Users earch for data available on the BIRN data grid through metadata applied to the data itself. Each data object and collection can be assigned any number of metadata tags. Currently the metadata tags consist of items such as: Modality (DTI, T1, T2, etc.), Institution, etc.
To see an example of metadata on a data file, take a look at the following UCI file: First
- Ensure you aare logged into the Portal
- Then view the contents of that archive
- Then to search for data, click the Query Metadata button
Replication of Data
Currently, the NAMIC data sets are stored in BIRN at:
/home/Projects/NAMIC__0003/Files (physicall located at NCMIR)
/home/Projects/Study5000__0005/Files (physically located at UCI)
For redundancy in case a Birn Rack were to go offline we have replicated the data to the BWH rack in preparation for the Programmer's Week.
This can be seen when using 'Sls -a'
[jgerk@ncmir-gpop scratch]$ Sls -a UCI07110773.tgz jgerk 0 z-ucsd-ncmir-nas1 451315092 2005-02-18-14.00 % /home/Projects/Study5000__0005/Files/Archive/UCI07110773.tgz jgerk 1 z-harvard-bwh-nas1 451315092 2005-06-24-17.02 % /home/Projects/Study5000__0005/Files/Archive/UCI07110773.tgz
During regular use it is not important that the file resides in two places, but it should be understood that removing the file will result in removing both copies.
As more data is added to NAMIC we can periodically run the following command to update the BWH resource (provided you have the correct permissions).
Sbkupsrb -r -S z-harvard-bwh-nas1 Files
Notes
Instructions on creating Data Repository Pages are here.
- Data Access Uses the BIRN Controlled Infrastructure (requires account)
Namic Domain Creation
Please note that users who previously belonged to the external domain have been added to the namic domain. If you use the SRB S-commands, this mandates a change to your .MdasEnv file if you wish for your default directory to be /home/<your_user_name>.namic or if your default directory was previously /home/<your_user_name>.external. Please update your .MdasEnv file to point to your new default directory.
MIDAS
MIDAS is a public repository of freely available data. New contributions from NAMIC collaborators are welcome.
MIDAS is a system for collecting, processing, and distributing massive collections of data. It is particularly well suited for large images and their meta-data.
MIDAS provides the following services:
- Indexing: MIDAS automatically indexes text in any of over 20 different files types including XML, Microsoft Word(TM) and Adobe PDF(TM) files. Also, it automatically indexes header information from over 20 different image types including MetaImage and DICOM header information.
- Searching: MIDAS supports the OAI data harvesting standard, so that public data is readily discovered by Google and other search engines. Private data is readily searched using a local search engine.
- Storing: MIDAS provides batch upload methods and supports arbitrary file types and meta-data fields so that it can manage all of your raw and processed data as well as your companion reports and summary statistics.
- Processing: Local, server-side, and distributed processing can be automatically initiated on upload of data to MIDAS or initiated using thin or thick clients. Server-side and distributed processing is provided using the BatchMake scripting language.
- Accessing: You can browse large images on MIDAS, without downloading the data; you can download MIDAS's data over the web or directly into your legacy software using SSHFS. SSHFS presents the hierarchical organization of the data on MIDAS as a file system that can be mounted on Windows, Linux, and MacIntosh platforms.
MIDAS has been used to host publications for e-journals, to host and process data for multi-center research projects, and to offer public access to data in compliance with NIH and NSF data sharing policies.
