Difference between revisions of "2016 Winter Project Week/Projects/SlicerROSIntegration"
From NAMIC Wiki
| (28 intermediate revisions by 5 users not shown) | |||
| Line 2: | Line 2: | ||
<gallery> | <gallery> | ||
Image:PW-MIT2016.png|[[2016_Winter_Project_Week#Projects|Projects List]] | Image:PW-MIT2016.png|[[2016_Winter_Project_Week#Projects|Projects List]] | ||
| + | Image:2016WinterProjectWeek_SlicerROS_1.jpg|Our engineering team | ||
| + | Image:2016WinterProjectWeek_SlicerROS_2.jpg|Sharing polygon data between ROS (left) and 3D Slicer (right) | ||
| + | Image:2016WinterProjectWeek_SlicerROS_3.jpg|Our "robot" prototype | ||
</gallery> | </gallery> | ||
| + | |||
| + | Visit [https://goo.gl/photos/DTx6NwCtcZ673BfG9 our album] for more photos! | ||
==Key Investigators== | ==Key Investigators== | ||
| Line 8: | Line 13: | ||
* Axel Krieger (Children's National Medical Center) | * Axel Krieger (Children's National Medical Center) | ||
* Simon Leonard (Johns Hopkins University) | * Simon Leonard (Johns Hopkins University) | ||
| + | * Tobias Frank (University Hannover, Germany) | ||
| + | * Jayender Jagadeesan (BWH) | ||
| + | *Niravkumar Patel(Worcester Polytechnic Institute) | ||
| + | |||
| + | ==Project Description== | ||
| + | {| class="wikitable" | ||
| + | ! style="text-align: left; width:27%" | Objective | ||
| + | ! style="text-align: left; width:27%" | Approach and Plan | ||
| + | ! style="text-align: left; width:27%" | Progress and Next Steps | ||
| + | |- style="vertical-align:top;" | ||
| + | | | ||
| + | <!-- Objective bullet points --> | ||
| + | * Define requirements and system architecture for medical robotics software system based on 3D Slicer and Robot Operating System (ROS) | ||
| + | | | ||
| + | * Needs for 3D Slicer / ROS integration in ongoing research projects (presentations by participants) (See [[2016_Winter_Project_Week/Breakout_Sessions/SlicerForMedicalRoboticsResearch|Breakout session]]) | ||
| + | ** Axel (Autonomous Surgery using the KUKA LWR) | ||
| + | ** Simon (dVRK?) | ||
| + | ** Junichi (OpenIGTLink and medical robotics research) | ||
| + | ** Tobias (OCT robot / Integration of KUKA robot and 3D Slicer | ||
| + | * Brainstorming 1: Requirements | ||
| + | ** Applications (e.g. endoscopic surgery, percutaneous interventions, catheterization, etc.) | ||
| + | ** Platforms -- Linux, Windows, MAC | ||
| + | ** Roles for 3D Slicer -- visualization, image processing, etc. | ||
| + | ** Roles for ROS -- vision, sensors, devices, etc. | ||
| + | * Brainstorming 2: Architecture for 3D Slicer-ROS integration | ||
| + | ** Types of data exchanged between ROS and 3D Slicer | ||
| + | ** Communication scheme between ROS and 3D Slicer | ||
| + | ** Software package to provide | ||
| + | *** Independent middleware? | ||
| + | *** 3D Slicer plug-in modules | ||
| + | *** ROS modules | ||
| + | * Brainstorming 3: Collaborative tools / teams | ||
| + | ** Platforms to support (ROS mainly support Linux) | ||
| + | ** Repository | ||
| + | | | ||
| + | * Considered the following clinical scenario: | ||
| + | ** Obtain preoperative 3D image of the patient | ||
| + | ** Create 3D surface model of the patient from the 3D image on 3D Slicer | ||
| + | ** Set up the patient on the OR table | ||
| + | ** Scan the patient with a surface scanner. The point cloud data is imported to ROS (NOTE: This can be 3D Slicer, and then transferred to ROS through OpenIGTLink) | ||
| + | ** 3D Slicer send the 3D surface model to ROS through OpenIGTLink as POLYDATA | ||
| + | ** Perform surface matching on ROS and send the result registration transform to 3D Slicer | ||
| + | ** Define target on the original image (or the model) on 3D Slicer | ||
| + | ** Send the target to ROS | ||
| + | ** Move the robot to the target | ||
| + | * Achievements: | ||
| + | ** ROS-OpenIGTLink interface to synchronize data between Slicer and ROS including | ||
| + | *** Points | ||
| + | *** Transforms | ||
| + | *** Polydata | ||
| + | *** Image | ||
| + | ** Installed Debian Linux for Lego Mindstroms ([http://www.ev3dev.org Linux for EV3 ]) | ||
| + | ** Installed ROS on Ev3 Linux | ||
| + | ** Built printer robot | ||
| + | ** Implemented a proof-of-concept system using LEGO MINDSTORMS and ROS. | ||
| + | *** [[File:WinterProjectWeek_2016_ROS_Slicer_Integration_ProjectOutcomePresentation.pptx]] | ||
| + | |} | ||
| − | = | + | =Future Plan= |
| − | + | * Immediate action items | |
| − | + | ** Upload all software components to Github | |
| − | + | ** Create a complete tutorial | |
| − | * | + | * Generalize ROS-Slicer bridge |
| − | + | ** Synchronize MRML node with ROS message | |
| − | + | ** Support | |
| − | + | *** Transform (4x4 matrix and quaternion) | |
| − | * | + | *** Image (2D / 3D) |
| − | + | *** Polygon / point cloud | |
| − | + | ** Research application | |
| − | |||
| − | * | ||
| − | |||
| − | |||
Latest revision as of 14:53, 8 January 2016
Home < 2016 Winter Project Week < Projects < SlicerROSIntegration
Our engineering team
Sharing polygon data between ROS (left) and 3D Slicer (right)
Our "robot" prototype
Visit our album for more photos!
Key Investigators
- Junichi Tokuda (Brigham and Women's Hospital)
- Axel Krieger (Children's National Medical Center)
- Simon Leonard (Johns Hopkins University)
- Tobias Frank (University Hannover, Germany)
- Jayender Jagadeesan (BWH)
- Niravkumar Patel(Worcester Polytechnic Institute)
Project Description
| Objective | Approach and Plan | Progress and Next Steps |
|---|---|---|
|
|
|
Future Plan
- Immediate action items
- Upload all software components to Github
- Create a complete tutorial
- Generalize ROS-Slicer bridge
- Synchronize MRML node with ROS message
- Support
- Transform (4x4 matrix and quaternion)
- Image (2D / 3D)
- Polygon / point cloud
- Research application
