Home < 2016 Winter Project Week < Breakout Sessions < SlicerROSIntegration
Sharing polygon data between ROS (left) and 3D Slicer (right)
Visit our album for more photos!
Key Investigators
- Junichi Tokuda (Brigham and Women's Hospital)
- Axel Krieger (Children's National Medical Center)
- Simon Leonard (Johns Hopkins University)
- Tobias Frank (University Hannover, Germany)
- Jayender Jagadeesan (BWH)
- Niravkumar Patel(Worcester Polytechnic Institute)
Project Description
| Objective
|
Approach and Plan
|
Progress and Next Steps
|
- Define requirements and system architecture for medical robotics software system based on 3D Slicer and Robot Operating System (ROS)
|
- Needs for 3D Slicer / ROS integration in ongoing research projects (presentations by participants) (See Breakout session)
- Axel (Autonomous Surgery using the KUKA LWR)
- Simon (dVRK?)
- Junichi (OpenIGTLink and medical robotics research)
- Tobias (OCT robot / Integration of KUKA robot and 3D Slicer
- Brainstorming 1: Requirements
- Applications (e.g. endoscopic surgery, percutaneous interventions, catheterization, etc.)
- Platforms -- Linux, Windows, MAC
- Roles for 3D Slicer -- visualization, image processing, etc.
- Roles for ROS -- vision, sensors, devices, etc.
- Brainstorming 2: Architecture for 3D Slicer-ROS integration
- Types of data exchanged between ROS and 3D Slicer
- Communication scheme between ROS and 3D Slicer
- Software package to provide
- Independent middleware?
- 3D Slicer plug-in modules
- ROS modules
- Brainstorming 3: Collaborative tools / teams
- Platforms to support (ROS mainly support Linux)
- Repository
|
- Considered the following clinical scenario:
- Obtain preoperative 3D image of the patient
- Create 3D surface model of the patient from the 3D image on 3D Slicer
- Set up the patient on the OR table
- Scan the patient with a surface scanner. The point cloud data is imported to ROS (NOTE: This can be 3D Slicer, and then transferred to ROS through OpenIGTLink)
- 3D Slicer send the 3D surface model to ROS through OpenIGTLink as POLYDATA
- Perform surface matching on ROS and send the result registration transform to 3D Slicer
- Define target on the original image (or the model) on 3D Slicer
- Send the target to ROS
- Move the robot to the target
- Achievements:
- ROS-OpenIGTLink interface to synchronize data between Slicer and ROS including
- Points
- Transforms
- Polydata
- Image
- Installed Debian Linux for Lego Mindstroms (Linux for EV3 )
- Installed ROS on Ev3 Linux
- Built printer robot
- Implemented a proof-of-concept system using LEGO MINDSTORMS and ROS.
|
Future Plan
- Immediate action items
- Upload all software components to Github
- Create a complete tutorial
- Generalize ROS-Slicer bridge
- Synchronize MRML node with ROS message
- Support
- Transform (4x4 matrix and quaternion)
- Image (2D / 3D)
- Polygon / point cloud
- Research application
