2016 Winter Project Week/Projects/SlicerROSIntegration

• 

  Projects List

• 

  Our engineering team

• 

  Sharing polygon data between ROS (left) and 3D Slicer (right)

• 

  Our "robot" prototype

Visit our album for more photos!

Key Investigators

• Junichi Tokuda (Brigham and Women's Hospital)
• Axel Krieger (Children's National Medical Center)
• Simon Leonard (Johns Hopkins University)
• Tobias Frank (University Hannover, Germany)
• Jayender Jagadeesan (BWH)
• Niravkumar Patel(Worcester Polytechnic Institute)

Project Description

Objective	Approach and Plan	Progress and Next Steps
Define requirements and system architecture for medical robotics software system based on 3D Slicer and Robot Operating System (ROS)	Needs for 3D Slicer / ROS integration in ongoing research projects (presentations by participants) (See Breakout session) Axel (Autonomous Surgery using the KUKA LWR) Simon (dVRK?) Junichi (OpenIGTLink and medical robotics research) Tobias (OCT robot / Integration of KUKA robot and 3D Slicer Brainstorming 1: Requirements Applications (e.g. endoscopic surgery, percutaneous interventions, catheterization, etc.) Platforms -- Linux, Windows, MAC Roles for 3D Slicer -- visualization, image processing, etc. Roles for ROS -- vision, sensors, devices, etc. Brainstorming 2: Architecture for 3D Slicer-ROS integration Types of data exchanged between ROS and 3D Slicer Communication scheme between ROS and 3D Slicer Software package to provide Independent middleware? 3D Slicer plug-in modules ROS modules Brainstorming 3: Collaborative tools / teams Platforms to support (ROS mainly support Linux) Repository	Considered the following clinical scenario: Obtain preoperative 3D image of the patient Create 3D surface model of the patient from the 3D image on 3D Slicer Set up the patient on the OR table Scan the patient with a surface scanner. The point cloud data is imported to ROS (NOTE: This can be 3D Slicer, and then transferred to ROS through OpenIGTLink) 3D Slicer send the 3D surface model to ROS through OpenIGTLink as POLYDATA Perform surface matching on ROS and send the result registration transform to 3D Slicer Define target on the original image (or the model) on 3D Slicer Send the target to ROS Move the robot to the target Achievements: ROS-OpenIGTLink interface to synchronize data between Slicer and ROS including Points Transforms Polydata Image Installed Debian Linux for Lego Mindstroms (Linux for EV3 ) Installed ROS on Ev3 Linux Built printer robot Implemented a proof-of-concept system using LEGO MINDSTORMS and ROS. File:WinterProjectWeek 2016 ROS Slicer Integration ProjectOutcomePresentation.pptx

Future Plan

• Immediate action items
  • Upload all software components to Github
  • Create a complete tutorial
• Generalize ROS-Slicer bridge
  • Synchronize MRML node with ROS message
  • Support
    • Transform (4x4 matrix and quaternion)
    • Image (2D / 3D)
    • Polygon / point cloud
  • Research application