Difference between revisions of "2016 Winter Project Week/Projects/SlicerROSIntegration"

From NAMIC Wiki
Jump to: navigation, search
 
(25 intermediate revisions by 5 users not shown)
Line 2: Line 2:
 
<gallery>
 
<gallery>
 
Image:PW-MIT2016.png|[[2016_Winter_Project_Week#Projects|Projects List]]
 
Image:PW-MIT2016.png|[[2016_Winter_Project_Week#Projects|Projects List]]
 +
Image:2016WinterProjectWeek_SlicerROS_1.jpg|Our engineering team
 +
Image:2016WinterProjectWeek_SlicerROS_2.jpg|Sharing polygon data between ROS (left) and 3D Slicer (right)
 +
Image:2016WinterProjectWeek_SlicerROS_3.jpg|Our "robot" prototype
 
</gallery>
 
</gallery>
 +
 +
Visit [https://goo.gl/photos/DTx6NwCtcZ673BfG9 our album] for more photos!
  
 
==Key Investigators==
 
==Key Investigators==
Line 8: Line 13:
 
* Axel Krieger (Children's National Medical Center)
 
* Axel Krieger (Children's National Medical Center)
 
* Simon Leonard (Johns Hopkins University)
 
* Simon Leonard (Johns Hopkins University)
−
 
+
* Tobias Frank (University Hannover, Germany)
 +
* Jayender Jagadeesan (BWH)
 +
*Niravkumar Patel(Worcester Polytechnic Institute)
  
 
==Project Description==
 
==Project Description==
−
<div style="margin: 20px;">
+
{| class="wikitable"
−
<div style="width: 27%; float: left; padding-right: 3%;">
+
! style="text-align: left; width:27%" |  Objective
−
<h3>Objective</h3>
+
! style="text-align: left; width:27%" |  Approach and Plan
 +
! style="text-align: left; width:27%" |  Progress and Next Steps
 +
|- style="vertical-align:top;"
 +
|
 +
<!-- Objective bullet points -->
 
* Define requirements and system architecture for medical robotics software system based on 3D Slicer and Robot Operating System (ROS)
 
* Define requirements and system architecture for medical robotics software system based on 3D Slicer and Robot Operating System (ROS)
−
 
+
|
−
</div>
+
* Needs for 3D Slicer / ROS integration in ongoing research projects (presentations by participants) (See [[2016_Winter_Project_Week/Breakout_Sessions/SlicerForMedicalRoboticsResearch|Breakout session]])
−
<div style="width: 27%; float: left; padding-right: 3%;">
+
** Axel (Autonomous Surgery using the KUKA LWR)
−
<h3>Approach, Plan</h3>
 
−
* Needs for 3D Slicer / ROS integration in ongoing research projects (presentations by participants)
 
−
** Axel (Suturing robot)
 
 
** Simon (dVRK?)
 
** Simon (dVRK?)
 
** Junichi (OpenIGTLink and medical robotics research)
 
** Junichi (OpenIGTLink and medical robotics research)
Line 26: Line 34:
 
* Brainstorming 1: Requirements
 
* Brainstorming 1: Requirements
 
** Applications (e.g. endoscopic surgery, percutaneous interventions, catheterization, etc.)
 
** Applications (e.g. endoscopic surgery, percutaneous interventions, catheterization, etc.)
 +
** Platforms -- Linux, Windows, MAC
 
** Roles for 3D Slicer -- visualization, image processing, etc.
 
** Roles for 3D Slicer -- visualization, image processing, etc.
 
** Roles for ROS -- vision, sensors, devices, etc.
 
** Roles for ROS -- vision, sensors, devices, etc.
Line 38: Line 47:
 
** Platforms to support (ROS mainly support Linux)
 
** Platforms to support (ROS mainly support Linux)
 
** Repository
 
** Repository
 +
|
 +
* Considered the following clinical scenario:
 +
** Obtain preoperative 3D image of the patient
 +
** Create 3D surface model of the patient from the 3D image on 3D Slicer
 +
** Set up the patient on the OR table
 +
** Scan the patient with a surface scanner. The point cloud data is imported to ROS (NOTE: This can be 3D Slicer, and then transferred to ROS through OpenIGTLink)
 +
** 3D Slicer send the 3D surface model to ROS through OpenIGTLink as POLYDATA
 +
** Perform surface matching on ROS and send the result registration transform to 3D Slicer
 +
** Define target on the original image (or the model) on 3D Slicer
 +
** Send the target to ROS
 +
** Move the robot to the target
 +
* Achievements:
 +
** ROS-OpenIGTLink interface to synchronize data between Slicer and ROS including
 +
*** Points
 +
*** Transforms
 +
*** Polydata
 +
*** Image
 +
** Installed Debian Linux for Lego Mindstroms ([http://www.ev3dev.org Linux for EV3 ])
 +
** Installed ROS on Ev3 Linux
 +
** Built printer robot
 +
** Implemented a proof-of-concept system using LEGO MINDSTORMS and ROS.
 +
*** [[File:WinterProjectWeek_2016_ROS_Slicer_Integration_ProjectOutcomePresentation.pptx]]
 +
|}
  
  
−
 
+
=Future Plan=
−
</div>
+
* Immediate action items
−
<div style="width: 27%; float: left; padding-right: 3%;">
+
** Upload all software components to Github
−
<h3>Progress</h3>
+
** Create a complete tutorial
−
*
+
* Generalize ROS-Slicer bridge
−
</div>
+
** Synchronize MRML node with ROS message
−
</div>
+
** Support
 +
*** Transform (4x4 matrix and quaternion)
 +
*** Image (2D / 3D)
 +
*** Polygon / point cloud
 +
** Research application

Latest revision as of 14:53, 8 January 2016

Home < 2016 Winter Project Week < Projects < SlicerROSIntegration

  • Our engineering team

  • Sharing polygon data between ROS (left) and 3D Slicer (right)

  • Our "robot" prototype

Visit our album for more photos!

Key Investigators

  • Junichi Tokuda (Brigham and Women's Hospital)
  • Axel Krieger (Children's National Medical Center)
  • Simon Leonard (Johns Hopkins University)
  • Tobias Frank (University Hannover, Germany)
  • Jayender Jagadeesan (BWH)
  • Niravkumar Patel(Worcester Polytechnic Institute)

Project Description

Objective Approach and Plan Progress and Next Steps
  • Define requirements and system architecture for medical robotics software system based on 3D Slicer and Robot Operating System (ROS)
  • Needs for 3D Slicer / ROS integration in ongoing research projects (presentations by participants) (See Breakout session)
    • Axel (Autonomous Surgery using the KUKA LWR)
    • Simon (dVRK?)
    • Junichi (OpenIGTLink and medical robotics research)
    • Tobias (OCT robot / Integration of KUKA robot and 3D Slicer
  • Brainstorming 1: Requirements
    • Applications (e.g. endoscopic surgery, percutaneous interventions, catheterization, etc.)
    • Platforms -- Linux, Windows, MAC
    • Roles for 3D Slicer -- visualization, image processing, etc.
    • Roles for ROS -- vision, sensors, devices, etc.
  • Brainstorming 2: Architecture for 3D Slicer-ROS integration
    • Types of data exchanged between ROS and 3D Slicer
    • Communication scheme between ROS and 3D Slicer
    • Software package to provide
      • Independent middleware?
      • 3D Slicer plug-in modules
      • ROS modules
  • Brainstorming 3: Collaborative tools / teams
    • Platforms to support (ROS mainly support Linux)
    • Repository
  • Considered the following clinical scenario:
    • Obtain preoperative 3D image of the patient
    • Create 3D surface model of the patient from the 3D image on 3D Slicer
    • Set up the patient on the OR table
    • Scan the patient with a surface scanner. The point cloud data is imported to ROS (NOTE: This can be 3D Slicer, and then transferred to ROS through OpenIGTLink)
    • 3D Slicer send the 3D surface model to ROS through OpenIGTLink as POLYDATA
    • Perform surface matching on ROS and send the result registration transform to 3D Slicer
    • Define target on the original image (or the model) on 3D Slicer
    • Send the target to ROS
    • Move the robot to the target
  • Achievements:


Future Plan

  • Immediate action items
    • Upload all software components to Github
    • Create a complete tutorial
  • Generalize ROS-Slicer bridge
    • Synchronize MRML node with ROS message
    • Support
      • Transform (4x4 matrix and quaternion)
      • Image (2D / 3D)
      • Polygon / point cloud
    • Research application
Retrieved from "https://www.na-mic.org/w/index.php?title=2016_Winter_Project_Week/Projects/SlicerROSIntegration&oldid=92008"