Difference between revisions of "2016 Summer Project Week/Auditory Display"
From NAMIC Wiki
m (Text replacement - "[http://www.na-mic.org/Wiki/images/" to "[https://na-mic.org/w/images/") |
|||
| (11 intermediate revisions by 2 users not shown) | |||
| Line 9: | Line 9: | ||
*Christian Hansen, Uni Magdeburg | *Christian Hansen, Uni Magdeburg | ||
*Longquan Chen, BWH/HMS | *Longquan Chen, BWH/HMS | ||
| + | * Tamas Ungi | ||
| + | * Rocio Lopez | ||
| + | * Elvis Chen | ||
| + | *Julian Hettig | ||
==Related Work== | ==Related Work== | ||
| Line 16: | Line 20: | ||
*VIDEO: [https://www.dropbox.com/s/35rvm7kx30hwl08/WirbelBeispielMitAudio.mov?dl=0 for ablation needle insertion, with Univeristy of Magdeburg] | *VIDEO: [https://www.dropbox.com/s/35rvm7kx30hwl08/WirbelBeispielMitAudio.mov?dl=0 for ablation needle insertion, with Univeristy of Magdeburg] | ||
*VIDEO: [https://www.youtube.com/watch?v=gCg5nJSI2pY Auditory Display for Liver Resection] [http://www.ncbi.nlm.nih.gov/pubmed/23192891 Paper] | *VIDEO: [https://www.youtube.com/watch?v=gCg5nJSI2pY Auditory Display for Liver Resection] [http://www.ncbi.nlm.nih.gov/pubmed/23192891 Paper] | ||
| − | *[http://ioi.cs.uni-bremen.de/?page_id=780 Towards Uncertainty-Aware Auditory Display for Surgical Navigation (CARS 2016) | + | *[http://ioi.cs.uni-bremen.de/?page_id=780 Towards Uncertainty-Aware Auditory Display for Surgical Navigation (CARS 2016)] |
| − | *D. Black, J. Al Issawi, C. Rieder, and H. Hahn. Enhancing medical needle placement with auditory display. In Mensch & Computer 2013: Interaktive Vielfalt, pages 289–292, 2013. [ | + | *D. Black, J. Al Issawi, C. Rieder, and H. Hahn. Enhancing medical needle placement with auditory display. In Mensch & Computer 2013: Interaktive Vielfalt, pages 289–292, 2013. [https://na-mic.org/w/images/7/7d/Curac2013_dblack_Auditory_support_for_Navigated_Radiofrequency_Ablation_FINAL.pdf PDF ] |
==Possible application areas / IDEAS== | ==Possible application areas / IDEAS== | ||
| Line 25: | Line 29: | ||
* reduce complexity of displays by offloading to audio | * reduce complexity of displays by offloading to audio | ||
*Depth cues for 3D data (shown on 2D screens) | *Depth cues for 3D data (shown on 2D screens) | ||
| + | *coverage of needle in US | ||
==Project Description== | ==Project Description== | ||
| Line 37: | Line 42: | ||
<div style="width: 27%; float: left; padding-right: 3%;"> | <div style="width: 27%; float: left; padding-right: 3%;"> | ||
<h3>Approach, Plan</h3> | <h3>Approach, Plan</h3> | ||
| − | * | + | * common sound synthesis software: [https://puredata.info PureData], [https://cycling74.com/products/max/ Max], or [http://supercollider.github.io/ SuperCollider] DOWNLOAD PUREDATA EXTENDED>>> [https://puredata.info/downloads/pd-extended here] |
| − | * libraries for OpenSoundControl include [http://liblo.sourceforge.net/ liblo], [http://das.nasophon.de/pyliblo/ PyLiblo] | + | * Test program for Puredata to listen for incoming OSC messages: [http://wiki.na-mic.org/Wiki/images/8/81/Test_incoming_OSC.zip link] |
| + | * libraries for OpenSoundControl include C++: [http://liblo.sourceforge.net/ liblo], Python: [http://das.nasophon.de/pyliblo/ PyLiblo] | ||
| + | |||
</div> | </div> | ||
<div style="width: 27%; float: left; padding-right: 3%;"> | <div style="width: 27%; float: left; padding-right: 3%;"> | ||
<h3>Progress</h3> | <h3>Progress</h3> | ||
* Existing methods for intraoperative navigation guidance already implemented in [https://puredata.info Puredata], see [https://www.dropbox.com/s/6wfrklm8by4fx29/Dual-frequency-feedback.mov?dl=0 for ureteroscopy][http://wiki.na-mic.org/Wiki/index.php/File:Dual-frequency-feedback.mov download link], [https://www.dropbox.com/s/35rvm7kx30hwl08/WirbelBeispielMitAudio.mov?dl=0 for ablation needle insertion] | * Existing methods for intraoperative navigation guidance already implemented in [https://puredata.info Puredata], see [https://www.dropbox.com/s/6wfrklm8by4fx29/Dual-frequency-feedback.mov?dl=0 for ureteroscopy][http://wiki.na-mic.org/Wiki/index.php/File:Dual-frequency-feedback.mov download link], [https://www.dropbox.com/s/35rvm7kx30hwl08/WirbelBeispielMitAudio.mov?dl=0 for ablation needle insertion] | ||
| + | * Tried out iplementing liblo library for OSC messages, this workson Mac but not well on Windows. | ||
| + | * Switched to python-based library, which works well. Rocio tried this out but not enough time to integrate with their existing apps. | ||
| + | * Friendly contact between groups who are interested in working together (Rocio, Tamas, Elvis, Longquan) | ||
| − | [[Code from Jay Jagadeesan for OgenIGTLink]], [ | + | *[[Code from Jay Jagadeesan for OgenIGTLink]], [https://na-mic.org/w/images/3/3d/ReceiveClient.cxx.zip here as zip.] |
| − | + | *Only for Linux and Mac system, as the "lo" library has some build issue with visual studio. For windows user, it is recommended to use the python version of lo [http://das.nasophon.de/pyliblo/ Python library for lo] | |
| − | [http://www.na-mic.org/Wiki/ | + | *[http://www.na-mic.org/Wiki/index.php/File:AudioSimulator.zip A small lo simulator from Longquan Chen] |
| + | ** This simulator use the lo library, sends note to the local/remote server running PD-Extended. | ||
| + | ** Instructions about how to build and test the program is written in the readme.txt file | ||
</div> | </div> | ||
</div> | </div> | ||
Latest revision as of 18:27, 10 July 2017
Home < 2016 Summer Project Week < Auditory Display
Key Investigators
- David Black, University of Bremen, Fraunhofer MEVIS
- Sarah Frisken, BWH/HMS
- Christian Hansen, Uni Magdeburg
- Longquan Chen, BWH/HMS
- Tamas Ungi
- Rocio Lopez
- Elvis Chen
- Julian Hettig
Related Work
Existing methods for intraoperative navigation guidance already implemented in Puredata, see
- VIDEO: for ureteroscopy, with BWH/HMS and with Slicerdownload link
- VIDEO: for ablation needle insertion, with Univeristy of Magdeburg
- VIDEO: Auditory Display for Liver Resection Paper
- Towards Uncertainty-Aware Auditory Display for Surgical Navigation (CARS 2016)
- D. Black, J. Al Issawi, C. Rieder, and H. Hahn. Enhancing medical needle placement with auditory display. In Mensch & Computer 2013: Interaktive Vielfalt, pages 289–292, 2013. PDF
Possible application areas / IDEAS
- Acquiring 3D data sets for US, did we acquire all we need?
- Uncertainty in navigation information
- Brain / Structure Shift
- reduce complexity of displays by offloading to audio
- Depth cues for 3D data (shown on 2D screens)
- coverage of needle in US
Project Description
Objective
- Explore possibilities of auditory display for intraoperative use
- Find opportunities for extending existing projects with auditory display
Approach, Plan
Progress
- Existing methods for intraoperative navigation guidance already implemented in Puredata, see for ureteroscopydownload link, for ablation needle insertion
- Tried out iplementing liblo library for OSC messages, this workson Mac but not well on Windows.
- Switched to python-based library, which works well. Rocio tried this out but not enough time to integrate with their existing apps.
- Friendly contact between groups who are interested in working together (Rocio, Tamas, Elvis, Longquan)
- Code from Jay Jagadeesan for OgenIGTLink, here as zip.
- Only for Linux and Mac system, as the "lo" library has some build issue with visual studio. For windows user, it is recommended to use the python version of lo Python library for lo
- A small lo simulator from Longquan Chen
- This simulator use the lo library, sends note to the local/remote server running PD-Extended.
- Instructions about how to build and test the program is written in the readme.txt file
