Note8

• How to specify requirements?
  • Is there something inbetween highly rigorous industrial practices and back of the envelope specifications?
  • synchronization of code and requirements
  • different scenarios, could be iterative process = no single requirement specification
    • often need refinement
    • ISO9000 lite
  • see FDA requirements
    • need method for reporting problems

• Should we collect and disseminate case studies?
  • Workshop?
  • record processes = best practices manual?
  • sell the development process...
• How do we develop guidelines for validation?
  • need mechanism to pool our results
  • testing: gathing information, evaluation metrics
  • case data costs money
  • uniform methods/tools for gathering data (e.g., OpenTracker)
  • see Nafis study for methodology for tracking validation - example of how to share methods?
  • NEED consensus on how to evaluate "correctness"
  • "Open Data", just like "Open Source" used to be?
  • suggestion: when designing IRBs, try to broaden a little bit so that the results can be used more widely or more effectively.
  • patient advocacy groups - mechanisms to make data available for patients to share if they wish,

• Do we need to develop better/additional hardware and software standards for IGT?
  • let the software be the driver by using "open source", at least initially
  • standards problem = no standards and too many standards simultaneously (standards, but not enough agreement) - this is probably not a research problem - where should it come from?
  • e.g., motivation for APIs exist for trackers (due to demand of customer), but for others (e.g., interventional imaging systems) there is less motivation for the manufacturer.
  • NIH play a role in funding industry to develop these standard interfaces (APIs?)
  • portability of standards is important (e.g., explicit support of legacy systems)
  • what is the value proposition for industry to invest into opening interfaces or developing standards?
    • there will always be competing standards - it is up to the marketplace which will win - where is the marketplace located in our community?
    • therapy is smaller niche than diagnostic, how to convince manufacturers of new methods
    • work with imaging people who also need to open up interfaces for new diagnostic methods
    • Chris Hasser: incentive to undustry - take advantage of resources, brain power that are being brought to bear by the research community - low cost and accelerated prototyping
    • shouldn't underestimate amount of engineering required to produce stable API
    • existing hooks exist in most vendor's equipment - until vendors agree on standards, community can contribute work to interface with these interfaces under research agreements,
    • Guy Schechter (Philips): these things are understood
      • need to take the lead and chrystallize the needs (leverage the power of the group)
      • but real power is buying power? research community is often weak!
      • critical mass (need to convince a few companies)
      • standards may need to be narrow in their respective areas in order to be feasible (broad standards will take too much time, or not happen at all)

• Develop algorithm repositories?
  • awareness - disseminate links to information about things like OpenTracker
  • participants of this workshop should add these links on the workshop wiki
  • perhaps recast: ask companies to permit us to provide a uniform interface, with having them be forced to comply with the interface.
  • hardware knowledge repositories should also be considered,
  • Open Source (1. freely available use) (2. distributed community of contributors)
    • should we choose prototype applications to drive an open interaction?
    • heirarchy of need - simple lab-based tests, versus complex assembly of many modules (IGT-lite and IGT-serious)
    • evolutionary process of developing open collaborations
  • PeterK - how about a forum for user evaluation and comments? (e.g., just like Amazon customer reviews)
    • Insight Journal could host such discussions. Forum for discussion or review.
      • what is the incentive for the review? danger in review bias.
      • won't this just end up evolving into peer review?
  • This community could come together to write requirements to give to industry - agree on common elements of functionality.
    • Larry Clarke: there is dialog between NIST and FDA on this topic
  • FDA/legal/regulatory impact on opening access to machine

• Database of simulators (see work in the MRI neuro-imaging world)

• Is there some need for focused study section in this area?
  • absence of reference standards for judging IGT research proposals
  • the community is its own worst enemy - study section
  • what role can NIH play?
  • new study section?
  • how many IGT proposals are actually going in.
    • there is a coding for diseases, but not for technologies, so this is difficult to track. keywords? this group could help define these keywords.
    • e.g., what should we call this community?
    • goal for breakouts: identify keywords