Technical Name AIAugSurgery: Key Technologies of Artificial Intelligent Surgical Auxiliary System
Project Operator National Taiwan University
Project Host 簡韶逸
Summary
Several key technologies for AIAugSurgery, an image guided surgery with artificial intelligenceaugmented reality, have been developed. For AIAugSurgery system, we develop a marker based flexible 6-DoF high precision camera pose estimation technique. We also propose an end-to-end deep model to perform single-image blind inpainting to remove the specular highlights for endoscope imaging. Moreover, for the computing platform of AIAugSurgery, we have developed processors for super-resolutionimage deblurring used for AR-aided surgical auxiliary systems. A high-throughput, energy-area-efficient CIM for neural network circuits for surgical information processing is also developed.
Scientific Breakthrough
Several break-through technologies for AIAugSurgery, an image guided surgery with artificial intelligenceaugmented reality, have been developed. For AIAugSurgery system, we develop the most flexible marker based 6-DoF camera pose estimation technique with 0.5mm in accuracyan end-to-end deep model for removing specular highlights in endoscope imaging. The normalized area efficiencyenergy efficiency of the proposed super-resolution processor chip is 3x6.5x more than the previous work. The proposed CIM SRAM design achieves 2.26×, 1.12×,2.89× performance improvements in throughput, energy,area efficiency, respectively, compared to the state-of-the-art designs.
Industrial Applicability
Several technologies for AIAugSurgery, an image guided surgery with artificial intelligenceaugmented reality, have been developed in this project. The development of high-performance image processing ICs, computing-in-memory circuits, 6-DoF marker based pose estimation, AI based image processing for endoscopic images, human-computer-interface with haptics enable to lay the foundation of mobile augmented reality. Not only be used for AR surgery systems, for the coming metaverse era, these technologies can also be employed as the key technologies for consumer metaversemedical metaverse.
Keyword Virtual reality (VR) and augmented reality (AR) Surgery Artificial intelligence (AI) Pose estimation Image processor architecture Energy-efficient integrated circuits Haptics Low-power neural-network circuit Computing-in-memory Head-mounted display
Notes
  • Contact
  • Shao-Yi Chien
other people also saw