Catalogue Search | MBRL
Are you sure you want to remove the book from the shelf?
{{itemTitle}}
1,111
result(s) for
"real-time rendering"
Sort by:
ARssist: augmented reality on a head-mounted display for the first assistant in robotic surgery
In robot-assisted laparoscopic surgery, the first assistant (FA) is responsible for tasks such as robot docking, passing necessary materials, manipulating hand-held instruments, and helping with trocar planning and placement. The performance of the FA is critical for the outcome of the surgery. The authors introduce ARssist, an augmented reality application based on an optical see-through head-mounted display, to help the FA perform these tasks. ARssist offers (i) real-time three-dimensional rendering of the robotic instruments, hand-held instruments, and endoscope based on a hybrid tracking scheme and (ii) real-time stereo endoscopy that is configurable to suit the FA's hand–eye coordination when operating based on endoscopy feedback. ARssist has the potential to help the FA perform his/her task more efficiently, and hence improve the outcome of robot-assisted laparoscopic surgeries.
Journal Article
Foveated rendering: A state-of-the-art survey
Recently, virtual reality (VR) technology has been widely used in medical, military, manufacturing, entertainment, and other fields. These applications must simulate different complex material surfaces, various dynamic objects, and complex physical phenomena, increasing the complexity of VR scenes. Current computing devices cannot efficiently render these complex scenes in real time, and delayed rendering makes the content observed by the user inconsistent with the user’s interaction, causing discomfort. Foveated rendering is a promising technique that can accelerate rendering. It takes advantage of human eyes’ inherent features and renders different regions with different qualities without sacrificing perceived visual quality. Foveated rendering research has a history of 31 years and is mainly focused on solving the following three problems. The first is to apply perceptual models of the human visual system into foveated rendering. The second is to render the image with different qualities according to foveation principles. The third is to integrate foveated rendering into existing rendering paradigms to improve rendering performance. In this survey, we review foveated rendering research from 1990 to 2021. We first revisit the visual perceptual models related to foveated rendering. Subsequently, we propose a new foveated rendering taxonomy and then classify and review the research on this basis. Finally, we discuss potential opportunities and open questions in the foveated rendering field. We anticipate that this survey will provide new researchers with a high-level overview of the state-of-the-art in this field, furnish experts with up-to-date information, and offer ideas alongside a framework to VR display software and hardware designers and engineers.
Journal Article
Polynomial for real-time rendering of neural radiance fields
In neural radiance fields (NeRF), generating highly realistic rendering results requires extensive sampling of rays and online query of multilayer perceptrons. However, this results in slow rendering speeds. Previous research has addressed this issue by designing faster evaluation of neural scene representations or precomputing scene properties to reduce rendering time. In this paper, we propose a real-time rendering method called PNeRF. PNeRF utilizes continuous polynomial functions to approximate spatial volume density and color information. Additionally, we separate the view direction information from the rendering equation, leading to a new expression for the volume rendering equation. By taking the starting coordinates of the observation viewpoint and the observation direction vector as inputs to the neural network, we obtain the rendering result for the corresponding observation ray. Thus, the rendering for each ray only requires a single forward inference of the neural network. To further improve rendering speed, we design a six-axis spherical method to store the rendering results corresponding to the starting coordinates of the observation viewpoint and the observation direction vector. This allows us to significantly improve the rendering speed and maintain the rendering quality, with minimal storage space requirements. Experimental validation on LLFF datasets demonstrates that our method improves rendering speed while preserving rendering quality and requiring minimal storage space. These results indicate the potential of our method in the real-time rendering field, providing an effective solution for more efficient rendering.
Journal Article
A Photogrammetry-Based Workflow for the Accurate 3D Construction and Visualization of Museums Assets
Nowadays digital replicas of artefacts belonging to the Cultural Heritage (CH) are one of the most promising innovations for museums exhibitions, since they foster new forms of interaction with collections, at different scales. However, practical digitization is still a complex task dedicated to specialized operators. Due to these premises, this paper introduces a novel approach to support non-experts working in museums with robust, easy-to-use workflows based on low-cost widespread devices, aimed at the study, classification, preservation, communication and restoration of CH artefacts. The proposed methodology introduces an automated combination of acquisition, based on mobile equipment and visualization, based on Real-Time Rendering. After the description of devices used along the workflow, the paper focuses on image pre-processing and geometry processing techniques adopted to generate accurate 3D models from photographs. Assessment criteria for the developed process evaluation are illustrated. Tests of the methodology on some effective museum case studies are presented and discussed.
Journal Article
Robust Gaussian Splatting via Depth Reliability Masking and Edge-Aware Optimization
3D Gaussian Splatting (3DGS) achieves real-time, high-fidelity rendering through explicit Gaussian primitives and efficient rasterization techniques. However, the absence of geometric information often leads to artifacts around object edges and weakly textured regions. Although existing methods attempt to optimize geometric representation by imposing depth constraints, their efficacy remains limited due to interference from pervasive sensor noise. To address this, we propose a novel optimization framework integrating edge-aware mechanisms with depth reliability detection. Specifically, our approach employs multi-scale local depth statistics and gradient information to strategically exclude depth loss computations in ambiguous background edge regions. Simultaneously, it utilizes neighborhood depth consistency to construct a robust reliability mask that actively suppresses the influence of depth outliers. Experiments on the TUM-RGBD dataset demonstrate that our method significantly mitigates blurring and visual artifacts while improving the evaluation metrics holistically.
Journal Article
Neural Scene Baking for Permutation Invariant Transparency Rendering with Real-Time Global Illumination
Neural rendering provides a fundamentally new way to render photorealistic images. Similar to traditional light-baking methods, neural rendering utilizes neural networks to bake representations of scenes, materials, and lights into latent vectors learned from path-tracing ground truths. However, existing neural rendering algorithms typically use G-buffers to provide position, normal, and texture information about scenes. These are prone to occlusion by transparent surfaces, leading to distortion and loss of detail in rendered images. To address this limitation, we propose a novel neural rendering pipeline that accurately renders the scene behind transparent surfaces with global illumination and variable scenes. Our method separates the G-buffers for opaque and transparent objects, retaining G-buffer information behind transparent objects. Additionally, to render transparent objects with permutation invariance, we have designed a new permutation-invariant neural blending function. We have integrated our algorithm into an efficient custom renderer, achieving real-time performance. Our results show that our method is capable of rendering photorealistic images for variable scenes and viewpoints, accurately capturing complex transparent structures along with global illumination. Our renderer can achieve real-time performance (256 × 256 at 63 frames/s and 512 × 512 at 32 frames/s) for scenes with multiple variable transparent objects.
Journal Article
Scene reconstruction techniques for autonomous driving: a review of 3D Gaussian splatting
As the latest research result of the explicit radiated field technology, 3D Gaussian Splatting (3D GS) replaces the implicit expression represented by Neural Radiated Field (NeRF) and has become the hottest research direction in 3D scene reconstruction. Given the innovative work and vigorous development of 3D GS in autonomous driving, this paper comprehensively reviews and summarizes the existing related research to showcase the evolution of the 3D GS technology and possible future development directions. First, the overall research background of 3D GS is introduced based on two aspects 3D scene reconstruction methods and 3D GS research progress. Second, the relevant knowledge points of 3D GS and the core formulas to clarify the mathematical mechanism of 3D GS are presented. Third, the primary applications of the 3D scene reconstruction technology based on 3D GS in automatic driving are presented through new perspective synthesis, scene understanding, and simultaneous localization and map building (SLAM). Finally, the research frontier directions of 3D GS in autonomous driving are described, including structure optimization, 4D scene reconstruction, and cross-domain research. This paper may provide an effective and convenient pathway for researchers to understand, explore, apply this novel research method, and promote the development and application of 3D GS in automatic driving.
Journal Article