Publications

You can also find my articles on my Google Scholar profile.

Journal Articles

Medical Implicit Shape Neural Representation from a Single X-Ray

Published in Medical Image Analysis, 2025

This manuscript describes a method for reconstructing 3D anatomical shapes from a single XR, named MISNeX. MISNeX extends the work of previous chapters by leveraging a contrastive learning framework to overcome the inherent limitations of 2D input, enabling the reconstruction of complex 3D shapes without requiring volumetric data. The method directly regresses an implicit function from a 2D XR image to represent the surface of an anatomical structure. Unlike conventional methods that rely on template mesh deformation or CT reconstruction from XR images, MISNeX avoids the topological constraints of template mesh deformation and the inaccuracies often introduced by CT reconstruction. By embedding 3D spatial information into the 2D latent features through contrastive learning, MISNeX enables high-quality 3D shape reconstructions, even for complex anatomical structures like the pelvis. The method was evaluated using publicly available femur and pelvis datasets, where it achieved superior results compared to state-of-the-art template mesh deformation and implicit shape representation method using 2D input, and achieved comparable results with implicit shape representation method using 3D input.

Recommended citation: G. Jin, Y. Jung, and J. Kim, “Medical Implicit Shape Neural Representation from a Single X-Ray,” in preparation to submit to Medical Image Analysis.
Download Paper | Download Slides

MISNeR: Medical Implicit Shape Neural Representation for Image Volume Visualisation

Published in Computer Graphics Forum, 2024

Three-dimensional visualisation of mesh reconstruction of the medical images is commonly used for various clinical applica- tions including pre / post-surgical planning. Such meshes are conventionally generated by extracting the surface from volumetric segmentation masks. Therefore, they have inherent limitations of staircase artefacts due to their anisotropic voxel dimensions. The time-consuming process for manual refinement to remove artefacts and/or the isolated regions further adds to these lim- itations. Methods for directly generating meshes from volumetric data by template deformation are often limited to simple topological structures, and methods that use implicit functions for continuous surfaces, do not achieve the level of mesh recon- struction accuracy when compared to segmentation-based methods. In this study, we address these limitations by combining the implicit function representation with a multi-level deep learning architecture. We introduce a novel multi-level local feature sampling component which leverages the spatial features for the implicit function regression to enhance the segmentation result. We further introduce a shape boundary estimator that accelerates the explicit mesh reconstruction by minimising the number of the signed distance queries during model inference. The result is a multi-level deep learning network that directly regresses the implicit function from medical image volumes to a continuous surface model, which can be used for mesh reconstruction from arbitrary high volume resolution to minimise staircase artefacts. We evaluated our method using pelvic computed tomography (CT) dataset from two public sources with varying z-axis resolutions. We show that our method minimised the staircase arte- facts while achieving comparable results in surface accuracy when compared to the state-of-the-art segmentation algorithms. Furthermore, our method was 9 times faster in volume reconstruction than comparable implicit shape representation networks.

Recommended citation: G. Jin, Y. Jung, L. Bi, and J. Kim, “MISNeR: Medical Implicit Shape Neural Representation for Image Volume Visualisation,”, Computer Graphics Forum, accepted, 2024.
Download Paper | Download Slides

A Generative Adversarial Network for Upsampling of Direct Volume Rendering Images

Published in Computer Graphics Forum, 2024

Direct volume rendering (DVR) is an important tool for scientific and medical imaging visualisation. Modern GPU acceleration has made DVR more accessible, however, the production of high-quality rendered images with high frame rates, is computa- tionally expensive. We propose a deep learning method with a reduced computational demand. We leveraged a conditional Generative Adversarial Network (cGAN) to upsample DVR images (a rendered scene), with a reduced sampling rate to obtain similar visual quality to that of a fully-sampled method. Our dvrGAN is combined with a colour-based loss function that is op- timised for DVR images where different structures such as skin, bone etc. are distinguished by assigning them distinct colours. The loss function highlights the structural differences between images, by examining pixel-level colour, and thus helps identify, for instance, small bones in the limbs that may not be evident with reduced sampling rates. We evaluated our method in DVR of human computed tomography (CT) and CT angiography (CTA) volumes. Our method retained image quality and reduced computation time when compared to fully sampled methods and outperformed existing state-of-the-art upsampling methods.

Recommended citation: G. Jin, Y. Jung, M. Fulham, D. Feng, and J. Kim, “A Generative Adversarial Network for Upsampling of Direct Volume Rendering Images,” Computer Graphics Forum, accepted, 2024.
Download Paper | Download Slides

Conference Papers

Challenges and Constraints in Deformation-Based Medical Mesh Representation.

Published in Computer Graphics International, 2023

Mesh representation of medical imaging isosurfaces are essential for medical analysis. These representations are typically obtained using mesh extraction methods to segment 3D volumes. However, the meshes extracted from such methods often suffer from undesired staircase artefacts. In this paper, we evaluate the existing mesh deformation methods that deform a template mesh to desired shapes. We evaluate two variants of such method on three datasets of varying topological complexity. Our objective is to demonstrate that, despite the mesh deformation methods having their limitations, they avoid the generation of staircase artefacts.

Recommended citation: G. Jin, Y. Jung, and J. Kim, "Challenges and Constraints in Deformation-Based Medical Mesh Representation." Computer Graphics International, pp. 146-156. Cham: Springer Nature Switzerland, 2023
Download Paper

Gary Jin