Research Projects

  • Structure analysis and interactive synthesis of Internet visual media, Major international (regional) joint research program, National Natural Science Foundation, PT: Shi-Min Hu, Project number: 61120106007, 2012-2015.
  • Intelligent processing of internet visual media, National Basic Research Program (973 program) of China, PI: SHi-Min Hu, Project Number: 2011CB302200, 2011-2015.
  • Exact-Geodesic-based Voronoi Diagrams on 2-Manifolds and its Applications, National Natural Science Foundation, PI: Yong-Jin Liu, Project number: 60970099, 2010-2012.
  • Video stylization and rendering based on structure analysis, National Natural Science Foundation, PI: Song-Hai Zhang, Project number: 60970100, 2010-2012.

    • 2013




    Internet visual media processing: a survey with graphics and vision applications
    The Visual Computer, 2013, accepted.  
    Shi-Min Hu, Tao Chen, Kun Xu, Ming-Ming Cheng, Ralph R. Martin

    In recent years, the computer graphics and computer vision communities have devoted significant attention to research based on Internet visual media resources. The huge number of images and videos continually being uploaded by millions of people have stimulated a variety of visual media creation and editing applications, while also posing serious challenges of retrieval, organization, and utilization. This article surveys recent research as regards processing of large collections of images and video, including work on analysis, manipulation, and synthesis. It discusses the problems involved, and suggests possible future directions in this emerging research area.



    Motion-Aware Gradient Domain Video Composition
    IEEE Transactions on Image Processing, 2013, accepted.  
    Tao Chen, Jun-Yan Zhu, Ariel Shamir, and Shi-Min Hu

    For images, gradient domain composition methods like Poisson blending offer practical solutions for uncertain object boundaries and differences in illumination conditions. However, adapting Poisson image blending to video faces new challenges due to the added temporal dimension. In video, the human eye is sensitive to small changes in blending boundaries across frames, and slight differences in motions of the source patch and target video. We present a novel video blending approach that tackles these problems by merging the gradient of source and target videos and optimizing a consistent blending boundary based on a user provided blending trimap for the source video. Our approach extends mean-value coordinates interpolation to support hybrid blending with a dynamic boundary while maintaining interactive performance. We also provide a user interface and source object positioning method that can efficiently deal with complex video sequences beyond the capabilities of alpha blending.



    Aesthetic Image Enhancement by Dependence-Aware Object Re-Composition
    IEEE Transactions on Multimedia, 2013, accepted.  
    Fang-Lue Zhang, Miao Wang, Shi-Min Hu

    This paper proposes an image enhancement method to optimize photo composition, by rearranging foreground objects in the photo. To adjust objects¡¯ positions while keeping the original scene content, we first perform a novel structure dependence analysis on the image to obtain the dependencies between all background regions. To determine the optimal positions for foreground objects, we formulate an optimization problem based on widely used heuristics for aesthetically pleasing pictures. Semantic relations between foreground objects are also taken into account during optimization. The final output is produced by moving foreground objects, together with their dependent regions, to optimal positions. The results show that our approach can effectively optimize photos with single or multiple foreground objects without compromising the original photo content.



    Manipulating Perspective in Stereoscopic Images
    IEEE Transactions on Visualization and Computer Graphics, 2013, accepted.  
    Song-Pei Du, Shi-Min Hu and Ralph R Martin

    Stereoscopic ("3D") devices and content relying on stereopsis are now widely available. However, traditional image editing techniques cannot be directly used to edit stereoscopic media, as extra constraints are needed to ensure consistent changes are made to both left and right images. This paper addresses the problem of manipulating perspective in stereoscopic pairs. We note that a straightforward approach based on depth recovery is unsatisfactory. Instead, our method relies on feature correspondences between stereoscopic image pairs. Given a new, user-specified perspective, we determine correspondence constraints under this perspective, and optimize a 2D warp for each image which preserves straight lines, and guarantees proper stereopsis relative to the new camera. Experiments demonstrate that our method generates new views with suitable stereoscopic output which correspond well to expected projections, for a wide range of specified perspective. Various advanced camera effects, such as dolly zoom and wide angle effects, can also be readily generated for stereoscopic image pairs using our method.



    Time-Line Editing of Objects in Video
    IEEE Transactions on Visualization and Computer Graphics, 2013, Vol. 19.    
    Shao-Ping Lu, Song-Hai Zhang, Jin Wei, Shi-Min Hu and Ralph R Martin

    We present a video editing technique based on changing the time-lines of individual objects in video, which leaves them in their original places but puts them at different times. This allows the production of object-level slow motion effects, fast motion effects, or even time reversal. This is more flexible than simply applying such effects to whole frames, as new relationships between objects can be created. As we restrict object interactions to the same spatial locations as in the original video, our approach can produce high-quality results using only coarse matting of video objects. Coarse matting can be done efficiently using automatic video object segmentation, avoiding tedious manual matting. To design the output, the user interactively indicates the desired new life-spans of objects, and may also change the overall running time of the video. Our method rearranges the time-lines of objects in the video whilst applying appropriate object interaction constraints. We demonstrate that, while this editing technique is somewhat restrictive, it still allows many interesting results.



    Mixed-Domain Edge-Aware Image Manipulation
    IEEE Transactions on Image Processing, 2013, Vol. 22, No. 5, 1915 - 1925.  
    Xian-Ying Li, Yan Gu, Shi-Min Hu, and Ralph R. Martin

    This paper gives a novel approach to edge-aware image manipulation. Our method processes a Gaussian pyramid from coarse to fine, and at each level, we apply a nonlinear filter bank to the neighborhood of each pixel. Outputs of these spatially-varying filters are merged using global optimization, and this optimization problem is solved using an explicit mixeddomain (real space and DCT transform space) solution, which is efficient, accurate, and easy-to-implement. We demonstrate applications of our method to a set of problems including detail and contrast manipulation, HDR compression, non-photorealistic rendering, and haze removal.



    PoseShop: A Human Image Database and Personalized Content Synthesis
    IEEE Transactions on Visualization and Computer Graphics, 2013, Vol.19, No. 5, 824-837.   
    Tao Chen, Ping Tan, Li-Qian Ma, Ming-Ming Cheng, Ariel Shamir and Shi-Min Hu

    We present a human image database collected from online images where human figures are segmented out of their background. The images are organized based on action semantic, clothes attributes and indexed by the shape of their poses. The database is built by downloading, analyzing, and filtering over 3 million human images from the Internet and can be queried using either silhouette sketch or a skeleton to find a given pose. We demonstrate the application of this database for multi-frame personalized content synthesis in the form of comic-strips, where the main character is the user or his/her friends. We address the two challenges of such synthesis, namely personalization and consistency over a set of frames, by introducing head swapping and clothes swapping techniques. We also demonstrate an action correlation analysis application to show the usefulness of the database for vision application.



    Efficient Synthesis of Gradient Solid Textures
    Graphical Models, Vol. 75, No. 3, 104-117, 2013   
    (An earlier version has been presented in Computaional Visual Media 2013, Beijing, and received Best paper Award)
    Guo-Xin Zhang, Yu-Kun Lai and Shi-Min Hu

    Solid textures require large storage and are computationally expensive to synthesize. In this paper, we propose a novel solid representation called gradient solids to compactly represent solid textures, including a tricubic interpolation scheme of colors and gradients for smooth variation and a region-based approach for representing sharp boundaries. We further propose a novel approach based on this to directly synthesize gradient solid textures from exemplars. Compared to existing methods, our approach avoids the expensive step of synthesizing the complete solid textures at voxel level and produces optimized solid textures using our representation. This avoids significant amount of unnecessary computation and storage involved in the voxel-level synthesis while producing solid textures with comparable quality to the state of the art. The algorithm is much faster than existing approaches for solid texture synthesis and makes it feasible to synthesize high-resolution solid textures in full. Our compact representation also supports efficient novel applications such as instant editing propagation on full solids.


    Semi-Regular Solid Texturing from 2D Image Exemplars
    IEEE Transactions on Visualization and Computer Graphics, 2013, Vol. 19, No. 3, 460-469.    
    Song-Pei Du, Shi-Min Hu and Ralph R.Martin

    Solid textures, comprising 3D particles embedded in a matrix in a regular or semi-regular pattern, are common in natural and man-made materials, such as brickwork, stone walls, plant cells in a leaf, etc. We present a novel technique for synthesizing such textures, starting from 2D image exemplars which provide cross-sections of the desired volume texture. The shapes and colors of typical particles embedded in the structure are estimated from their 2D cross-sections. Particle positions in the texture images are also used to guide spatial placement of the 3D particles during synthesis of the 3D texture. Our experiments demonstrate that our algorithm can produce higher-quality structures than previous approaches; they are both compatible with the input images, and have a plausible 3D nature.



    Poisson Coordinates
    IEEE Transactions on Visualization and Computer Graphics, 2013, Vol.19, No. 2, 344-352.    
    Xian-Ying Li and Shi-Min Hu,

    Harmonic functions are the critical points of a Dirichlet energy functional, the linear projections of conformal maps. They play an important role in computer graphics, particularly for gradient-domain image processing and shape-preserving geometric computation. We propose Poisson coordinates, a novel transfinite interpolation scheme based on the Poisson integral formula, as a rapid way to estimate a harmonic function on a certain domain with desired boundary values. Poisson coordinates are an extension of the Mean Value coordinates (MVCs) which inherit their linear precision, smoothness, and kernel positivity. We give explicit formulae for Poisson coordinates in both continuous and 2D discrete forms. Superior to MVCs, Poisson coordinates are proved to be pseudoharmonic (i.e., they reproduce harmonic functions on n-dimensional balls). Our experimental results show that Poisson coordinates have lower Dirichlet energies than MVCs on a number of typical 2D domains (particularly convex domains). As well as presenting a formula, our approach provides useful insights for further studies on coordinates-based interpolation and fast estimation of harmonic functions.



    View-Dependent Multiscale Fluid Simulation
    IEEE Transactions on Visualization and Computer Graphics, 2013, Vol. 19, No. 2, 178-188.   
    Yue Gao, Chen-Feng Li, Bo Ren and Shi-Min Hu

    Fluid motions are highly nonlinear and non-stationary, with turbulence occurring and developing at different length and time scales. In real-life observations, the multiscale flow generates different visual impacts depending on the distance to the viewer. We propose a new fluid simulation framework that adaptively allocates computational resources according to the human visual perception. First, a 3D empirical model decomposition scheme is developed to obtain the velocity spectrum of the turbulent flow. Then, depending on the distance to the viewer, the fluid domain is divided into a sequence of nested simulation partitions. Finally, the multiscale fluid motions revealed in the velocity spectrum are distributed non-uniformly to these view-dependent partitions, and the mixed velocity fields defined on different partitions are solved separately using different grid sizes and time steps. The fluid flow is solved at different spatial-temporal resolutions, such that higher-frequency motions closer to the viewer are solved at higher resolutions and vice versa. The new simulator better utilizes the computing power, producing visually plausible results with realistic fine-scale details in a more efficient way. It is particularly suitable for large scenes with the viewer inside the fluid domain. Also, as high-frequency fluid motions are distinguished from low-frequency motions in the simulation, the numerical dissipation is effectively reduced.


    2012




    Structure Recovery by Part Assembly
    ACM Transactions on Graphics, Vol. 31, No. 6, ACM SIGGRAPH ASIA 2012.    (click for project webpage, data set is available)
    Chao-Hui Shen, Hongbo Fu, Kang Chen and Shi-Min Hu

    This work presents a technique that allows quick conversion of acquired low-quality data from consumer-level scanning devices to high-quality 3D models with labeled semantic parts and meanwhile their assembly reasonably close to the underlying geometry. This is achieved by a novel structure recovery approach that is essentially local to global and bottom up, enabling the creation of new structures by assembling existing labeled parts with respect to the acquired data. We demonstrate that using only a small-scale shape repository, our part assembly approach is able to faithfully recover a variety of high-level structures from only a single-view scan of man-made objects acquired by the Kinect system, containing a highly noisy, incomplete 3D point cloud and a corresponding RGB image.



    An Optimization Approach for Extracting and Encoding Consistent Maps in a Shape Collection
    ACM Transactions on Graphics, Vol. 31, No. 6, ACM SIGGRAPH ASIA 2012.   
    Qi-xing Huang, Guoxin Zhang, Lin Gao, Shi-Min Hu, Adrian Butscher and Leonidas Guibas

    We introduce a novel diffusion-based approach for computing high quality point-to-point maps among a collection of shapes so that several desirable properties are satisfied. The proposed approach takes as input a sparse set of initial maps between pairs of shapes (sufficient to connect the model graph) and implicitly builds a new set of pointwise maps between all pairs of shapes which aim to (1) align with the initial maps, (2) map neighboring points to neighboring points, and (3) provide cycle-consistency, so that map compositions along cycles approximate the identity map. Maps among subsets of the shapes that admit nearly perfect loop closure are highly redundant and can be compactly represented by maps from a single base shape to other shapes. Our algorithm extracts such a set of base shapes so that every other shape is ¡°covered¡± by at least one of the base shapes.



    ImageAdmixture: Putting Together Dissimilar Objects from Groups
    IEEE Transactions on Visualization and Computer Graphics, 2012, Vol. 18, No.11, 1849-1857.    
    Fang-Lue Zhang, Ming-Ming Cheng, Jiaya Jia, Shi-Min Hu

    We present a semi-automatic image editing framework dedicated to individual structured object replacement from groups. The major technical difficulty is element separation with irregular spatial distribution, hampering previous texture and image synthesis methods from easily producing visually compelling results. Our method uses the object-level operations and finds grouped elements based on appearance similarity and curvilinear features. This framework enables a number of image editing applications, including natural image mixing, structure preserving appearance transfer, and texture mixing.



    Fisheye Video Correction
    IEEE Transactions on Visualization and Computer Graphics, 2012, Vol. 18, No.10, 1771-1783.    
    Jin Wei, Chen-Feng Li, Shi-Min Hu, Ralph Martin, and Chiew-Lan Tai

    Various types of video are captured with fisheye lenses, particularly surveillance video, due to their ability to capture a wide field of view. However, distortion changes as objects in the scene move, making fisheye video difficult to interpret and uncomfortable to watch. Current still fisheye image correction methods are either limited to small angles of view, or are strongly content-dependent, and therefore not suitable for processing video streams. We present a novel scheme for fisheye video correction, which minimizes time-varying distortions and preserves salient content features in a coherent manner. Our optimization process is controlled by user annotation, and includes a comprehensive set of measures addressing different aspects of natural scene appearance. These terms are all formulated in quadratic form, leading to a quadratic programming problem which can be solved in a closed form using a sparse linear system. We illustrate our method with a range of examples, demonstrating coherent natural-looking video output in which the visual quality of individual frames is comparable to state-of-the-art methods for still fisheye photograph correction.



    Interactive Images: Proxy-based Scene Understanding for Smart Manipulation
    ACM Transactions on Graphics (ACM SIGGRAPH),2012,Vol. 31, No. 4, article number 99,    
    Youyi Zheng, Xiang Chen, Ming-Ming Cheng, Kun Zhou, Shi-Min Hu, Niloy J. Mitra

    Images are static and lack important depth information of underlying 3D scenes. We introduce interactive images in the context of man-made environments wherein objects are simple, regular, share various non-local relations (e.g., coplanarity, repetitions, etc.), and are often repeated. We present an interactive framework to create a partial scene reconstruction based on cuboid-proxies using minimal user interaction. This enables a range of intuitive image edits mimicking real-world behavior, which are otherwise difficult to achieve. Effectively, the user simply provides high-level semantic hints, while our system ensures plausible operations by conforming to the extracted non-local relations. We demonstrate our system on a range of real-world images and validate the plausibility of the results using a user study.


    Other publications in 2012

    1. Yong-Liang Yang and Chao-Hui Shen, Multi-Scale Salient Features for Analyzing 3D Shapes, Journal of Computer Science and technology, Vol. 27, No. 6, 1092-1099, 2012.   
    2. Long Zeng, Yong-Jin Liu, Sang-Hun Lee, Ming-Fai Yuen, Q-Complex: efficient non-manifold boundary representation with inclusion topology, Computer-Aided Design, Vol. 44, No. 11, 1115-1126, 2012.    
    3. Ling-Qi Yan, Yahan Zhou, Kun Xu and Rui Wang,Accurate Translucent Material Rendering under Spherical Gaussian Lights, Computer Graphics Forum, Vol. 31, No. 7, 2267-2276, 2012.   
    4. Long Zeng, Yong-Jin Liu, Ming Chen, Ming-Fai Yuen, Least squares quasi-developable mesh approximation, Computer Aided Geometric Design, Vol. 29, No. 7, 565-578, 2012.    
    5. Chen Goldberg, Tao Chen, Fang-Lue Zhang, Ariel Shamir, Shi-Min Hu, Data-Driven Object Manipulation in Images, Computer Graphics Forum, Vol. 31, No. 2, 265-274, 2012 (Eurographics 2012).   
    6. Tao Chen, Aidong Lu and Shi-Min Hu, Visual storylines:Semanticvisualizationofmoviesequence, Computer & Graphics, Vol. 36, No. 4, 2012.   
    7. Cui-Xia Ma, Yong-Jin Liu, Hong-An Wang, Dong-Xing Teng, Guo-Zhong Dai, Sketch-based Annotation and Visualization in Video Authoring, IEEE Transactions on Multimedia, Vol. 14, No. 4, pp. 1153-1165, 2012.    
    8. Yong-Jin Liu, Yi-Fu Zheng, Lu Lv, Yu-Ming Xuan, Xiao-Lan Fu, 3D Model Retrieval based on Color+Geometry Signatures, The Visual Computer, Vol. 28, No. 1, pp.75-86, 2012.    

    2011




    Interactive Hair Rendering and Appearance Editing under Environment Lighting
    ACM Transactions on Graphics, Vol. 30, No. 6, ACM SIGGRAPH ASIA 2011.    
    Kun Xu, Li-Qian Ma, Bo Ren, Rui Wang, Shi-Min Hu

    We present an interactive algorithm for hair rendering and appearance editing under complex environment lighting represented as spherical radial basis functions (SRBFs). Our main contribution is to derive a compact 1D circular Gaussian representation that can accurately model the hair scattering function introduced by [Marschner et al. 2003]. The primary benefit of this representation is that it enables us to evaluate, at run-time, closed-form integrals of the scattering function with each SRBF light, resulting in efficient computation of both single and multiple scatterings. In contrast to previous work, our algorithm computes the rendering integrals entirely on the fly and does not depend on expensive precomputation. Thus we allow the user to dynamically change the hair scattering parameters, which can vary spatially. Analyses show that our 1D circular Gaussian representation is both accurate and concise. In addition, our algorithm incorporates the eccentricity of the hair. We implement our algorithm on the GPU, achieving interactive hair rendering and simultaneous appearance editing under complex environment maps for the first time.



    Adaptive Partitioning of Urban Facades
    ACM Transactions on Graphics, Vol. 30, No. 6, ACM SIGGRAPH ASIA 2011.    
     
    Chao-Hui Shen, Shi-Sheng Huang, Hongbo Fu, Shi-Min Hu

    Automatically discovering high-level facade structures in unorganized 3D point clouds of urban scenes is crucial for applications like digitalization of real cities. However, this problem is challenging due to poor-quality input data, contaminated with severe missing areas, noise and outliers. This work introduces the concept of adaptive partitioning to automatically derive a flexible and hierarchical representation of 3D urban facades. Our key observation is that urban facades are largely governed by concatenated and/or interlaced grids. Hence, unlike previous automatic facade analysis works which are typically restricted to globally rectilinear grids, we propose to automatically partition the facade in an adaptive manner, in which the splitting direction, the number and location of splitting planes are all adaptively determined. Such an adaptive partition operation is performed recursively to generate a hierarchical representation of the facade. We show that the concept of adaptive partitioning is also applicable to flexible and robust analysis of image facades. We evaluate our method on a dozen of LiDAR scans of various complexity and styles, and the image facades from the eTRIMS database and the Ecole Centrale Paris database. A series of applications that benefit from our approach are also demonstrated.



    Online Video Stream Abstraction and Stylization
    IEEE Transactions on Multimedia, vol.13, no.6, pp.1286-1294, Dec. 2011    
    Song-Hai Zhang, Xian-Ying Li, Shi-Min Hu, and Ralph R. Martin

    This paper gives an automatic method for online video stream abstraction, producing a temporally coherent output video stream, in the style with large regions of constant color and highlighted bold edges. Our system includes two novel components. Firstly, to provide coherent and simplified output, we segment frames, and use optical flow to propagate segmentation information from frame to frame; an error control strategy is used to help ensure that the propagated information is reliable. Secondly, to achieve coherent and attractive coloring of the output, we use a color scheme replacement algorithm specifically designed for an online video stream. We demonstrate real-time performance for CIF videos, allowing our approach to be used for live communication and other related applications. Index Terms¡ªAbstraction, color scheme replacement, optical flow, segmentation, temporal coherence, video stream.



    A Geometric Study of V-style Pop-ups: Theories and Algorithms
    ACM Transactions on Graphics 2011, Vol. 30, No. 4, ACM SIGGRAPH 2011    
    Xian-Ying Li, Tao Ju, Yan Gu, Shi-Min Hu

    Pop-up books are a fascinating form of paper art with intriguing geometric properties. In this paper, we present a systematic study of a simple but common class of pop-ups consisting of patches falling into four parallel groups, which we call v-style pop-ups. We give sufficient conditions for a v-style paper structure to be pop-uppable. That is, it can be closed flat while maintaining the rigidity of the patches, the closing and opening do not need extra force besides holding two patches and are free of intersections, and the closed paper is contained within the page border. These conditions allow us to identify novel mechanisms for making pop-ups. Based on the theory and mechanisms, we developed an interactive tool for designing v-style pop-ups and an automated construction algorithm from a given geometry, both of which guaranteeing the popuppability of the results.



    Global Contrast based Salient Region Detection
    IEEE CVPR, p. 409-416, 2011,   [bib]
    Ming-Ming Cheng, Guo-Xin Zhang, Niloy J. Mitra, Xiaolei Huang, Shi-Min Hu

    Reliable estimation of visual saliency allows appropriate processing of images without prior knowledge of their content, and thus remains an important step in many computer vision tasks including image segmentation, object recognition, and adaptive compression. We propose a regional contrast based saliency extraction algorithm, which simultaneously evaluates global contrast differences and spatial coherence. The proposed algorithm is simple, efficient, and yields full resolution saliency maps. Our algorithm consistently outperformed existing saliency detection methods, yielding higher precision and better recall rates, when evaluated using one of the largest publicly available data sets. We also demonstrate how the extracted saliency map can be used to create high quality segmentation masks for subsequent image processing.



    Construction of Iso-contours, Bisectors and Voronoi Diagrams on Triangulated Surfaces
    IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 33, No. 8, 1502-1517, 2011  
    Yong-Jin Liu, Zhan-Qing Chen, Kai Tang

    In the research of computer vision and machine perception, three-dimensional objects are usually represented by 2-manifold triangular meshes M. In this paper, we present practical and efficient algorithms to construct iso-contours, bisectors and Voronoi diagrams of point sites onM, based on an exact geodesic metric. Compared to Euclidean metric spaces, the Voronoi diagrams on M exhibit many special properties that fail all the existing Euclidean Voronoi algorithms. To provide practical algorithms for constructing geodesic-metric-based Voronoi diagrams on M, this paper studies the analytic structure of iso-contours, bisectors and Voronoi diagrams on M. After a necessary preprocessing of model M, practical algorithms are proposed for quickly obtaining full information about iso-contours, bisectors and Voronoi diagrams on M. The complexity of the construction algorithms is also analyzed. Finally three interesting applications, surface sampling and reconstruction, 3D skeleton extraction and point pattern analysis are presented that show the potential power of the proposed algorithms in pattern analysis.



    Image Retargeting Quality Assessment
    Computer Graphics Forum, 2011, Vol. 30, No. 2, Eurographics 2011,   
    Yong-Jin Liu, Xi Luo, Yu-Ming Xuan, Wen-Feng Chen, Xiao-Lan Fu

    Content-aware image retargeting is a technique that can flexibly display images with different aspect ratios and simultaneously preserve salient regions in images. Recently many image retargeting techniques have been proposed. To compare image quality by different retargeting methods fast and reliably, an objective metric simulating the human vision system (HVS) is presented in this paper. Different from traditional objective assessment methods that work in bottom-up manner (i.e., assembling pixel-level features in a local-to-global way), in this paper we propose to use a reverse order (top-down manner) that organizes image features from global to local viewpoints, leading to a new objective assessment metric for retargeted images. A scale-space matching method is designed to facilitate extraction of global geometric structures from retargeted images. By traversing the scale space from coarse to fine levels, local pixel correspondence is also established. The objective assessment metric is then based on both global geometric structures and local pixel correspondence. To evaluate color images, CIE Lab color space is utilized. Experimental results are obtained to measure the performance of objective assessments with the proposed metric. The results show good consistency between the proposed objective metric and subjective assessment by human observers.



    Connectedness of Random Walk Segmentation
    IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011. 33(1): p. 200 -202..
    Ming-Ming Cheng, Guo-Xin Zhang

    Connectedness of random walk segmentation is examined, and novel properties are discovered, by considering electrical circuits equivalent to random walks. A theoretical analysis shows that earlier conclusions concerning connectedness of random walk segmentation results are incorrect, and counterexamples are demonstrated.


    Other publications in 2011

    1. Wen-Qi Zhang, Yong-Jin Liu, Approximating the Longest Paths in Grid Graphs, Theoretical Computer Science, 2011, Vol. 412, No. 39, 5340-5350.    
    2. Yong-Jin Liu, Kai Tang, Wen-Yong Gong, Tie-Ru Wu, Industrial Design using Interpolatory Discrete Developable Surfaces, Computer-Aided Design, 2011, Vol. 43, No. 9, 1089-1098, 2011.    
    3. Guo-Xin Zhang, Song-Pei Du, Yu-Kun Lai, Tianyun Ni, Shi-Min Hu, Sketch Guided Solid Texturing, Graphics Models, 2011, Vol. 73, No.3, 59-73.    
    4. Cui-Xia Ma, Yong-Jin Liu, Hai-Yan Yang, Dong-Xing Teng, Hong-An Wang, Guo-Zhong Dai, KnitSketch: A Sketch Pad for Conceptual Design of 2D Garment Patterns, IEEE Transactions on Automation Science and Engineering, 2011, Vol. 8, No. 2,    
    5. Yong-Jin Liu, Cui-Xia Ma, Dong-Liang Zhang, Easytoy: a plush toy design system using editable sketch curves, IEEE Computer Graphics & Applications, 2011, Vol. 31, No. 2,    
    6. Shao-Ping Lu and Song-Hai Zhang, Saliency-Based Fidelity Adaptation Preprocessing for Video Coding, Journal of Computer Science and Technology, 2011, Vol. 26, No. 1, 195-202    

    2010




    Instant Propagation of Sparse Edits on Images and Videos
    Computer Graphics Forum, Special issue of Pacific Graphics 2010, Vol. 29, No. 7, 2049-2054
    Yong Li, Tao Ju, Shi-Min Hu

    The ability to quickly and intuitively edit digital contents has become increasingly important in our everyday life. We propose a novel method for propagating a sparse set of user edits (e.g., changes in color, brightness, contrast, etc.) expressed as casual strokes to nearby regions in an image or video with similar appearances. Existing methods for edit propagation are typically based on optimization, whose computational cost can be prohibitive for large inputs. We re-formulate propagation as a function interpolation problem in a high-dimensional space, which we solve very efficiently using radial basis functions. While simple to implement, our method significantly improves the speed and space cost of existing methods, and provides instant feedback of propagation results even on large images and videos.



    Popup: Automatic Paper Architectures from 3D Models
    ACM Transactions on Graphics 2010, Vol. 29, No. 4, ACM SIGGRAPH 2010  
    Xian-Ying Li, Chao-Hui Shen, Shi-Sheng Huang, Tao Ju, Shi-Min Hu

    Paper architectures are 3D paper buildings created by folding and cutting. The creation process of paper architecture is often labor intensive and highly skill-demanding, even with the aid of existing computer-aided design tools. We propose an automatic algorithm for generating paper architectures given a user-specified 3D model. The algorithm is grounded on geometric formulation of planar layout for paper architectures that can be popped-up in a rigid and stable manner, and sufficient conditions for a 3D surface to be popped up from such a planar layout. Based on these conditions, our algorithm computes a class of paper architectures containing two sets of parallel patches that approximate the input geometry while guaranteed to be physically realizable. The method is demonstrated on a number of architectural examples, and physically engineered results are presented.



    RepFinder: Finding Approximately Repeated Scene Elements for Image Editing
    ACM Transactions on Graphics 2010, Vol. 29, No. 4, ACM SIGGRAPH 2010    
    Ming-Ming Cheng, Fang-Lue Zhang, Niloy J. Mitra, Xiaolei Huang, Shi-Min Hu

    Repeated elements are ubiquitous and abundant in both manmade and natural scenes. Editing such images while preserving the repetitions and their relations is nontrivial due to overlap, missing parts, deformation between instances, illumination variation, etc. Manually enforcing such relations is laborious and error prone. We propose a novel framework where simple user input in the form of scribbles are used to guide detection and extraction of such repeated elements. Our detection process is based on a novel boundary band method, and robustly extracts the repetitions along with their mutual depth relations. We then use topological sorting to establish a partial depth ordering of overlapping repeated instances. Missing parts on occluded instances are completed using information from other instances. The extracted repeated instances can then be seamlessly edited and manipulated for a variety of high level tasks that are otherwise difficult to perform. We demonstrate the versatility of our framework on a large set of inputs of varying complexity, showing applications to image rearrangement, edit transfer, deformation propagation, and instance replacement.



    Metric-Driven RoSy Fields Design
    IEEE Transactions on Visualization and Computer Graphics, 2010, Vol. 16, No. 1, 95-108.   
    Yu-Kun Lai, Miao Jin, Xuexiang Xie, Ying He, Jonathan Palacios, Eugene Zhang, Shi-Min Hu and Xianfeng David Gu

    This work introduces a rigorous and practical approach for automatic N-RoSy field design on arbitrary surfaces with user defined field topologies. The user has full control of the number, positions and indices of the singularities, the turning numbers of the loops, and is able to edit the field interactively. We formulate N-RoSy field construction as designing a Riemannian metric, such that the holonomy along any loop is compatible with the local symmetry of N-RoSy fields. We prove the compatibility condition using discrete parallel transport. The complexity of N-RoSy field design is caused by curvatures. In our work, we propose to simplify the Riemannian metric to make it flat almost everywhere. This approach greatly simplifies the process and improves the flexibility, such that, it can design N-RoSy fields with single singularity, and mixed-RoSy fields. To demonstrate the effectiveness of our approach, we apply our design system to pen-and-ink sketching and geometry remeshing.


    Other publications in 2010

    1. Yong-Jin Liu, Dong-Liang Zhang, Matthew Ming-Fai Yuen, A survey on CAD methods in garment design, Computers in Industry, 2010, Vol. 61, No. 6, 576-593    
    2. Yong-Jin Liu, Kam-Lung Lai, Gang Dai, Ming-Fai Yuen, A semantic feature model in concurrent engineering, IEEE Transactions on Automation Science and Engineering, 2010, Vol. 7, No. 3, 659-665    
    3. Yu-Ping Wang, Shi-Min Hu, Optimization approach for 3D model watermarking by linear binary programming, Computer Aided Design, 2010, Vol. 27, No. 5, 395-404    
    4. Yong-Jin Liu, Wen-Qi Zhang, Kai Tang, Some notes on maximal arc intersection of spherical polygons: its NP-hardness and approximation algorithms, The Visual Computer, 2010, Vol. 26, No. 4, 287-292    
    5. Chao-Hui Shen, Guo-Xin Zhang, Yu-Kun Lai, Shi-Min Hu, Harmonic Field Based Volume Model Construction from Triangle Soup, Journal of Computer Science and Technology, 2010, Vol. 25, No. 3, 562-571    
    6. Jin Wei and Yu Lou, Feature Preserving Mesh Simplification Using Feature Sensitive Metric, Journal of Computer Science and Technology, 2010, Vol. 25, No. 3, 595-605    
    7. Yu-Kun Lai, Leif Kobbelt and Shi-Min Hu, Feature aligned quad dominant remeshing using iterative local updates, Computer Aided Design, 2010, Vol. 42, No. 2, 109-117
    (An earlier version has been presented in ACM Symosium on Solid and Physical Modeling, June 2-4, 2008)


    2009




    Sketch2Photo: Internet Image Montage
    ACM Transactions on Graphics, Vol. 28, No. 5, Article No. 124, ACM SIGGRAPH ASIA 2009   
    Tao Chen, Ming-Ming Cheng, Ping Tan, Ariel Shamir, Shi-Min Hu

    We present a system that composes a realistic picture from a user provided sketch with text labels. The composed picture is generated by seamlessly stitching several photographs automatically searched from internet according to the sketch and its text labels. While on line image search generates noisy results, our system can automat ically select suitable photographs to generate a high quality com position. To achieve this, we first design a filtering scheme to exclude undesirable images from searched results. Then we propose a novel image blending algorithm for seamless image composition. Our blending algorithm returns a numeric score for each blending, which is used to optimize the combination of searched images. Several vivid results are generated in the experiments. We also perform a user study to demonstrate the advantages of our system.



    Efficient Affinity-based Edit Propagation using K-D Tree
    ACM Transactions on Graphics, Vol. 28, No. 5, Article No. 118, ACM SIGGRAPH ASIA 2009    
    Kun Xu, Yong Li, Tao Ju, Shi-Min Hu, Tian-Qiang Liu

    Image/video editing by strokes has become increasingly popular due to the ease of interaction. Propagating the user inputs to the rest of the image/video, however, is often time and memory consuming especially for large data. We propose here an efficient scheme that allows affinity-based edit propagation to be computed on data containing tens of millions of pixels at interactive rate (in matter of seconds). The key in our scheme is a novel means for approximately solving the optimization problem involved in edit propagation, using adaptive clustering in a high-dimensional, affinity space. Our approximation significantly reduces the cost of existing affinitybased propagation methods while maintaining visual fidelity, and enables interactive stroke-based editing even on high resolution images and long video sequences using commodity computers.



    Simulating Gaseous Fluids with Low and High Speeds
    Computer Graphics Forum, Special issue of Pacific Graphics 2009, Vol. 28, No. 7, 1845-1852    
    Yue Gao, Chen-Feng Li, Shi-Min Hu, Brian A. Barsky

    Gaseous fluids may move slowly, as smoke does, or at high speed, such as occurs with explosions. High-speed gas flow is always accompanied by low-speed gas flow, which produces rich visual details in the fluid motion. Realistic visualization involves a complex dynamic flow field with both low and high speed fluid behavior. In computer graphics, algorithms to simulate gaseous fluids address either the low speed case or the high speed case, but no algorithm handles both efficiently. With the aim of providing visually pleasing results, we present a hybrid algorithm that efficiently captures the essential physics of both low- and high-speed gaseous fluids. We model the low speed gaseous fluids by a grid approach and use a particle approach for the high speed gaseous fluids. In addition, we propose a physically sound method to connect the particle model to the grid model. By exploiting complementary strengths and avoiding weaknesses of the grid and particle approaches, we produce some animation examples and analyze their computational performance to demonstrate the effectiveness of the new hybrid method.



    Edit Propagation on Bidirectional Texture Functions
    Computer Graphics Forum, Special issue of Pacific Graphics 2009, Vol. 28, No. 7, 1871-1877    
    Kun Xu, Jiaping Wang, Xin Tong, Shi-Min Hu, Baining Guo

    We propose an efficient method for editing bidirectional texture functions (BTFs) based on edit propagation scheme. In our approach, users specify sparse edits on a certain slice of BTF. An edit propagation scheme is then applied to propagate edits to the whole BTF data. The consistency of the BTF data is maintained by propagating similar edits to points with similar underlying geometry/reflectance. For this purpose, we propose to use view independent features including normals and reflectance features reconstructed from each view to guide the propagation process. We also propose an adaptive sampling scheme for speeding up the propagation process. Since our method needn't any accurate geometry and reflectance information, it allows users to edit complex BTFs with interactive feedback.



    A Shape-Preserving Approach to Image Resizing
    Computer Graphics Forum, Special issue of Pacific Graphics 2009, Vol. 28, No. 7, 1897-1906  
    Guo-Xin Zhang, Ming-Ming Cheng, Shi-Min Hu, Ralph R. Martin

    We present a novel image resizing method which attempts to ensure that important local regions undergo a geometric similarity transformation, and at the same time, to preserve image edge structure. To accomplish this, we define handles to describe both local regions and image edges, and assign a weight for each handle based on an importance map for the source image. Inspired by conformal energy, which is widely used in geometry processing, we construct a novel quadratic distortion energy to measure the shape distortion for each handle. The resizing result is obtained by minimizing the weighted sum of the quadratic distortion energies of all handles. Compared to previous methods, our method allows distortion to be diffused better in all directions, and important image edges are well-preserved. The method is efficient, and offers a closed form solution.



    Generalized Discrete Ricci Flow
    Computer Graphics Forum, Special issue of Pacific Graphics 2009, Vol. 28, No. 7, 2005-2014    
    Yong-Liang Yang, Ren Guo, Feng Luo, Shi-Min Hu, Xianfeng Gu

    Surface Ricci flow is a powerful tool to design Riemannian metrics by user defined curvatures. Discrete surface Ricci flow has been broadly applied for surface parameterization, shape analysis, and computational topology. Conventional discrete Ricci flow has limitations. For meshes with low quality triangulations, if high conformality is required, the flow may get stuck at the local optimum of the Ricci energy. If convergence to the global optimum is enforced, the conformality may be sacrificed. This work introduces a novel method to generalize the traditional discrete Ricci flow. The generalized Ricci flow is more flexible, more robust and conformal for meshes with low quality triangulations. Conventional method is based on circle packing, which requires two circles on an edge intersect each other at an acute angle. Generalized method allows the two circles either intersect or separate from each other. This greatly improves the flexibility and robustness of the method. Furthermore, the generalized Ricci flow preserves the convexity of the Ricci energy, this ensures the uniqueness of the global optimum. Therefore the algorithm won't get stuck at the local optimum. Generalized discrete Ricci flow algorithms are explained in details for triangle meshes with both Euclidean and hyperbolic background geometries. Its advantages are demonstrated by theoretic proofs and practical applications in graphics, especially surface parameterization.



    Automatic and Topology-Preserving Gradient Mesh Generation for Image Vectorization
    ACM Transactions on Graphics, Vol. 28, No. 3, article 85, ACM SIGGRAPH 2009    
    Yu-Kun Lai, Shi-Min Hu, Ralph R. Martin

    Gradient mesh vector graphics representation, used in commercial software, is a regular grid with specified position and color, and their gradients, at each grid point. Gradient meshes can compactly represent smoothly changing data, and are typically used for single objects. This paper advances the state of the art for gradient meshes in several significant ways. Firstly, we introduce a topology-preserving gradient mesh representation which allows an arbitrary number of holes. This is important, as objects in images often have holes, either due to occlusion, or their 3D structure. Secondly, our algorithm uses the concept of image manifolds, adapting surface parameterization and fitting techniques to generate the gradient mesh in a fully automatic manner. Existing gradient-mesh algorithms require manual interaction to guide grid construction, and to cut objects with holes into disk-like regions. Our new algorithm is empirically at least 10 times faster than previous approaches. Furthermore, image segmentation can be used with our new algorithm to provide automatic gradient mesh generation for a whole image. Finally, fitting errors can be simply controlled to balance quality with storage.



    Vectorizing Cartoon Animations
    IEEE Transactions on Visualization and Computer Graphics, 2009, Vol. 15, No. 4, May/June, 618-629  
    Song-hai Zhang, Tao Chen, Yi-Fei Zhang, Shi-Min Hu, Ralph R. Martin

    We present a system for vectorizing 2D raster format carton animations. The output animations are visually flicker free, smaller in file size, and easy to edit. We identify decorative lines separately from coloured regions. We use an accurate and semantically meaningful image decomposition algorithm which supports an arbitrary color model for each region. To ensure temporal coherence in the output cartoon, we reconstruct a universal background for all frames, and separately extract foreground regions. Simple user-assistance is required to complete the background. Each region and decorative line is vectorized and stored together with their motions from frame to frame.



    A new watermarking method for 3D model based on integral invariant
    IEEE Transactions on Visualization and Computer Graphics, 2009, Vol. 15, No. 2, March/April, 285-294   
    Yu-Ping Wang and Shi-Min Hu

    In this report, we propose a new semi-fragile watermarking algorithm for the authentication of 3D models based on integral invariants. To do so, we embed a watermark image by modifying the integral invariants of some of the vertices. Basically, we shift a vertex and its neighbors in order to change the integral invariants. To extract the watermark, test all the vertices for the embedded information, and combine them to recover the watermark image. How many parts can
    the watermark image be recovered would help us to make the authentication decision. Experimental test shows that this method is robust against rigid transform and noise attack, and useful to test purposely attack besides transferring noise and geometrical transforming noise. An additional contribution of this paper is a new algorithm for computing two kinds of integral invariants.


    Other publications in 2009

    1. Yong-Jin Liu, Yu-Kun Lai and Shi-Min Hu, Stripification of Free-Form Surfaces with Global Error Bounds for Developable Approximation, IEEE Transactions on Automation Science and Engineering, 2009, Vol. 6, No. 4, 700-709    
    2. Yu-Kun Lai, Shi-Min Hu, Ralph R. Martin and Paul L. Rosin,apid and Effective Segmentation of 3D Models using Random Walks, Computer Aided Geometric Design, 2009, Vol. 26, No. 6, 665-679.  
    (An earlier version has been presented in ACM Symosium on Solid and Physical Modeling, June 2-4, 2008)
    3. Song-hai Zhang, Tao Chen, Yi-Fei Zhang, Shi-Min Hu, Ralph R. Martin, Video-Based Running Water Animation in Chinese Painting Style, Science in China Series F: Information Sciences, 2009, Vol. 52, No. 2, 162-171    
    4. Zhe Bian, Shi-Min Hu and Ralph R Martin, Evaluation for Small Visual Difference Between Conforming Meshes on Strain Field, Journal of Computer Science and Technology, 2009, Vol. 24, No. 1, 65-75    
    The preliminary version of this work has been presented on GMP2008.

    2008




    Optimal Surface Parameterization Using Inverse Curvature Map
    IEEE Transactions on Visualization and Computer Graphics, 2008, Vol. 14, No. 5, Septmber/Octber, 1054-1066.   
    Yong-Liang Yang, Junho Kim, Feng Luo, Shi-Min Hu, and Xianfeng Gu

    Mesh parameterization is a fundamental technique in computer graphics. The major goals during mesh parameterization are to minimize both the angle distortion and the area distortion. Angle distortion can be eliminated by use of conformal mapping, in principle. Our paper focuses on solving the problem of nding the best discrete conformal mapping that also minimizes area distortion. Major theoretical results and practical algorithms are presented for optimal parameterization based on the inverse curvature map. Comparisons are conducted with existing methods and using different energies. Novel parameterization applications are also introduced. The theoretical framework of the inverse curvature map can be applied to further study discrete conformal mappings.



    Shrinkability Maps for Content-Aware Video Resizing
    Computer Graphics Forum, Special issue of Pacific Graphics 2008 , Vol. 27, No. 7, 1797-1804 .
    Yi-Fei Zhang,Shi-Min Hu, Ralph R. Martin

    A novel method is given for content-aware video resizing, i.e. targeting video to a new resolution (which may involve aspect ratio change) from the original. We precompute a per-pixel cumulative shrinkability map which takes into account both the importance of each pixel and the need for continuity in the resized result. (If both x and y resizing are required, two separate shrinkability maps are used, otherwise one suffices). A random walk model is used for efficient offline computation of the shrinkability maps. The latter are stored with the video to create a multi-sized video, which permits arbitrarysized new versions of the video to be later very efficiently created in real-time, e.g. by a video-on-demand server supplying video streams to multiple devices with different resolutions. These shrinkability maps are highly compressible, so the resulting multi-sized videos are typically less than three times the size of the original compressed video. A scaling function operates on the multi-sized video, to give the new pixel locations in the result, giving a high-quality content-aware resized video.


    Shape Deformation using a Skeleton to Drive Simplex Transformations
    IEEE Transactions on Visualization and Computer Graphics, 2008, Vol. 14, No. 3, May/June, 693-706  
    Han-Bing Yan, Shi-Min Hu, Ralph R Martin, and Yong-Liang Yang
    The preliminary version of this work has been presented on CGI 2006

    This paper presents a skeleton-based method for deforming meshes (the skeleton need not be the medial axis). The significant difference from previous skeleton-based methods is that the latter use the skeleton to control movement of vertices whereas we use it to control the simplices defining the model. By doing so, errors that occur near joints in other methods can be spread over the whole mesh, via an optimization process, resulting in smooth transitions near joints of the skeleton. By controlling simplices, our method has the additional advantage that no vertex weights need be defined on the bones, which is a tedious requirement in previous skeleton-based methods. Furthermore, by incorporating the translation vector in our optimisation, unlike other methods, we do not need to fix an arbitrary vertex, and the deformed mesh moves with the deformed skeleton. Our method can also easily be used to control deformation by moving a few chosen line segments, rather than a skeleton.



    Spherical Piecewise Constant Basis Functions for All-Frequency Precomputed Radiance Transfer
    IEEE Transactions on Visualization and Computer Graphics, 2008, Vol. 14, No. 2, March/April, 454-467  
    Kun Xu, Yun-Tao Jia, Hongbo Fu, Shi-Min Hu and Chiew-Lan Tai

    This paper presents a novel basis function, called spherical piecewise constant basis function (SPCBF), for precomputed radiance transfer. SPCBFs have several desirable properties: rotatability, ability to represent all-frequency signals, and support for efficient multiple product. By partitioning the illumination sphere into a set of subregions, and associating each subregion with an SPCBF valued 1 inside the region and 0 elsewhere, we precompute the light coefficients using the resulting SPCBFs. We run-time approximate BRDF and visibility coefficients with the same set of SPCBFs through fast lookup of summed-area-table (SAT) and visibility distance table (VDT), respectively. SPCBFs enable new effects such as object rotation in all-frequency rendering of dynamic scenes and onthe-fly BRDF editing under rotating environment lighting. With graphics hardware acceleration, our method achieves real-time frame rates.

    Video: Download video here (13.0MB).


    Other publications in 2008

    1. Yu-Kun Lai, Yong-Jin Liu, Yu Zang and Shi-Min Hu, Fairing Wireframes in Industrial Design, IEEE International Conference on Shape Modeling and Applications, June 4-6, 2008, 29-35.  
    2. Yong-Jin Liu, Matthew Ming-Fai Yuen, Geometry-optimized virtual human head and its applications, Computer & Graphics, 2008, Vol. 32, No. 6, 624-631    

    2007




    Editing The Topology of 3D Models by Sketching
    ACM Transactions on Graphics, Vol. 26, No. 3, Article 42, ACM SIGGRAPH 2007
    Tao Ju, Qian-Yi Zhou and Shi-Min Hu

    We present a method for modifying the topology of a 3D model with user control. The heart of our method is a guided topology editing algorithm. Given a source model and a user-provided target shape, the algorithm modifies the source so that the resulting model is topologically consistent with the target. Our algorithm permits removing or adding various topological features (e.g., handles, cavities and islands) in a common framework and ensures that each topological change is made by minimal modification to the source model. To create the target shape, we have also designed a convenient 2D sketching interface for drawing 3D line skeletons. As demonstrated in a suite of examples, the use of sketching allows more accurate removal of topological artifacts than previous methods, and enables creative designs with specific topological goals.

    Video: Download video here (31.8MB). (Cannot open the video? Cannot hear the audio? Get latest QuickTime player.)

    Software: A software MendIT based on this paper will come soon, please refer to webpage: http://graphics.usc.edu/~qianyizh/software.html



    Real-time homogeneous translucent material editing
    EuroGraphics 2007, Computer Graphics Forum, Vol. 26, No. 3, 545–552.
    Kun Xu, Yue Gao, Yong Li, Tao Ju and Shi-Min Hu

    This paper presents a novel method for real-time homogeneous translucent material editing under fixed illumination. We consider the complete analytic BSSRDF model proposed by Jensen et al.[JMLH01], including both multiple scattering and single scattering. Our method allows the user to adjust the analytic parameters of BSSRDF and provides high-quality, real-time rendering feedback. Inspired by recently developed Precomputed Radiance Transfer (PRT) techniques, we approximate both the multiple scattering diffuse reflectance function and the single scattering exponential attenuation function in the analytic model using basis functions, so that re-computing the outgoing radiance at each vertex as parameters change reduces to simple dot products. In addition, using a non-uniform piecewise polynomial basis, we are able to achieve smaller approximation error than using bases adopted in previous PRT-based works, such as spherical harmonics and wavelets. Using hardware acceleration, we demonstrate that our system generates images comparable to [JMLH01] at real-time frame-rates.

    Video: Download video here (17.3MB).



    Topology Repair of Solid Models Using Skeletons
    IEEE Transactions on Visualization and Computer Graphics, 2007, Vol. 13, No. 4, 675-685.
    Qian-Yi Zhou, Tao Ju and Shi-Min Hu

    We present a method for repairing topological errors on solid models in the form of small surface handles, which often arise from surface reconstruction algorithms. We utilize a skeleton representation that offers a new mechanism for identifying and measuring handles. Our method presents two unique advantages over previous approaches. First, handle removal is guaranteed not to introduce invalid geometry or additional handles. Second, by using an adaptive grid structure, our method is capable of processing huge models efficiently at high resolutions.

    Slides: Download slides here (24.8MB). A poster for this paper is also available, download poster here (7.5MB).

    Software: A software TopoMender based on this paper is now available, please refer to webpage: http://graphics.usc.edu/~qianyizh/software.html



    Robust Feature Classification and Editing
    IEEE Transactions on Visualization and Computer Graphics, 2007, Vol. 13, No.1, January/Feburary, 34-45.
    Yu-Kun Lai, Qian-Yi Zhou, Shi-Min Hu, Johannes Wallner and Helmut Pottmann

    Sharp edges, ridges, valleys and prongs are critical for the appearance and an accurate representation of a 3D model. In this paper, we propose a novel approach that deals with the global shape of features in a robust way. Based on a remeshing algorithm which delivers an isotropic mesh in a feature sensitive metric, features are recognized on multiple scales via integral invariants of local neighborhoods. Morphological and smoothing operations are then used for feature region extraction and classification into basic types such as ridges, valleys and prongs. The resulting representation of feature regions is further used for feature-specific editing operations.


    Other publications in 2007

    1. Jean-Baptiste Debard(Yang Fei), Romain Balp (Bai Luomin) , Raphaelle Chaine, Dynamic Delaunay tetrahedralisation of a deforming surface, The Visual Computers, 2007, Vol. 23, No. 12, 975 - 986
    2. Yong-Jin Liu, Qian-Yi Zhou and Shi-Min Hu, Handling Degenerate Cases in Exact Geodesic Computation on Triangle Meshes, The Visual Computers, 2007, Vol. 23, No. 9-11, 661-668. 
    3. Yong-Jin Liu, Kai Tang, Ajay Joneja, Modeling dynamic developable meshes by the Hamilton principle, Computer-Aided Design, 2007, Vol. 39, No. 9, 719-731.
    4. Han-Bing Yan, Shi-Min Hu, Ralph R Martin, 3D morphing using strain field interpolation, Journal of Computer Science and Technology, 2007, Vol. 22, No. 1, 147-155.

    2006



    Geometry and Convergence Analysis of Algorithms for Registration of 3D Shapes
    Geometry and convergence analysis of algorithms for registration of 3D shapes, International Journal of Computer Vision, 2006, Vol. 67, No. 3, 277-296.
    Helmut Pottmann, Qi-Xing Huang, Yong-Liang Yang and Shi-Min Hu

    The computation of a rigid body transformation which optimally aligns a set of measurement points with a surface and related registration problems are studied from the viewpoint of geometry and optimization.We provide a convergence analysis for widely used registration algorithms such as ICP, using either closest points (Besl and McKay [2]) or tangent planes at closest points (Chen and Medioni [4]), and for a recently developed approach based on quadratic approximants of the squared distance function [24]. ICP based on closest points exhibits local linear convergence only. Its counterpart which minimizes squared distances to the tangent planes at closest points is a Gauss-Newton iteration; it achieves local quadratic convergence for a zero residual problem and { if enhanced by regularization and step size control { comes close to quadratic convergence in many realistic scenarios. Quadratically convergent algorithms are based on the approach in [24]. The theoretical results are supported by a number of experiments; there, we also compare the algorithms with respect to global convergence behavior, stability and running time.



    Robust Principal Curvatures on Multiple Scales
    Proceedings of 4th Eurographics Symposium on Geometry Processing (2006). Eurographics Association, 223-226.
    Yong-Liang Yang, Yu-Kun Lai, Shi-Min Hu and Helmut Pottmann

    Geometry processing algorithms often require the robust extraction of curvature information. We propose to achieve this with principal component analysis (PCA) of local neighborhoods, defined via spherical kernels centered on the given surface F. Intersection of a kernel ball Br or its boundary sphere Sr with the volume bounded by F leads to the so-called ball and sphere neighborhoods. Information obtained by PCA of these neighborhoods turns out to be more robust than PCA of the patch neighborhood Br \F previously used. The relation of the quantities computed by PCA with the principal curvatures of F is revealed by an asymptotic analysis as the kernel radius r tends to zero. This also allows us to define principal curvatures “at scale r?in a way which is consistent with the classical setting. The advantages of the new approach are discussed in a comparison with results obtained by normal cycles and local fitting; whereas the former method somewhat lacks in robustness, the latter does not achieve a consistent behavior at features on coarse scales. As to applications, we address computing principal curves and feature extraction on multiple scales.


    Other publications in 2006

    1. Qi-Xing Huang, Simon Flory, Natasha Gelfand, Michael Hofer and Helmut Pottmann, Reassembling Fractured Objects by Geometric Matching, ACM Transactions on Graphics, Vol. 25, No. 3, 569-578, ACM SIGGRAPH 2006
    2. Yang Liu, Helmut Pottmann, Johannes Wallner, Yong-Liang Yang and Wenping Wang, Geometric Modeling with Conical Meshes and Developable Surfaces, ACM Transactions on Graphics, Vol. 25 , No. 3, 681-689, ACM SIGGRAPH 2006
    3. Yu-Kun Lai, Shi-Min Hu and Helmut Pottmann, Surface Fitting Based on a Feature Sensitive Parameterization, Computer-Aided Design, 2006, Vol. 38, No. 7, 800--807.
    4. Li Jin, Donguk Kim, Lisen Mu, Deok-Soo Kim and Shi-Min Hu, A Sweepline Algorithm for Euclidean Voronoi Diagram of Circles, Computer-Aided Design, 2006, Vol. 38, No. 3, 260-278.
    5. Yu-Kun Lai, Shi-Min Hu and Ralph R. Martin, Surface Mosaics, The Visual Computer, 2006, Vol. 22, No. 9-10, 604-611 (Pacific Graphics 2006).
    6. Jiaping Wang, Kun Xu, Kun Zhou, Stephen Lin, Shi-Min Hu and Baining Guo, Spherical Harmonics Scaling, The Visual Computer, 2006, Vol. 22, No. 9-10, 713-720 (Pacific Graphics 2006).
    7. Yu-Kun Lai, Qian-Yi Zhou, Shi-Min Hu and Ralph R. Martin, Feature Sensitive Mesh Segmentation, ACM Symp. Solid and Physical Modeling, 7-16, 2006.
    8. Xiao-Hua Cai, Yun-Tao Jia, Xi Wang, Shi-Min Hu and Ralph R. Martin, Rendering Soft Shadows using Multilayered Shadow Fins, Computer Graphics Forum, 2006, Vol.25, No.1, 1-14.

    2005



    Video Completion using Tracking and Fragment Merging
    The Visual Computer, 2005, Vol. 21, No. 8-10, 601-601. (Pacific Graphics 2005)
    Yun-Tao Jia, Shi-Min Hu and Ralph R. Martin

    Video completion is the problem of automatically filling space-time holes in video sequences left by the removal of unwanted objects in a scene. We solve it using texture synthesis, filling a hole inwards using three steps iteratively: we select the most promising target pixel at the edge of the hole, we find the source fragment most similar to the known part of the target neighborhood, and we merge source and target fragments to complete the target neighborhood, reducing the size of the hole. Earlier methods were slow, due to searching the whole video data for source fragments or completing holes pixel by pixel; they also produced blurred results due to sampling and smoothing. For speed, we track moving objects, allowing us to use a much smaller search space when seeking source fragments; we also complete holes fragment by fragment instead of pixelwise. Fine details are maintained by use of a graph cut algorithm when merging source and target fragments. Further techniques ensure temporal consistency of hole filling over successive frames. Examples demonstrate the effectiveness of our method.


    Other publications in 2005

    1. Xu-Ping Zhu, Shi-Min Hu, Chiew-Lan Tai, and Ralph R Martin, A Marching Method for Computing Intersection Curves of Two Subdivision Solids, in Mathematics of Srufaces XI, Eds. R. R. Martin, H. Bez, M. A. Sabin, 458-471, 2005.
    2. Johannes Wallner, Hans-Peter Schrocker, Shi-min Hu, Tolerance in geometric constraints solving, Reliable Computing, 2005, Vol. 11, No. 3, 235-251.
    3. Yu-Kun Lai, Shi-Min Hu, Xianfeng Gu, Ralph R. Martin, Geometric texture synthesis and transfer via geometry images, ACM Solid and Physical Modeling, MIT, USA, June 13-15, 2005, 15-26.
    4. Shi-min Hu, Johannes Wallner, A second order algorithm for orthogonal projection onto curves and surfaces, Computer Aided Geometric Design, 2004, Vol.22, No. 3, 251-260.
    5. Qi-Xing Huang, Shi-Min Hu and Ralph R. Martin, Fast degree elevation and knot insertion for B-spline curves, Computer Aided Geometric Design, 2005, Vol 22, No. 2, 183-197.

    2004



    Generalized Displacement Maps
    Proceedings of Eurographics Symposium on Rendering, 2004
    Xi Wang, Xin Tong, Stephen Lin, Shimin Hu, Baining Guo and Heung-Yeung Shum

    In this paper, we introduce a real-time algorithm to render the rich visual effects of general non-height-field geometric details, known as mesostructure. Our method is based on a five-dimensional generalized displacement map (GDM) that represents the distance of solid mesostructure along any ray cast from any point within a volumetric sample. With this GDM information, we propose a technique that computes mesostructure visibility jointly in object space and texture space which enables both control of texture distortion and efficient computation of texture coordinates and shadowing. GDM can be rendered with either local or global illumination as a per-pixel process in graphics hardware to achieve real-time rendering of general mesostructure.

    Other publications in 2004

    1. Han-Bing Yan, Shi-Min Hu, Ralph R. Martin, Morphing Based on Strain Field Interpolation, Journal of Computer Animation and Virtual Worlds (CAVW), 2004, Vol.15, No.3-4, 443-452.
    2. Shi-min Hu, Johannes Wallner, Error Propagation through Geometric Transformations, Journal for Geometry and Graphics, 2004, Vol.8, No.2, 171-183.
    3. Shi-Min Hu, Chen-feng Li, Hui Zhang, Actual Morphing: A phsical-based approach for blending two 2D/3D shapes, ACM Symposium on Solid Modeling and Applications, Genova, Italy, June 9-11, 2004.

    2003



    View-Dependent Displacement Mapping
    ACM Transaction on Graphics, Vol. 22, No. 3. 334-339, ACM SIGGRAPH 2003
    Lifeng Wang, Xi Wang, Xin Tong, Steve Lin, Shimin Hu, Baining Guo and Heung-Yeung Shum

    Significant visual effects arise from surface mesostructure, such as fine-scale shadowing, occlusion and silhouettes. To efficiently render its detailed appearance, we introduce a technique called viewdependent displacement mapping (VDM) that models surface displacements along the viewing direction. Unlike traditional displacement mapping, VDM allows for efficient rendering of selfshadows, occlusions and silhouettes without increasing the complexity of the underlying surface mesh. VDM is based on per-pixel processing, and with hardware acceleration it can render mesostructure with rich visual appearance in real time.

    Other publications in 2003

    1. Xu-Ping Zhu, Shi-Min Hu and Martin Ralph, Skeleton-Based Seam Computation for Triangulated Surface Parameterization, In Proceedings of Mathematics in Surfaces X, Sept 2003, Leeds, UK; Lecture Notes in Computer Science, 2003. [PS]
    2. Chiew-Lan Tai, Hu Shi-Min and Qixing Huang, Approximate merging of B-Spline curves via knot adjustment and constrained optimization, Computer Aided Design, 2003, Vol. 35, No. 10, 893 - 899.
    3. Xi Wang, Lifeng Wang, Ligang Liu, Shi-Min Hu and Baining Guo, Interactive Modeling of Tree Bark, In: Proceedings of Pacific Graphics 2003, IEEE CS Press, Oct 8-10, 2003
    4. Tao Wang, Yong Rui, Shi-Min Hu and Jia-guang Sun, Adaptive tree similarity learning for image retrieval, Multimedia Systems, 2003, Vol. 9, 131-143.

    2002

    1. Shi-Min Hu, Chiew-Lan Tai, Song-Hai Zhang, An Extension algorithm for B-spline curves by curve unclamping, Computer Aided Design, 2002, Vol. 34, No. 5, 415-4191.
    2. Yan-Tao Li, Shi-Min Hu and Jia-Guang Sun, A Constructive Approach to Solving 3-D Geometric Constraint Systems Using Dependence Analysis, Computer Aided Design, 2002, Vol. 34, No. 2, 97-108.
    3. Liu Shi-Xia, Hu Shi-Min, Sun Jiaguang, Two accelerating techniques for 3D reconstruction, Journal of Computer Science and Technology, 17(3), 362-368, 2002.


    2001

    1. Shi-Xia Liu, Shi-Min Hu, Yu-Jian Chen, Jia-Guang Sun, Reconstruction of curved solids from engineering drawings, Computer Aided Design, 2001, Vol. 33, No. 14, 1059-1072.
    2. Shi-Min Hu, Youfu Li, Tao JU, Xiang Zhu, Modifying the shape of NURBS surfaces with geometric constraints, Computer Aided Design, 2001, Vol. 33, No. 12, 903-912.
    3. Shi-Min, Hu Conversion between triangular and rectangular Bezier patches, Computer Aided Geometric Design, 2001, Vol.18, No. 7, 667-671. (In Special issue of memory of P. Bezier).
    4. Shi-Min Hu, Hui Zhang, Chiew-Lan Tai, Jia-Guang Sun, Direct Manipulation of FFD: Efficient Explicit Solutions and Decomposible Multiple Point Constraints, The Visual Computers, 2001, Vol. 17, No. 6, 370-379.
    5. Jun-Hai Yong, Shi-Min Hu, Jia-Guang Sun, Degree reduction of B-spline curves, Computer Aided Geometric Design, 2001, Vol. 13, NO. 2, 2001, 117-127.
    6. Shi-Min Hu, Ruofeng Tong, Tao JU, Jia-Guang Sun, Approximate merging of a pair of Bezier curves, Computer Aided Design, Vol 33, No. 2, 125-136, 2001.
    7. Jun-Hai Yong, Shi-Min Hu, JIa-Guang Sun, CIM Algorithm for Approximating Three Dimensional Polygonal Curves, Journal of Computer Science and Technology, 16(6), 489-497,2001.
    8. Tao Wang, Yong Rui and Shi-Min Hu, Optimal Adaptive Learning for Image Retrieval, Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR 2001), I-1140 to 1147, Kauai, Hawaii, December 11-13, 2001.
    9. Yan-Tao Li, Shi-Min Hu, Jia-Guang Sun, On the numerical redundancies of geometric constraint systems, Proceedings of Pacific Graphics 2001, 118-123, IEEE Computre Society Press, 2001, Tokyo.
    10. Jian-Hua Wu, Shi-Min Hu, Chiew-Lan Tai and Jia-Guang Sun, An effective feature-preserving mesh simplification scheme based on face constriction, Proceedings of Pacific Graphics 2001, 12-21, IEEE Computre Society Press, 2001, Tokyo.