We present a novel method to interpolate smoke and liquid simulations. Our approach calculates a dense space-time deformation using grid-based signed-distance functions of the inputs. A key advantage of this implicit Eulerian representation is that it allows us to use powerful techniques from the optical flow area. In combination with novel multi-stage and surface projection algorithms, we achieve a very robust matching of the inputs. Once the match is computed, we can create arbitrary in-between variants very efficiently. We will demonstrate the numerous advantages of our approach with complex example flows. Among others, these advantages include volumetric deformations that can be applied to details around the surface, automatic matches without any user input, and the inherent handling of topology changes. As a result, we can interpolate swirling smoke clouds, and a whole array of liquid simulations. We can even match and interpolate phenomena with fundamentally different physics: a drop of liquid, and a blob of heavy smoke.
Parametric surfaces are an essential modeling tool in computer aided design and movie production. Even though their use is well established in industry, generating ray-traced images adds significant cost in time and memory consumption. Ray tracing such surfaces is usually accomplished by subdividing the surfaces on-the-fly, or by conversion to polygonal representations. However, on-the-fly subdivision is computationally very expensive, whereas polygonal meshes require large amounts of memory. This is a particular problem for parametric surfaces with displacement, where very fine tessellation is required to faithfully represent the shape. Hence, memory restrictions are the major challenge in production rendering. In this paper, we present a novel solution to this problem. We propose a compression scheme for BVHs on parametric patches, that significantly reduces data requirements for hierarchies. We further propose an approximate evaluation method that does not require leaf geometry, yielding an overall reduction of memory consumption by a factor of 60 over regular BVHs on indexed face sets and by a factor of 16 over established state-of-the-art compression schemes. Alternatively, our compression can simply be applied to a standard BVH while keeping the leaf geometry, resulting in a compression rate of up to 2:1 over current methods.
We present an approach to generate plausible acoustic effects at interactive rates in large dynamic environments containing many sound sources. Our formulation combines sound source clustering, listener-based backward ray tracing, and hybrid audio rendering to handle complex scenes. We present a new algorithm for perceptually clustering distant sound sources and for computing dynamic late reverberation based on high-order reflections. We also describe a hybrid convolution-based audio rendering technique that can process hundreds of thousands of sound paths at interactive rates. We demonstrate the performance on many indoor and outdoor scenes with tens or hundreds of sources. In practice, our algorithm can compute 50 orders of specular or diffuse reflections at interactive rates on a multi-core PC, and we observe more than an order of magnitude improvement.
Closed eyes and look-aways in photographs often ruin precious captured moments. In this paper, we present a novel framework for eye-editing in photographs, in a fully automatic way. We leverage on a user's personal photo collection to find a ``good'' set of reference eyes, and transfer them onto the target image. Our example-based editing approach is robust and reliable for realistic image editing. A fully automatic pipeline for realistic eye-editing is challenging due to the unconstrained conditions under which the face appears in a typical photo collection. We use crowdsourcing to learn from human evaluations - what aspects of the target--reference image pair will produce realistic results. We subsequently train a model that automatically selects the best reference candidate(s) by narrowing the gap in terms of pose, local contrast, lighting conditions and even expressions. Finally, we develop a comprehensive pipeline of 3D face estimation, image warping, relighting, image harmonization, automatic segmentation and image compositing to achieve highly believable results. We evaluate the performance of our method via quantitative and crowd-sourced experiments.
We present the first method to efficiently and accurately predict antialiasing footprints to pre-filter color-, normal-, and displacement-mapped appearance in the context of multi-bounce global illumination. We derive Fourier spectra for radiance and importance functions that allow us to compute spatial-angular filtering footprints at path vertices, for both uni- and bi-directional path construction. We then use these footprints to antialias reflectance modulated by high-resolution color, normal, and displacement maps encountered along a path. In doing so, we also unify the traditional path-space formulation of light-transport with our frequency-space interpretation of global illumination pre-filtering. Our method is fully compatible with all existing single bounce pre-filtering appearance models, not restricted by path length, and easy to implement atop existing path-space renderers. We illustrate its effectiveness on several radiometrically complex scenarios where previous approaches either completely fail or require orders of magnitude more time to arrive at similarly high-quality results.
The human visual system is sensitive to relative differences in luminance but light transport simulation algorithms based on Metropolis sampling often result in a highly non-uniform relative error distribution over the rendered image. We present a new target function for Metropolis photon tracing that ensures good stratification of photons leading to pixel estimates with equalized relative error. We develop a hierarchical scheme for progressive construction of the target function from paths sampled during rendering. As opposed to previous work that only defined a target function in the image plane, ours can be associated with spatial regions. This allows us to take advantage of illumination coherence to receive robust estimates of the target function while adapting to geometry discontinuities. To sample from this target function, we design a new replica exchange Metropolis scheme. We apply our algorithm in progressive photon mapping and show that it often outperforms alternative approaches in terms of image quality by a large margin.
In digital image editing software, layers organize images. However, layers are not explicitly represented in the final image, and may never have existed for a physical painting or a photograph. We propose a technique to decompose an image into layers. In our decomposition, each layer represents a single-color coat of paint applied with varying opacity. Our decomposition is based on the image's RGB-space geometry. In RGB-space, the linear nature of the standard Porter-Duff "over" pixel compositing operation implies a geometric structure. The vertices of the convex hull of image pixels in RGB-space correspond to a palette of paint colors. These colors may be hidden and inaccessible to algorithms based on clustering visible colors. For our layer decomposition, users choose the palette size (degree of simplification to perform on the convex hull), as well as a layer order for the paint colors (vertices). We then solve a constrained optimization problem to find translucent, spatially coherent opacity for each layer, such that the composition of the layers reproduces the original image. We demonstrate the utility of the resulting decompositions for recoloring and object insertion. Our layers can be interpreted as generalized barycentric coordinates; we compare to these and other recoloring approaches.
Designing a unified framework for simulating multiple human behaviors has proven very difficult. In this paper, we present an approach for control system design that can generate animations that closely resemble a diverse set of captured reference motions: walking, running, and gymnastic behaviors such as a roundoff, Popa jump, hand walking, and backflip. We achieve this generalization by using a balancing strategy that relies on a new model, which we call the momentum-mapped inverted pendulum model (MMIPM). We analyze the reference motion in a pre-processing step to extract the important properties of balance for that particular motion. That information is contained in the sequence of footsteps/handholds and in the basic balance control of an inverted pendulum controller. At run-time, the controller plans a desired motion frame by frame based on the current estimate of the pendulum state and a predicted pendulum trajectory. By tracking this time-varying trajectory, our controller creates a character that dynamically balances, changes speed, makes turns, jumps and performs a backflip. The initial controller can be optimized to further improve the motion quality. We demonstrate the power of this approach by generating controllers that produce natural looking animations and are robust to changes in the environment, user commands and external disturbances.
A major goal of research on virtual humans is the animation of expressive characters that display distinct psychological attributes. Body motion is an effective way of portraying different personalities and differentiating characters. The purpose and contribution of this study is to describe a formal, broadly applicable, procedural, and empirically grounded association between personality and body motion and apply this association to modify a given virtual human body animation that can be represented by these formal concepts. Because the body movement of virtual characters may involve different choices of parameter sets depending on the context, situation or application, formulating a link from personality to body motion requires an intermediate step to assist generalization. For this intermediate step, we refer to Laban Movement Analysis, which is a movement analysis technique for systematically describing and evaluating human motion. We have developed an expressive human motion generation system with the help of movement experts and conducted a user study to explore how the psychologically validated OCEAN personality factors were perceived in motions with various Laban parameters. We have then applied our findings to procedurally animate expressive characters with personality, and validated the generalizability of our approach across different models and animations via another perception study.
Image pipelines arise frequently in modern computational photography systems and consist of multiple processing stages where each stage produces an intermediate image that serves as input to a future stage. Inspired by recent work on loop perforation [Sidiroglou-Douskos et al. 2011], this paper introduces image perforation, a new optimization technique that allows automatically exploring the space of performance-accuracy trade-offs within an image pipeline. Image perforation works by transforming loops over the image at each pipeline stage into coarser loops that effectively "skip" certain samples. These missing samples are reconstructed for later stages using a number of different interpolation strategies that are relatively inexpensive to perform compared to the original cost of computing the sample. We describe a genetic algorithm for automatically exploring the resulting combinatoric search space of which loops to perforate, in what manner, by how much, and using what reconstruction method. We also present a prototype language that implements image perforation along with several other domain-specific optimizations and show results for a number of different image pipelines and inputs. For these cases, image perforation achieves speedups of 2x-10x with acceptable loss in visual quality and significantly outperforms loop perforation.
While collections of parametric shapes are growing in size and use, little progress has been made on the fundamental problem of shape-based matching and retrieval for parametric shapes in a collection. The search space for such collections is both discrete (number of shapes) and continuous (parameters values). In this work we propose representing this space using descriptors which have shown to be effective for single shape retrieval. While single shapes can be represented as points in a descriptor space, parametric shapes are mapped into larger continuous regions. For smooth descriptors, we can assume that these regions are bounded low dimensional manifolds where the dimensionality is given by the number of shape parameters. We propose representing these manifolds with a set of primitives, namely: points and bounded tangent spaces. Our algorithm describes how to define these primitives and how to use them to construct a manifold approximation that allows accurate and fast retrieval. We show how to compute decision variables with no need for empirical parameters adjustments and discuss theoretical guarantees on retrial accuracy. We validate our approach with experiments that use different types of descriptors on a collection of shapes from multiple categories.