text:
Neural Volumes: Learning Dynamic Renderable Volumes from Images
by Stephen Lombardi, et al.
Oculus VR, LLC
Modeling and rendering of dynamic scenes is challenging, as natural scenes often contain complex phenomena such as thin structures, evolving topology, translucency, scattering, occlusion, and biological motion. Mesh-based reconstruction and tracking often fail in these cases, and other approaches (e.g., light field video) typically rely on constrained viewing conditions, which limit interactivity. We circumvent these difficulties by presenting a learning-based approach to representing dynamic objects inspired by the integral projection model used in tomographic imaging. The approach is supervised directly from 2D images in a multi-view capture setting and does not require explicit reconstruction or tracking of the object. Our method has two primary components: an encoder-decoder network that transforms input images into a 3D volume representation, and a differentiable ray-marching operation that enables end-to-end training. By virtue of its 3D representation, our construction extrapolates better to novel viewpoints compared to screen-space rendering techniques. The encoder-decoder architecture learns a latent representation of a dynamic scene that enables us to produce novel content sequences not seen during training. To overcome memory limitations of voxel-based representations, we learn a dynamic irregular grid structure implemented with a warp field during ray-marching. This structure greatly improves the apparent resolution and reduces grid-like artifacts and jagged motion. Finally, we demonstrate how to incorporate surface-based representations into our volumetric-learning framework for applications where the highest resolution is required, using facial performance capture as a case in point.
READ FULL TEXT VIEW PDF
Stephen Lombardi
Tomas Simon
Jason Saragih
Gabriel Schwartz
Andreas Lehrmann
Yaser Sheikh
Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Images
We present a deep learning approach to reconstruct scene appearance from...
Sai Bi, et al. ∙
Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering
Inferring representations of 3D scenes from 2D observations is a fundame...
Vincent Sitzmann, et al. ∙
HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Sampling
Volumetric scene representations enable photorealistic view synthesis fo...
Benjamin Attal, et al. ∙
Fristograms: Revealing and Exploiting Light Field Internals
In recent years, light field (LF) capture and processing has become an i...
Thorsten Herfet, et al. ∙
Nanomatrix: Scalable Construction of Crowded Biological Environments
We present a novel method for interactive construction and rendering of ...
Ruwayda Alharbi, et al. ∙
ACORN: Adaptive Coordinate Networks for Neural Scene Representation
Neural representations have emerged as a new paradigm for applications i...
Julien N. P. Martel, et al. ∙
HexPlane: A Fast Representation for Dynamic Scenes
Modeling and re-rendering dynamic 3D scenes is a challenging task in 3D ...
Ang Cao, et al. ∙
neuralvolumes
Training and Evaluation Code for Neural Volumes
view repo
neural-volumes
Polygon meshes are an extremely popular representation for 3D geometry in photo-realistic scenes. Mesh-based representations efficiently model solid surfaces and can be paired with sophisticated reflectance functions to generate compelling renderings of natural scenes. In addition, there has been significant progress recently in optimization techniques to support real-time ray-tracing, allowing for interactivity and immersion in demanding applications such as Virtual Reality (VR). However, little of the interactive photo-real content available today is data-driven because many real-world phenomena are challenging to reconstruct and track with high fidelity. State-of-the-art motion capture systems struggle to handle complex occlusions (e.g., running hands through one's hair), to account for reflectance variability (e.g., specularities in the sheen of a moving object), or to track topological evolution in dynamic participating media (e.g., smoke as it billows upward). Where solutions exist, they are typically specialized to an individual phenomenon (Xu et al., 2014; Atcheson et al., 2008; Roth and Black, 2006; Hawkins et al., 2005) , and are often aimed at either image generation (Buehler et al., 2001; Kalantari et al., 2016) or 3D reconstruction (Goesele et al., 2007; Nießner et al., 2013) , but not both. Since mesh-based representations rely heavily on the quality of reconstruction to produce compelling renderings, they are ill-suited to handle such cases. Nonetheless, these kinds of phenomena are necessary to create compelling renderings of much of our natural world.
To address limitations posed by inaccurate geometric reconstructions, great progress has been made in recent years by relaxing the physics-inspired representation of light transport, and instead leveraging machine learning to bridge the gap between the representation and observed images of the scene. In
Lombardi et al. (2018) , this technique was used to great effect in modeling the human face, where it was demonstrated that a neural network can be trained to compensate for geometric reconstruction and tracking inaccuracies through a view-dependent texture. Similar approaches have also been shown to be effective for modeling general far-field scenes (Overbeck et al., 2018) . An extreme variant of this technique is screen-space rendering, where no geometry of the scene is used at all (Karras et al., 2018; Wang et al., 2018) . Although these approaches have been shown to produce high quality renderings of complex scenes, they are limited to viewpoints available to the system at training time. Since their neural architectures are not 3D aware, the methods do not extrapolate to novel viewpoints in a way that is consistent with the real world. The problem is exacerbated when modeling near-field scenes, where variation in viewpoint is more common as a user interacts with objects in the scene, compared with far-field captures where there is less interactivity and the viewer is mainly stationary.
An important insight in this work is that if both geometry and appearance variations can be learned simultaneously, phenomena explainable by geometric variations may be modeled as such, leading to better generalization across viewpoints. The challenge, then, is to formulate this joint learning such that good solutions can be found. Directly optimizing over a mesh-representation using gradient-based optimization is prone to terminating in poor local minima. This is the case even when a model of both appearance and geometry are known a priori, and is exacerbated when these models are also unknown. One of the main reasons for this difficulty is the local support of the gradients of mesh-based representations. To address this, we propose using a volumetric representation consisting of opacity and color at each position in 3D space, where rendering is realized through integral projection. During optimization, this semi-transparent representation of geometry disperses gradient information along the ray of integration, effectively widening the basin of convergence, enabling the discovery of good solutions.
Although the volumetric representation has the ability to represent 3D geometric phenomena in a geometrically faithful way, it can easily over-fit from image-supervision for a typical density of viewpoints. As such, additional regularization is necessary to achieve good results. In this work, we show that a neural-network decoder is sufficient to encourage discovery of solutions that generalize across viewpoints. At first glance, this may appear surprising given the decoder network typically has enough capacity to reproduce solutions found through a direct solve of the volume's entries. We conjecture that the decoder network introduces spatial regularity into the gradients of the volume's entries (i.e., opacity and color), leading to more generalizable solutions without diminishing the volumetric representation's capacity. Additionally, the decoder network is paired with an encoder network that produces a low-dimensional latent space that encodes the state of the scene at each frame, enabling joint reconstruction of sequences rather than just individual frames. Analogous to non-rigid structure from motion (Torresani et al., 2008) , this architecture can leverage a scene's regularity across time to improve viewpoint generalization further. This latent code can be used to generate novel renderings of the scene's content by traversing the latent space, enabling realistic modifications of the recording, or even completely new sequence animation, without requiring object/scene/content specific solutions.
Despite these advantages of the volumetric representation, its main drawback is limited resolution. Using voxel-based data structures to represent a scene typically requires an order of magnitude more memory than its mesh-based counterpart to achieve similar levels of resolution. Furthermore, much of this memory is dedicated to modeling empty space or the inside of objects; neither of which have an impact on the rendered result. This limitation stems from the regular grid structure this representation exhibits. To overcome this limitation, we employ a warping technique that indirectly escapes the restrictions imposed by a regular grid structure, allowing the learning algorithm to make the best use of available memory. Using this technique, we demonstrate significantly higher fidelity than using only a conventional voxel data structure. Furthermore, as our representation is 3D based, we can naturally combine it with surface-based reconstruction and tracking methods when appropriate. This allows us to reach the highest levels of fidelity on objects in the scene for which state-of-the-art reconstruction and tracking work well, while also maintaining a complete model of the scene.
In summary, we propose a novel volumetric representation that is object/scene agnostic, can generalize well to novel viewpoints, reconstructs dynamic scenes jointly, facilitates novel content generation, requires only image-level supervision and is end-to-end trainable. The resulting models afford real-time rendering and support on-the-fly adjustments, suitable for interactive applications in VR. In §2 we cover related work, followed by an overview of our approach in §3. Details of the encoder and decoder architectures are covered in sections §4 and §5 respectively. Rendering through integral projection is discussed in §6 and details of the learning problem are covered in §7. We evaluate this architecture on a number of challenging scenes and present ablation study on various design choices in our construction in §8. We conclude in §9 with a discussion and directions of future work.
Figure 2. Pipeline of our method. We begin with a multi-view capture system, from which we choose a subset of the cameras as input to our encoder. The encoder produces a latent code z which is decoded into a volume that gives an RGB and α value for each point in space, as well as a warp field used to index into the RGBα volume. The decoder may optionally use an additional control signal (e.g., head pose) c
. We then use an accumulative ray marching algorithm to render the volume. The final output in image space is an RGB image with associated alpha mask and estimated background. We composite these components together and minimize the
L2 loss between the rendered and target images w.r.t. network parameters.
2. Related Work
Our approach is driven by learning and rendering techniques spanning multiple domains, from volumetric reconstruction and deformable volumes to neural rendering and novel view synthesis. The following paragraphs discuss similarities and differences to previous works in these areas.
2.1. Classical Surface and Volumetric Reconstruction
Point- and surface-based reconstruction techniques have a long history in computer vision (see
(Furukawa and Hernández, 2015) for a review). Following successful efforts in stereo matching (Scharstein and Szeliski, 2002) , most of the subsequent literature has focused on the multi-view case, including extensions of photometric consistency (Furukawa and Ponce, 2010) and depth map fusion (Merrell et al., 2007; Zach et al., 2007) . Despite some attempts at handling more complex materials or semi-transparent surfaces (Szeliski and Golland, 1999; Fitzgibbon and Zisserman, 2005) , many popular multi-view stereo (MVS) methods, such as COLMAP (Schönberger and Frahm, 2016; Schönberger et al., 2016) , still struggle with thin structures and dense semi-transparent materials (e.g., hair and smoke).
Volumetric reconstruction methods side-step this problem of explicit correspondence matching, beginning with Voxel Coloring (Seitz and Dyer, 1997; Prock and Dyer, 1998; Seitz and Dyer, 1999) and Space Carving (Kutulakos and Seitz, 2000; Broadhurst et al., 2001; Bonet and Viola, 1999) . These methods recover occupancy and color in a voxel grid from multi-view images by evaluating the photo-consistency of each voxel in a particular order. Our method is similar to these classical voxel-based techniques in spirit, but rather than using a strict photometric consistency criterion, we learn a generative model that tries to best match the input images. Since we do not assume that objects in a scene are composed of flat surfaces, this approach also allows us to overcome the typical limitations of MVS methods and capture rich materials and fine geometry.
2.2. Volumetric Reconstruction with Ray Potentials
A number of more recent works on volumetric reconstruction have explored the concept of ray potentials, i.e., cost functions between the first surface struck by a ray and the color (or other property) of the corresponding pixel. Ulusoy et al. (2015) , Savinov et al. (2016) , and Paschalidou et al. (2018) formulate graph-based energy or inference objectives using ray potentials as constraints. A differentiable ray consistency criterion that inspired our work is developed by Tulsiani et al. (2017, 2018)
, who use an encoder-decoder architecture to predict voxel occupancy probabilities from RGB images. A loss on ray potentials evaluated in this volume is then backpropagated to the underlying convolutional architecture.
A fundamental difference between our work and previous works using ray potentials is that we model voxel transparency rather than occupancy probability, as we are focused on rendering rather than reconstruction. This explicit image formation process allows us to reconstruct and render dynamic scenes of semi-transparent materials, such as smoke.
2.3. Deformable Volumes
Non-rigidly deforming objects pose challenges to both optimization- and learning-based approaches. Over the years, significant efforts have been spent on their reconstruction and tracking from RGB-D sensors: the DynamicFusion method of Newcombe et al. (2015) produces a base 3D template surface using a Truncated Signed Distance Function (TSDF) representation, and a time-dependent warp to fuse a sequence of depth frames. Zollhöfer et al. (2014) and Innmann et al. (2016) also deform a 3D template surface with an as-rigid-as-possible regularizer.
Our rendering model is based on classical volume ray marching (Levoy, 1988; Ikits et al., 2004) , but we introduce the concept of a warp field to it: instead of directly sampling color and opacity along the ray, we first sample a 3D warp field encoding the location (within a template voxel grid) from which color and opacity are sampled. In 2D, similar techniques have been used to learn warps that align images (Jaderberg et al., 2015; Dai et al., 2017; Shu et al., 2018) . In our work, the warp field is not only used to model motion but also increases the effective resolution of the voxel grid by simulating a dynamic non-uniform sampling grid.
2.4. Neural Rendering
Deep learning-based rendering has become an active area of exploration, with some methods relying on volumetric representations. Nguyen-Phuoc et al. (2018)
use a convolutional neural network applied on a voxel grid to produce an image, but their method requires correspondence between the images and the voxel reconstruction. DeepVoxels
(Sitzmann et al., 2018) automatically learns a 3D feature representation for novel view synthesis but is limited to static objects and scenes, and the employed architecture does not lend itself to real-time inference. Martin-Brualla et al. (2018) use neural networks to fill holes and generally improve the quality of a textured geometric representation. Kim et al. (2018) use a U-net architecture to convert an image of rasterized attributes to realistic images, similar to pix-to-pix (Isola et al., 2017) . Our method shares many similarities with these approaches but has one important difference: machine learning is only used to generate an RGBA volume, which is then rendered with a ray marching algorithm with no learned parameters. This is important as it gives us an interpretable volume, which may lead to better viewpoint generalization.
In our evaluation of hybrid rendering approaches, we employ the Deep Appearance Model of Lombardi et al. (2018)
that provides a textured mesh representation. The method learns a Variational Autoencoder (VAE) model representing the dynamic mesh and view-dependent texture of a specific person's face. While we also evaluate our model on human faces, our method does not require precise mesh tracking or other forms of pre-processing. Instead, it is trained end-to-end, using only raw images as supervision. Volumetric representations such as ours are also able to better represent complex surfaces like hair that are difficult to model using meshes.
2.5. Novel View Synthesis
Novel view synthesis aims to produce RGB images of novel views given a set of input RGB images. Typically, these methods use a geometric proxy to assist in reprojecting 3D points back into the input images and some blending is performed to produce a final pixel color. Buehler et al. (2001) and Davis et al. (2012)
use heuristics to blend contributions of different images based on the rays from the geometric proxy to each camera.
Hedman et al. (2018) uses neural networks to determine blending weights, which can overcome inaccurate geometric proxies. Zhou et al. (2018) skip the geometric proxy altogether and use a neural network to compute blending weights of each image projected along a set of planes. Penner and Zhang (2017) compute a soft volumetric representation using MVS depth maps to perform novel view synthesis. Unlike most novel view synthesis techniques, our method operates on sequences and creates an animatable model.
To compute geometric proxies, a multi-view stereo method is often employed. While free-viewpoint video methods (Collet et al., 2015; Prada et al., 2016) rely on a sophisticated combination of multi-view stereo techniques (including silhouettes and MVS) to reconstruct many kinds of objects, our method creates novel views of dynamic scenes with a single generative framework. In addition, our model's latent embedding of the scene allows us to generate novel animations more flexibly by producing new embedding sequences.
We present an end-to-end pipeline for rendering images from novel views with only image supervision that leverages an internal 3D volumetric representation. There are two main parts to the method: an encoder-decoder network that converts input images into a 3D volume V(x), and a differentiable raymarching step that renders an image from the volume V given a set of camera parameters. The method can be thought of as an autoencoder whose final layer is a fixed-function (i.e., no free parameters) volume rendering operation.
Formally, we model a volume that maps 3D positions, x∈R3, to a local RGB color and differential opacity at that point,
(1) V:R3→R4,V(x)=(Vrgb(x),Vα(x)),
where Vrgb(x)∈R3 is the color at x and Vα(x)∈R is its differential opacity in the range [0,∞], with 0 representing full transparency. The purpose of a semi-transparent volume is two-fold: first, it is a softening of a discrete volume representation which enables gradients to flow for learning; second, it allows us to model semi-transparent objects or bundles of thin structures that appear translucent at limited resolutions, like hair.
Fig. 2 shows a visual representation of the pipeline. We first capture a set of synchronized and calibrated video streams of a performance from different viewpoints. Next, an encoder network takes images from a subset of the cameras for each time instant and produces a latent code z that represents the state of the scene at that time. A volume decoder network produces a 3D volume given this latent code, V(x;z), which yields an RGBα value at each point x. Finally, an accumulative ray marching algorithm renders the volume from a particular point-of-view. We train this system end-to-end by reconstructing each of the input images and minimizing the squared pixel reconstruction loss over the entire training set. At training time, we run through the entire pipeline to train the weights of the encoder-decoder network. At inference time, we produce a stream of latent codes z (either the sequence of latent codes produced by the training images or a novel generated sequence) and decode and render in real time.
4. Encoder Network
Figure 3. Encoder architecture. Inputs are images from a subset of K-cameras, passed through camera-specific CNNs. The latent variable z is concatenated with the L-dimensional conditioning variable c before being passed to the decoder (see §5).
The main component of our system that enables novel sequence generation is the encoder-decoder architecture, where the scene's state is encoded using a consistent latent representation z∈R256. A traversal in this latent space can be decoded into a novel sequence of volumes that can then be rendered from any viewpoint (see §5 for details). This is in contrast to methods that rely on specialized mesh constructions per frame which only allow for playback (Collet et al., 2015) or limited control over the generative process (Prada et al., 2016) . Moreover, this representation allows for conditional decoding, where only part of the scene's state is modified on playback (i.e., expression during speech, view-dependent appearance effects, etc.). The encoder-decoder architecture naturally supports this capability without requiring specialized treatment on the decoder side so long as paired samples of the conditioning variable are available during training.
To build the latent space, the information state of the scene at any given time is codified by encoding a subset of views from the multi-camera capture system using a convolutional neural network (CNN). The architecture of the encoder is shown in Fig. 3. Each camera view is passed through a dedicated branch before being concatenated with those from other views and further encoded down to the final shape. Although using all camera views as input is optimal from an information-theoretic perspective, we found that using K=3 views worked well in practice, while being much more memory and computationally efficient. To maximize coverage, we select a subset of cameras that are roughly orthogonal, although our system is not especially sensitive to the specific choice of views. In practice, we used the frontal, left and right-most camera views and downsampled the images by a factor of 8 to size 334×512 pixels.
To generate plausible samples during a traversal through the latent space, the generative model needs to generalize well between training samples. This is typically achieved by learning a smooth latent space. To encourage smoothness, we use a variational architecture (Kingma and Welling, 2013) . The encoder outputs parameters of a diagonal 256-dimensional Gaussian (i.e., μ and σ
), whose KL-divergence from a standard Normal distribution is used as regularization. Generating an instance involves sampling from this distribution using the reparameterization trick, and decoding into the volumetric components described in §
In addition to encouraging latent space smoothness, the variational architecture also ensures that the decoder makes use of the conditioning variable when it is trained jointly with the encoder, as described in §7. Specifically, since the variational bottleneck maximizes the non-informative latent dimensions (Higgins et al., 2017) , information pertaining to the conditioning variable is projected out of the latent space, leaving the decoder no choice but to use the conditioning variable in its reconstruction.
This method of conditioning can be applied using any auxiliary information available to the user for controlling the rendered output. In §8
we show experiments demonstrating this for a few types of conditioning information. Of particular importance is view-conditioning, which allows view-dependent effects, such as specularity, to be rendered correctly. When viewed in VR, the auxiliary information in the form of a view-vector can be obtained from the relative orientation of the headset in the virtual scene.
Figure 4. Voxel grid decoders. (a) Convolutional and (b) Linear basis decoders. L denotes the size of the conditioning variable.
5. Volume Decoders
In this section we discuss parameterizing the RGBα volume function V using different neural network architectures which we call volume decoders
. We discuss possible representations for the volumes (voxel grids and multi-layer perceptrons) and a method for increasing the effective resolution by using warping fields. Finally, we discuss view-conditioning for modeling view-dependent appearance.
5.1. MLP Decoders
One possible model for the volume function V(x;z) at point x with state z, is an implicit one with a series of fully connected layers with non-linearities. A benefit of this approach is that we're not restricted by voxel grid resolution or storage space. Unfortunately, in practice an MLP requires prohibitive size to produce high-quality reconstructions. We must also evaluate the MLP at every step along each ray in the ray-marching process (see §6), imposing an equally restrictive upper bound on the MLP complexity for real-time applications.
5.2. Voxel Grid Decoders
Rather than trying to model the entire volume implicitly with an MLP, we may instead assume that the volume function can be modeled as a discrete 3D grid of voxels. We produce this explicit 3D voxel grid as the output tensor of a neural network. Let the tensor
Y∈RC×D×D×D represent a D×D×D grid of values in RC, with C the channel dimensions. Define S(x;Y):R3→RC
to be an interpolation function that samples from the grid
Y by scaling continuous values in the range [−1,1] to the grid [1,D] along each dimension, followed by trilinear interpolation. We can define a volume decoder for a 3D cube with center at xo and sides of size W,
(2) V(x;z) =S(x−xoW/2;g(z)),
where g(z) is a neural network that produces a tensor of size 4×D×D×D. Note that we only evaluate the decoder function inside the volume it covers by computing intersections with its bounding volume.
In practice, we use either a convolutional architecture or a series of fully-connected layers to implement g(z). In the former case, we first apply a fully-connected layer and non-linearity to transform z into a 1024-dimensional representation and reinterpret the resulting vector as a 1×1×1 cube with 1024 channels. We experiment with two convolutional architectures, one with a final size of 323 (achieved with 5 transposed convolutions and running at 90Hz) and one with a final size of 1283 (achieved with 7 transposed convolutions and running at 22Hz). As an alternative, we consider a bottleneck architecture capable running at 90Hz consisting of 3 fully-connected layers with output sizes of 128, 4, and 1283×4, respectively, with the last layer representing the decoded volume. After decoding the volume, we apply a softplus function to the RGBα values to ensure they are non-negative. Both decoder variants are illustrated in Fig. 4.
5.3. Warping Fields
On their own, voxel grids are limited because they can only represent details as small as a single voxel and are computationally expensive to evaluate and store at high resolution. Additionally, they are wasteful in typical scenes where much of the scene consists of empty space. Common solutions to these problems are spatial acceleration structures like octrees (e.g., Riegler et al. (2017) ), but it's difficult to modify these to work in a learning setting. They also typically require that the distribution of objects within the structure is known a priori, which is not true in our case as we do not use any 3D training data.
To solve these problems, we propose to use warping fields to both alter the effective resolution of the voxel volumes as well as model motion more naturally. In our warping formulation, we produce a template RGBα volume T(x) and a warp volume W−1(x). Each point in the warp volume gives a corresponding location in the template volume from which to sample, making this an inverse warp as it maps from output positions to template sample locations. As before, both template and warp are decoded from a dynamic (per-frame) latent code z, which we drop from the notation for conciseness.
The choice of inverse warps rather than forward warps allows representing resolution-increasing transformations by mapping a small area of voxels in the output space to a larger area in the template space without requiring additional memory. Thus, the inverse warp can represent details in the output space with higher resolution than uniform grid sampling, but remains well-defined everywhere in the output space, which is necessary for providing usable gradients during learning.
Formally, we define the inverse warp field,
(3) W−1(x)→yx,y∈R3,
where x is a 3D point in the output (rendering) space and y a 3D point in the RGBα template space. To generate the final volume value, we first evaluate the value of the inverse warp and then sample the template volume T at the warped point,
(4) VRGBα(x)=TRGBα(W−1(x)).
5.4. Mixture of Affine Warps
An important piece of including warp fields is determining how they're produced. As we show later, the architecture of the warp field decoder makes a large difference on the quality of the model.
A straightforward approach to decoding warp fields would be to use deconvolutions to produce a warp field with freely-varying template sample points at each output point. This parameterization, however, is too flexible, resulting in overfitting and poor generalization to novel views as we show in our experimental results. We instead take the approach that the basic building block of a warp field should be an affine warp. Since a single affine warp can't model non-linear bending, we use a spatial mixture of affine warps to produce an inverse warp field.
We write the affine mixture as,
(5) W−1(x)=∑iAi(x)ai(x),
(6) withAi(x)=Ri(si∘(x−ti)),ai(x)=wi(Ai(x))∑jwj(Aj(x)),
where Ai(x) is the ith affine transformation, {Ri,si,ti} define the rotation, scaling, and translation of the ith affine transformation parameters, ∘ is element-wise multiplication, and wi(x) is the weight volume of the ith warp. Note that we sample the spatial mixture weight wi after warping ("warped weights"). The intuition behind this is that the warped space represents different parts of the scene, and the weighting function should be in that space as well. This can be viewed as an extension of linear blend skinning (Lewis et al., 2000) applied to a volumetric space.
To compute the transformation parameters {Ri,si,ti} and the weighting volume wi we use 2 fully-connected layers after the encoding z. For rotation, we produce a rotation quaternion vector which is normalized and transformed into a rotation matrix. Before outputting the values of the weighting volume, we apply exp(⋅) to ensure the weights are non-negative. Unlike the voxel volume V, we clamp samples outside the weighting volume wi to the surface otherwise the warps can get "stuck" early in training if they land outside the volume. In practice, we found that a mixture of 16 warps provides sufficient expressiveness. In all experiments where warping is used, we learn an additional global warp {Rg,sg,tg} (with parameters produced by MLP) that is applied to x before the warping field, and we use a 323 voxel grid to represent the warp field as we found that the low resolution provides smoothness from the trilinear interpolation that helps learning.
5.5. View Conditioning
In order to model view-dependent appearance, we opt to condition the RGB decoder network on the viewpoint. This allows us to model specularities in a data-driven way, without specifying a particular functional form. To do this, we input the normalized direction of the camera to the decoder alongside the encoding. Note that for view-conditioned models, we use separate convolutional branches to produce the RGB values and α values as we only want to condition the RGB values on the viewpoint.
Irgb=0; Iα=0; t=tmin;
dα(t)=min(Iα+ΔVα(ro+trd),1)−Iα
Irgb=Irgb+Vrgb(ro+trd)dα(t)
Iα=Iα+dα(t)
t=t+Δ
until Iα=1 or t>tmax
return Irgb,Iα
ALGORITHM 1 Accumulative rendering of ray ro+trd with segment t∈[tmin,tmax] intersecting V. (Integration of Eq. (7) with stepsize Δ.)
6. Accumulative Ray Marching
We formulate a rendering process for semi-transparent volumes that mimics front-to-back additive blending: As a camera ray traverses a volume of inhomogeneous material, it accumulates color in proportion to the local color and density of the material at each point along its path.
6.1. Semi-Transparent Volume Rendering
We generate images from a volume function V(x) by marching rays through the volume it models. To model occlusions, the ray accumulates not only color but also opacity. If the accumulated opacity reaches 1 (for example, when the ray traverses an opaque region), then no further color can be accumulated on the ray. In particular, the color Irgb(p) at a pixel p in the focal plane of a camera with center ro∈R3 and image-to-world transformation P−1 is given by raymarching in the unit direction rd=(P−1p−ro)/∥P−1p−ro∥∈R3. This leads to the rendering process
(7) Irgb(p) =∫tmaxtminVrgb(ro+trd)dα(t)dt dt,
(8) withα(t) =min(∫ttminVα(ro+srd) ds,1),
where t∈[tmin,tmax] denotes the segment of the ray that intersects the volume modeled by V, and min(⋅,1) in Eq. (8) ensures that the accumulated ray opacity is clamped at 1. Furthermore, we set the final image opacities to Iα(p)=α(tmax).
Algorithm 1 shows the computation of the output color for a ray intersecting the volume V, and represents the numerical integration of Eqs. (7–8) using the rectangle rule111We use the rectangle rule twice with samples on the right of each step interval and backwards differences to discretize the derivative of α(t).. Importantly, this image formation model is differentiable, which allows us to optimize the parameters of the volume V to match target images. In practice, we set the step size to be 1128th the size of the volume, which provides adequate voxel coverage and allows the rendering process to run at 90Hz in an OpenGL shader at 512×600 resolution, enabling real-time stereo in Virtual Reality.
6.2. Hybrid Rendering
While the semi-transparent volume representation is very versatile, certain kinds of scene content can be more efficiently represented at high resolution using surface-based representations combined with unwrapped texture maps. One example is fine detail in human faces, for which specialized capture systems are available and commonly use texture resolutions larger than 10242 (Beeler et al., 2011; Lombardi et al., 2018; Fyffe et al., 2017) .
The volume representation described above offers a natural integration with such existing mesh-based representations. Rendering proceeds as described in Algorithm 1 with one modification: we set tmax to the mesh depth for all rays that intersect the mesh. Whenever one of these rays reaches tmax, any remaining color throughput is filled with the color of the mesh at the intersection.
This technique can also be used during learning to avoid expending representational power in these parts of the scene. As we show in §8.6, the resulting semi-transparent volume learned using the hybrid rendering process described above naturally avoids occluding the mesh in areas where the mesh provides a higher-fidelity representation.
7. End-to-end Training
In this section we discuss the details of training our method. Training the system consists of training the weights θ
of the encoder-decoder network. We discuss the estimation of a per-camera color calibration matrix and static background image, the construction of our loss function, and reconstruction priors that improve accuracy.
7.1. Color Calibration
Although we have geometrically calibrated the cameras in our multi-view system, we have not color calibrated them relative to one-another. We need to ensure that one radiance value will be converted to the same pixel value for each of the cameras. To do this, we introduce a per-camera and per-channel gain g and bias b that is applied to our reconstructed image before comparing to ground truth. This allows our system to account for slight differences in overall intensity in the image.
7.2. Backgrounds
In our training data we often have static backgrounds that the algorithm will try to reconstruct. To ensure that our algorithm only reconstructs the object of interest, we estimate a per-camera background image I(bg)rgb. The background image is static across the entire sequence, capturing only stationary objects that are generally outside of the reconstruction volume.
We obtain a final image ˆIrgb from a specific view by raymarching all pixels p according to Eq. (7) and merging Irgb(p) with its corresponding background pixel I(bg)rgb(p) according to the remaining opacity when exiting the volume,
(9) ˆIrgb(p)=(1−Iα(p))I(bg)rgb(p)+(gIrgb(p)+b).
This background estimation process greatly reduces the amount of artifacts in the reconstruction.
7.3. Reconstruction Priors
Figure 5. Warping method evaluation. In this experiment we evaluate different techniques for warping. We evaluate on three different datasets: a moving hand, swinging hair, and dry ice smoke. We evaluate models with: no warping, convolutional warp (using transposed convolutions), convolutional warp (using trilinear upsampling followed by convolution), no warp affine mixture, where the spatial weighting volume wi is not warped before evaluating (Eq. (13)), and warp-space affine mixture, as described in Eq. (6). In all cases, our warp-space affine mixture outperforms the other models.
Without using priors, the reconstructed volumes tend to include smoke-like artifacts. These artifacts are caused by slight differences in appearance from different viewpoints due to calibration errors or view-dependent effects. The system can learn to compensate for these differences by adding a small amount of opacity that becomes visible from one particular camera. To reduce these artifacts, we introduce two priors.
The first prior regularizes the total variation of the log voxel opacities,
where the sum is performed over all the voxel centers x and N is the number of voxels. This term helps recover sharp boundaries between opaque and transparent regions by enforcing sparse spatial gradients. We apply this prior in log space to increase the sensitivity of the prior to small α values because the artifacts tend to be mostly transparent.
The second prior is a beta distribution
(Beta(0.5,0.5)) on the final image opacities Iα(p). We write the regularization term using the negative log-likelihood of the beta distribution,
(11) JB(Iα)=1P∑pλB[log(Iα(p))+log(1−Iα(p))],
where p is an image pixel and P is the number of pixels. This prior reduces the entropy of the exit opacities and is based on the intuition that most of our rays should strike the object or the background; fewer rays will graze the surface of the object, picking up some opacity but not saturating.
7.4. Training Hyperparameters
Our full training objective is
ℓ(θ)= 1P∑p∥∥ˆIrgb(p)−I∗rgb(p)∥∥2+
(12) λKLDKL(z∥N(0,1))+Jtv(Vα)+JB(Iα),
where I∗rgb(p) is the ground truth image and DKL(z∥N(0,1)) is the KL divergence between the latent encoding z and a standard normal distribution used in a variational autoencoder (Kingma and Welling, 2013) .
We use Adam (Kingma and Ba, 2015) to optimize the loss function. In our experiments, we set λKL=0.001, λtv=0.01, and λB=0.1 and we use a batch size of 16 with a fixed learning rate of 10−4 on the encoder-decoder network weights (the estimated background images I(bg) and per-camera gain and bias g,b are given separate learning rates of 10−1 and 10−3, respectively). We randomly sample 128×128 pixels from the image to reduce memory usage. While raymarching, we compute step sizes in normalized voxel space (i.e., [−1,1]3) as it makes the end of the ray the expected saturation point at network initialization. We train for about 500,000 iterations, depending on dataset size, which takes 10 days on a single NVIDIA Tesla V100.
No view conditioning
View conditioning
Table 1. View conditioning comparison. Conditioning the decoder on viewpoint improves the reconstruction on a set of validation viewpoints.
8. Experiments
We perform a number of quantitative and qualitative experiments to validate our model. For all experiments, we use the convolutional volume decoder with size 1283. We show that the design choices of our model provide a good compromise between quality and speed, and that the model generalizes to new viewpoints. We show results of our method on objects that are typically hard to reconstruct (e.g., fuzz, smoke, and hair) and demonstrate the method combined with traditional triangle rasterization. Finally, we demonstrate animation of our models by interpolating in the latent space and driving the reconstruction from user input.
To capture data, we used a multi-camera capture system consisting of 34 4096×2668-resolution 30Hz color cameras placed on a hemisphere with a radius of approximately one meter. We calibrate the camera system using an icosahedral checkerboard pattern (Ha et al., 2017) . The raw images and camera calibration are then used as the only input to our method.
In our quantitative experiments, we compare mean-squared error of pixel reconstructions on the training cameras and also on a set of 7 held-out validation cameras. This allows us to test how well our model extrapolates to novel viewpoints.
Fuzzy Toy
Background Model
Known BG / priors 87.4 197.6
Known BG / no priors 76.1 330.5
Learned BG / priors 115.4 183.7
Learned BG / no priors 94.1 281.7
No BG / priors 208.8 386.7
No BG / no priors 86.6 727.6
Table 2. Evaluation of priors and background model. We show training and validation MSE for the "fuzzy toy" object with and without priors and using a known background, learned background, and no background model. Surprisingly, the learned background model with priors outperforms a known background.
8.1. Warping Method
To validate our warp representation, we compare against several variants: a model with no warping, a model with a warp produced by a convolutional neural network, a model that does not apply the warp before computing the affine mixture weight, and the proposed affine mixture warp model (i.e., Eq. (5)).
Figure 6. Qualitative results. In this figure we show ground truth, reconstructed images, and a visualization of the root-mean-squared error for each pixel for 3 datasets on 3 held-out validation viewpoints.
For the model that does not apply the warp to compute the mixture, we modify Eq. (6) as follows:
(13) withAi(x)=Ri(si∘(x−ti)),ai(x)=wi(x)∑jwj(x).
The main difference is that the mixture is done in "world" space rather than in warped space. Since the mixture weights are not warped, the decoder must match any motion of the template by moving the mixture weights.
Fig. 5 shows the results of our warping evaluation on three datasets: a moving hand, swinging hair, and dry ice smoke, each approximately 20 seconds long. In each dataset, our affine mixture model outperforms models with no warp and outperforms models with a convolutional warp field on the validation cameras. In particular, the convolutional warp fields completely fail on the "hair swing" dataset. The results also show that modeling the weighting volume in warp space is better than in "world" space.
8.2. View-Conditioning
An important part of rendering is being able to model view-dependent effects such as specularities. To do this, we can condition the RGB decoder Vrgb on the viewpoint of the rendered view. This allows the network to change the color of certain parts of the scene depending on the angle it's viewed from.
Table 1 shows a quantitative experiment comparing a view-conditioned model to a non-view-conditioned model for a human face. The results show that the view-conditioned model is better able to model the scene from novel viewpoints. This happens not only because the view-conditioned model can represent some of the view-dependent appearance of the scene, but also because the non-view-conditioned model incorrectly reconstructs extra semi-transparent voxels near the surface of the object to explain view-dependent phenomena.
8.3. Priors and Background Estimation
We impose several priors on the reconstructed volume to help reduce the occurrence of artifacts in the reconstruction. In this experiment, we evaluate the effectiveness of background estimation and the priors on the reconstructed volume quality in terms of mean-squared-error on the validation viewpoints. We evaluate 3 different scenarios: known background image, learned background image, and no background model, each scenario evaluated with and without priors.
Table 2 shows the results of this experiment. For each background setting, using the priors improves performance on the validation viewpoints. Surprisingly, learning a background image outperformed using a known background image, but the improvement is small.
8.4. Qualitative Results
Figure 7. Warped and template volumes through time. This figure depicts several frames from two sequences, showing (a) the final warped volumes, and (b) the learned template volumes before warping. For the face sequence, most changes are explained via warping and the face remains static in template space. However, because tracking is not explicitly enforced, the network may learn to represent motion as different templates if it is more parsimonious to do so, as can be seen for finger motion in the hand sequence. Note also how in the template, space is allocated to regions requiring texture details while flat regions are compressed, like the phalanges and the forearm in the hand example.
In this section, we show qualitative results on a number of different types of objects. We compare our renderings to held-out ground truth views and also demonstrate hybrid rendering with a mesh representation.
Fig. 6 shows renderings produced by our method compared to ground truth images for several validation (held-out) viewpoints. The renderings demonstrate that our method is able to model difficult phenomena like fuzz, smoke, and human skin and hair. The figure also shows typical artifacts produced by our reconstructions: typically, a very light smokey pattern is added which may be modeling view-dependent appearance for certain training cameras. Using more camera views tends to reduce all artifacts.
Fig. 7 shows how the template volume changes through time across several frames, compared to the final warped volume that is rendered. Ideally, object motion should be represented entirely by the warp field. However, this does not always happen when representing such changes requires more resolution than is available (e.g., as would be necessary to cleanly separate the rim of the glasses from the side of the head) or because representing the warp field becomes too complex.
Fig. 11 visualizes the learned RGBα volumes. While there is no explicit surface reconstruction objective in our method, we can visualize isosurfaces of constant opacity, shown in (b). Ideally, fully opaque surfaces such as the hand in the first row would be represented by delta functions in Vα(x), but we find that the method trades off some opacity to better match reconstruction error in the training views. Note how translucent materials such as the glasses in the second row, or materials which appear translucent at coarse resolution, such as hair in the 3rd and 4th rows, are modeled with lower opacity values but retain a distinct structure particular to the object.
8.5. Single Frame Estimation
We evaluate the quality of our algorithm using only a single frame as input. In some ways this should make the problem easier as there is less information to represent within the model. On the other hand, our model is less able to exploit regularities and redundancies of motion. This experiment helps us disentangle those factors and determine the contribution of each component of the model.
To perform this experiment, we run our model on only a single frame. Although the encoder will produce a constant value, we keep the entire encoder-decoder network intact while training. We also compare to "direct" volume estimation, i.e., we directly estimate the template voxel volume T and warp values W−1 without the encoder-decoder network.
Fig. 12 shows the results of the experiment on four objects we captured as well as one scene from a publicly available MVS dataset (Aanæs et al., 2016) . As shown, the "direct" voxel/warp estimation contains artifacts and incorrectly reconstructs the object. This experiment shows that the convolutional architecture provides a regularization that, even in the case of a single frame, allows us to accurately recover and re-render objects. Similar observations have been made in a recent work on deep image priors (Ulyanov et al., 2018) .
For comparison, we show the same frame reconstructed as a mesh using a commercial multi-view stereo system (Agisoft, 2019) and the open-source multi-view stereo system COLMAP (Schönberger and Frahm, 2016; Schönberger et al., 2016) , as well as a comparison to space carving (Kutulakos and Seitz, 2000) . Characteristically, the recovered resolution in the MVS texture map is greater than what we can achieve with volume reconstructions on current hardware. However, the mesh also shows artifacts in thin regions, like the frame of the glasses, and translucent regions, like the glass material and the hair at the top of the head, or the smoke in the 4th row. Space carving heavily relies on consistent appearance across views and struggles with translucent materials.
8.6. Animating
Figure 8. Novel content generation. First row: Interpolation in latent space. Leftmost and rightmost frames are reconstructed from real image encodings, intermediate frames are reconstructed from linear interpolation between the left and right codes. Second row: real-time avatar driving based on user input. The magenta dot represents the position of the user's hand, which the avatar's head turns to track.
With the proposed reconstruction method, we can playback and re-render the captured data from many different angles. We not only want to extrapolate in viewpoint, but also in the content of the performance. Our latent variable model allows us to create new sequences of content by modifying the latent variables.
Fig. 8 shows two examples of content modification by interpolating latent codes and by changing the conditioning variable based on user input. Our latent space interpolation shows that the encoder network learns a compact representation of the scene. In the second example, we condition the decoder on the head pose of the subject, allowing us to create novel sequences in real time.
Figure 9. Combining mesh and volumetric representations (images best viewed at high-resolution). Given an initial textured mesh model, all rays which intersect the mesh have tmax set to the depth of the mesh along each ray. Rays which terminate at tmax accumulate color from the mesh based on remaining opacity. From left to right: masked mesh (placed into voxel volume during learning), voxel representation, hybrid rendering, mesh-only reconstruction. The learned voxel representation avoids occluding the mesh to achieve a higher-quality reconstruction.
Fig. 9 shows the results of combining a textured mesh representation with our voxel representation. Textured meshes can efficiently and accurately represent fine detail in regions of the face like the skin and eyes while the voxel representation excels at modeling hair. For a mesh model we use the Deep Appearance Model of Lombardi et al. (2018) trained to reconstruct the same face we used to train our volume encoder/decoder network.
Figure 10. Depth map showing where raymarching terminates. The hole in the chest indicates that those rays passed through the entire volume and only terminated at the background. Since the chest area has limited texture variation and is a similar color to the background, this artifact does not greatly affect reconstruction error even when viewed from novel viewpoints.
Figure 11. Learned reconstructions for several objects showing (a) rendered color and opacity, (b) isosurfaces of Vα(x) for different levels of differential opacity, (c) contour plots of the differential opacity, (d) sliced render showing the interior of the volume at the x-y cut, (e) the complete render from the same viewpoint, and (f) the unwarped template.
In this paper, we presented a method for modeling objects and scenes with a semi-transparent volume representation that we learn end-to-end from multi-view RGB images. We showed that our method can convincingly reconstruct challenging objects such as moving hair, fuzzy toys, and smoke. Our method requires no explicit tracking, and can be run in real time alongside traditional triangle rasterization.
One limitation of our method is that given a surface with limited texture, our estimated volume may represent that surface as transparent and place its color in the background, so long as doing so does not cause otherwise occluded surfaces to appear. This is a challenge that affects traditional 3D reconstruction methods as well. With our method, however, the reconstruction degrades gracefully and still produces perceptually-pleasing results thanks to our image-space loss function. Fig. 10 shows an example of this via a depth map computed as the distance each ray travels before saturating or hitting the bounding box. In a more practical setting, this could be addressed simply by capturing the sequence with a bright background such as a green screen.
Although our method can handle transparent objects, like plastic bottles, the method doesn't currently consider refractive surfaces. We believe the approach can be extended to model refraction and even reflection, and we leave that to future work. Our model can represent dull specular highlights through view conditioning but high-frequency specular highlights are not correctly represented.
Figure 12. Single frame estimation. In this experiment, we reconstruct only a single frame rather than a sequence. For comparison, from left to right, the first three columns show MVS-based reconstructions: (a) the point cloud recovered by COLMAP (Schönberger and Frahm, 2016; Schönberger et al., 2016) , and (b) the mesh after Poisson reconstruction, and (c) the textured mesh recovered by Agisoft Metashape (2019) . Column (d) shows a colored voxel occupancy reconstruction using space carving (Kutulakos and Seitz, 2000) , and column (e) shows "direct" estimation of the voxel grid (i.e., the T and W−1 are directly estimated instead of the result of an encoder-decoder network). Finally, (f) shows the results of our full pipeline trained only on a single frame. The results show that the encoder-decoder architecture facilitates recovering accurate estimates.
The latent space is the feature enabling us to generate dynamic content, but we do not explicitly model any temporal dynamics. This is not a problem for playback, since the playback sequence implicitly encodes the same temporal dynamics as the recording. It is also not a problem when driving the representation from user input, so long as that user input has reasonable temporal dynamics of its own. However, if we traverse the latent space in some manner not guided by temporal information, we may generate sequences which, while visually accurate, do not represent real behaviors of the object we modeled.
Volumetric representations typically suffer from limited resolution due to the cubic relationship between resolution and memory requirement. In this work, we showed some ways to increase the effective resolution without simply increasing the voxel grid resolution by using warping fields. We believe that we can further improve this approach to achieve a level of fidelity and resolution previously only achievable with traditional textured mesh surfaces.
Aanæs et al. (2016) Henrik Aanæs, Rasmus Ramsbøl Jensen, George Vogiatzis, Engin Tola, and Anders Bjorholm Dahl. 2016. Large-Scale Data for Multiple-View Stereopsis. International Journal of Computer Vision (IJCV) (2016).
Agisoft (2019) Agisoft. 2019. Metashape. https://www.agisoft.com/.
Atcheson et al. (2008) Bradley Atcheson, Ivo Ihrke, Wolfgang Heidrich, Art Tevs, Derek Bradley, Marcus Magnor, and Hans-Peter Seidel. 2008. Time-resolved 3D Capture of Non-stationary Gas Flows. ACM Trans. Graph. 27, 5 (2008).
Beeler et al. (2011) Thabo Beeler, Fabian Hahn, Derek Bradley, Bernd Bickel, Paul Beardsley, Craig Gotsman, Robert W. Sumner, and Markus Gross. 2011. High-quality Passive Facial Performance Capture Using Anchor Frames. ACM Trans. Graph. 30, 4 (2011).
Bonet and Viola (1999) Jeremy S. De Bonet and Paul A. Viola. 1999. Poxels: Probabilistic Voxelized Volume Reconstruction. In International Conference on Computer Vision (ICCV).
Broadhurst et al. (2001) Adrian Broadhurst, Tom Drummond, and Roberto Cipolla. 2001. A Probabilistic Framework for Space Carving. In International Conference on Computer Vision (ICCV).
Buehler et al. (2001) Chris Buehler, Michael Bosse, Leonard McMillan, Steven Gortler, and Michael Cohen. 2001. Unstructured Lumigraph Rendering. In Conference on Computer Graphics and Interactive Techniques (SIGGRAPH).
Collet et al. (2015) Alvaro Collet, Ming Chuang, Pat Sweeney, Don Gillett, Dennis Evseev, David Calabrese, Hugues Hoppe, Adam Kirk, and Steve Sullivan. 2015. High-quality Streamable Free-viewpoint Video. ACM Trans. Graph. 34, 4 (2015).
Dai et al. (2017) J. Dai, H. Qi, Y. Xiong, Y. Li, G. Zhang, H. Hu, and Y. Wei. 2017. Deformable Convolutional Networks. In International Conference on Computer Vision (ICCV).
Davis et al. (2012) Abe Davis, Marc Levoy, and Fredo Durand. 2012. Unstructured Light Fields. Computer Graphics Forum 31, 2pt1 (2012).
Fitzgibbon and Zisserman (2005) Andrew Fitzgibbon and Andrew Zisserman. 2005. Image-based Rendering Using Image-based Priors. In International Conference on Computer Vision (ICCV).
Furukawa and Hernández (2015) Yasutaka Furukawa and Carlos Hernández. 2015. Multi-View Stereo: A Tutorial. Foundations and Trends in Computer Graphics and Vision 9, 1-2 (2015).
Furukawa and Ponce (2010) Yasutaka Furukawa and Jean Ponce. 2010. Accurate, Dense, and Robust Multiview Stereopsis. Pattern Analysis and Machine Intelligence (PAMI) 32, 8 (2010).
Fyffe et al. (2017) G. Fyffe, K. Nagano, L. Huynh, S. Saito, J. Busch, A. Jones, H. Li, and P. Debevec. 2017. Multi-View Stereo on Consistent Face Topology. Computer Graphics Forum 36, 2 (2017).
Goesele et al. (2007) Michael Goesele, Noah Snavely, Brian Curless, Hugues Hoppe, and Steven M. Seitz. 2007. Multi-View Stereo for Community Photo Collections. In International Conference on Computer Vision (ICCV).
Ha et al. (2017) H. Ha, M. Perdoch, H. Alismail, I. S. Kweon, and Y. Sheikh. 2017. Deltille Grids for Geometric Camera Calibration. In International Conference on Computer Vision (ICCV).
Hawkins et al. (2005) Tim Hawkins, Per Einarsson, and Paul Debevec. 2005. Acquisition of Time-varying Participating Media. ACM Trans. Graph. 24, 3 (2005).
Hedman et al. (2018) Peter Hedman, Julien Philip, True Price, Jan-Michael Frahm, George Drettakis, and Gabriel Brostow. 2018. Deep Blending for Free-Viewpoint Image-Based Rendering. ACM Trans. Graph. 37, 6 (2018).
Higgins et al. (2017) Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. 2017. β-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework. In International Conference on Learning Representations (ICLR).
Ikits et al. (2004) Milan Ikits, Joe Kniss, Aaron Lefohn, and Charles Hansen. 2004. GPU Gems: Programming Techniques, Tips and Tricks for Real-Time Graphics (Chapter 39, Volume Rendering Techniques). Addison Wesley.
Innmann et al. (2016) Matthias Innmann, Michael Zollhöfer, Matthias Nießner, Christian Theobalt, and Marc Stamminger. 2016. VolumeDeform: Real-Time Volumetric Non-rigid Reconstruction. In European Conference on Computer Vision (ECCV).
Isola et al. (2017) Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-Image Translation with Conditional Adversarial Networks.
Computer Vision and Pattern Recognition (CVPR)
Jaderberg et al. (2015) Max Jaderberg, Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu. 2015. Spatial Transformer Networks. In Advances in Neural Information Processing Systems (NeurIPS).
Kalantari et al. (2016) Nima Khademi Kalantari, Ting-Chun Wang, and Ravi Ramamoorthi. 2016. Learning-based View Synthesis for Light Field Cameras. ACM Trans. Graph. 35, 6 (2016).
Karras et al. (2018) Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2018. Progressive Growing of GANs for Improved Quality, Stability, and Variation. In International Conference on Learning Representations (ICLR).
Kim et al. (2018) Hyeongwoo Kim, Pablo Garrido, Ayush Tewari, Weipeng Xu, Justus Thies, Matthias Nießner, Patrick Pérez, Christian Richardt, Michael Zollhöfer, and Christian Theobalt. 2018. Deep Video Portraits. ACM Trans. Graph. 37, 4 (2018).
Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In International Conference for Learning Representations (ICLR).
Kingma and Welling (2013) Diederik P. Kingma and Max Welling. 2013. Auto-Encoding Variational Bayes. In International Conference on Learning Representations (ICLR).
Kutulakos and Seitz (2000) Kiriakos N. Kutulakos and Steven M. Seitz. 2000. A Theory of Shape by Space Carving. International Journal of Computer Vision 38, 3 (2000).
Levoy (1988) Marc Levoy. 1988. Display of Surfaces from Volume Data. IEEE Computer Graphics and Applications 8, 3 (1988).
Lewis et al. (2000) J. P. Lewis, Matt Cordner, and Nickson Fong. 2000. Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-driven Deformation. In Conference on Computer Graphics and Interactive Techniques (SIGGRAPH).
Lombardi et al. (2018) Stephen Lombardi, Jason Saragih, Tomas Simon, and Yaser Sheikh. 2018. Deep Appearance Models for Face Rendering. ACM Trans. Graph. 37, 4 (2018).
Martin-Brualla et al. (2018) Ricardo Martin-Brualla, Rohit Pandey, Shuoran Yang, Pavel Pidlypenskyi, Jonathan Taylor, Julien Valentin, Sameh Khamis, Philip Davidson, Anastasia Tkach, Peter Lincoln, Adarsh Kowdle, Christoph Rhemann, Dan B Goldman, Cem Keskin, Steve Seitz, Shahram Izadi, and Sean Fanello. 2018. LookinGood: Enhancing Performance Capture with Real-time Neural Re-rendering. ACM Trans. Graph. 37, 6 (2018).
Merrell et al. (2007) Paul Merrell, Amir Akbarzadeh, Liang Wang, Philippos Mordohai, Jan-Michael Frahm, Ruigang Yang, David Nistér, and Marc Pollefeys. 2007. Real-Time Visibility-Based Fusion of Depth Maps. In International Conference on Computer Vision (ICCV).
Newcombe et al. (2015) Richard A. Newcombe, Dieter Fox, and Steven M. Seitz. 2015. DynamicFusion: Reconstruction and Tracking of Non-Rigid Scenes in Real-Time. In Computer Vision and Pattern Recognition (CVPR).
Nguyen-Phuoc et al. (2018) Thu H Nguyen-Phuoc, Chuan Li, Stephen Balaban, and Yongliang Yang. 2018. RenderNet: A Deep Convolutional Network for Differentiable Rendering from 3D Shapes. In Advances in Neural Information Processing Systems (NeurIPS).
Nießner et al. (2013) Matthias Nießner, Michael Zollhöfer, Shahram Izadi, and Marc Stamminger. 2013. Real-time 3D Reconstruction at Scale Using Voxel Hashing. ACM Trans. Graph. 32, 6 (2013).
Overbeck et al. (2018) Ryan S. Overbeck, Daniel Erickson, Daniel Evangelakos, Matt Pharr, and Paul Debevec. 2018. A System for Acquiring, Processing, and Rendering Panoramic Light Field Stills for Virtual Reality. ACM Trans. Graph. 37, 6 (2018).
Paschalidou et al. (2018) Despoina Paschalidou, Ali Osman Ulusoy, Carolin Schmitt, Luc Gool, and Andreas Geiger. 2018. RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials. In Conference on Computer Vision and Pattern Recognition (CVPR).
Penner and Zhang (2017) Eric Penner and Li Zhang. 2017. Soft 3D Reconstruction for View Synthesis. ACM Trans. Graph. 36, 6 (2017).
Prada et al. (2016) Fabián Prada, Misha Kazhdan, Ming Chuang, Alvaro Collet, and Hugues Hoppe. 2016. Motion Graphs for Unstructured Textured Meshes. ACM Trans. Graph. 35, 4 (2016).
Prock and Dyer (1998) Andrew Prock and Charles R. Dyer. 1998. Towards Real-Time Voxel Coloring. In Image Understanding Workshop.
Riegler et al. (2017) Gernot Riegler, Ali Osman Ulusoy, and Andreas Geiger. 2017. OctNet: Learning Deep 3D Representations at High Resolutions. In Computer Vision and Pattern Recognition (CVPR).
Roth and Black (2006) S. Roth and M. J. Black. 2006. Specular Flow and the Recovery of Surface Structure. In Computer Vision and Pattern Recognition (CVPR).
Savinov et al. (2016) Nikolay Savinov, Christian Häne, Lubor Ladicky, and Marc Pollefeys. 2016. Semantic 3D Reconstruction with Continuous Regularization and Ray Potentials Using a Visibility Consistency Constraint. In Computer Vision and Pattern Recognition (CVPR).
Scharstein and Szeliski (2002) Daniel Scharstein and Richard Szeliski. 2002. A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms. International Journal of Computer Vision (IJCV) (2002).
Schönberger and Frahm (2016) Johannes Lutz Schönberger and Jan-Michael Frahm. 2016. Structure-from-Motion Revisited. In Computer Vision and Pattern Recognition (CVPR).
Schönberger et al. (2016) Johannes Lutz Schönberger, Enliang Zheng, Marc Pollefeys, and Jan-Michael Frahm. 2016. Pixelwise View Selection for Unstructured Multi-View Stereo. In European Conference on Computer Vision (ECCV).
Seitz and Dyer (1997) Steven M. Seitz and Charles R. Dyer. 1997. Photorealistic Scene Reconstruction by Voxel Coloring.
Seitz and Dyer (1999) Steven M. Seitz and Charles R. Dyer. 1999. Photorealistic Scene Reconstruction by Voxel Coloring. International Journal of Computer Vision 35, 2 (1999).
Shu et al. (2018) Zhixin Shu, Mihir Sahasrabudhe, Riza Alp Guler, Dimitris Samaras, Nikos Paragios, and Iasonas Kokkinos. 2018. Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearance. In European Conference on Computer Vision (ECCV).
Sitzmann et al. (2018) V. Sitzmann, J. Thies, F. Heide, M. Nießner, G. Wetzstein, and M. Zollhöfer. 2018. DeepVoxels: Learning Persistent 3D Feature Embeddings. arXiv:1812.01024 [cs.CV] (2018).
Szeliski and Golland (1999) Richard Szeliski and Polina Golland. 1999. Stereo Matching with Transparency and Matting. International Journal of Computer Vision (IJCV) 32, 1 (1999).
Torresani et al. (2008) L. Torresani, A. Hertzmann, and C. Bregler. 2008. Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 30, 5 (2008).
Tulsiani et al. (2018) Shubham Tulsiani, Alexei A. Efros, and Jitendra Malik. 2018. Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Prediction. In Computer Vision and Pattern Recognition (CVPR).
Tulsiani et al. (2017) Shubham Tulsiani, Tinghui Zhou, Alexei A. Efros, and Jitendra Malik. 2017. Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistency. In Computer Vision and Pattern Recognition (CVPR).
Ulusoy et al. (2015) Ali Osman Ulusoy, Andreas Geiger, and Michael J. Black. 2015. Towards Probabilistic Volumetric Reconstruction Using Ray Potentials. In 3DV.
Ulyanov et al. (2018) Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2018. Deep Image Prior. In Conference on Computer Vision and Pattern Recognition (CVPR).
Wang et al. (2018) Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Guilin Liu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. 2018. Video-to-Video Synthesis. In Advances in Neural Information Processing Systems (NeurIPS).
Xu et al. (2014) Zexiang Xu, Hsiang-Tao Wu, Lvdi Wang, Changxi Zheng, Xin Tong, and Yue Qi. 2014. Dynamic Hair Capture Using Spacetime Optimization. ACM Trans. Graph. 33, 6 (2014).
Zach et al. (2007) Christopher Zach, Thomas Pock, and Horst Bischof. 2007. A Globally Optimal Algorithm for Robust TV-L1 Range Image Integration. In International Conference on Computer Vision (ICCV).
Zhou et al. (2018) Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. 2018. Stereo Magnification: Learning View Synthesis using Multiplane Images. ACM Trans. Graph. 37, 4 (2018).
Zollhöfer et al. (2014) Michael Zollhöfer, Matthias Nießner, Shahram Izadi, Christoph Rehmann, Christopher Zach, Matthew Fisher, Chenglei Wu, Andrew Fitzgibbon, Charles Loop, Christian Theobalt, and Marc Stamminger. 2014. Real-time Non-rigid Reconstruction Using an RGB-D Camera. ACM Trans. Graph. 33, 4 (2014).


text:
Anna Seaton, född den 12 februari 1964 i Topeka, Kansas, är en amerikansk roddare.

Hon tog OS-brons i tvåa utan styrman i samband med de olympiska roddtävlingarna 1992 i Barcelona.

Referenser

Källor

Externa länkar
Sports-reference.com

Amerikanska roddare
Amerikanska olympiska bronsmedaljörer
Olympiska bronsmedaljörer 1992
Tävlande vid olympiska sommarspelen 1988 från USA
Tävlande i rodd vid olympiska sommarspelen 1988
Tävlande vid olympiska sommarspelen 1992 från USA
Tävlande i rodd vid olympiska sommarspelen 1992
Personer från Topeka
Födda 1964
Levande personer
Kvinnor

redpajama_set_name: RedPajamaWikipedia


text:
Q: how to add validation in gridview textbox? <asp:TemplateField HeaderText="Quantity">
    <ItemTemplate>
        <asp:Label ID="lbl_quantity" runat="server" Text='<%# DataBinder.Eval(Container.DataItem,"quantity") %>'>
        </asp:Label>

    </ItemTemplate>

    <EditItemTemplate>
        <asp:TextBox runat="server" ID="txtquantity" Width="90px" onkeypress="return validatenumerics(event);" Text='<%# DataBinder.Eval(Container.DataItem,"quantity") %>'></asp:TextBox>
        <asp:RequiredFieldValidator ID="RFquantity" runat="server" ControlToValidate="txtquntity" Display="None" ErrorMessage="This field is required value" ValidationGroup="quantity">
        </asp:RequiredFieldValidator>
        <ajaxToolkit:ValidatorCalloutExtender ID="VCquantity" runat="Server" TargetControlID="RFquantity"/>

    </EditItemTemplate>
</asp:TemplateField>

I want to shows an javascript message , if text box is empty..i tried this code. for validation purpose required field validation is added.then "edit" button is not fired..please help me.
This is my grid with edit ,update ,cancel and cancel button
  protected void gvproducts_RowCancelingEdit(object sender, GridViewCancelEditEventArgs e)
{
    gvproducts.EditIndex = -1;
    grid();

}
protected void gvproducts_RowEditing(object sender, GridViewEditEventArgs e)
{
    gvproducts.EditIndex = e.NewEditIndex;
    grid();
}
protected void gvproducts_RowUpdating(object sender, GridViewUpdateEventArgs e)
{

    TextBox txtquantity = (TextBox)gvproducts.Rows[e.RowIndex].Cells[4].Controls[1];

    int st = Convert.ToInt32(gvproducts.DataKeys[e.RowIndex].Values[0].ToString());

    objsupplyPL.quantity = Convert.ToInt32(txtquantity.Text.ToString());

    objsupplyPL.sno = st;
    DataTable scmpurchase = new DataTable();
    scmpurchase = objsupplyBAL.updatepurchase(objsupplyPL);

    ScriptManager.RegisterClientScriptBlock(this.Page, this.GetType(), "UpdateDetails", "alert('Update Successful');", true);
    gvproducts.EditIndex = -1;
    grid();
}

A: What you could do is add a ValidationSummery with ShowMessageBox and a ValidationGroup just above the GridView
<asp:ValidationSummary ID="ValidationSummary1" runat="server" ShowMessageBox="true" ShowSummary="false" ValidationGroup="quantity" />

And change the TemplateField to this (make sure EnableClientScript=true) and make sure the EditButton has same validationgroup
<asp:TemplateField HeaderText="Quantity">
    <ItemTemplate>
        <asp:Label ID="lbl_quantity" runat="server" Text='<%# DataBinder.Eval(Container.DataItem,"quantity") %>' />
    </ItemTemplate>
    <EditItemTemplate>
        <asp:TextBox runat="server" ID="txtquantity" Width="90px" Text='<%# DataBinder.Eval(Container.DataItem,"quantity") %>' ValidationGroup="quantity"  />
        <asp:RequiredFieldValidator ID="RFquantity" runat="server" ControlToValidate="txtquantity" EnableClientScript="true"  Display="None" ErrorMessage="This field is required value" ValidationGroup="quantity" />
    </EditItemTemplate>
</asp:TemplateField>
<asp:CommandField ShowEditButton="True" ValidationGroup="quantity" />

A: just add a class to your TextBox and find the control using class finder with jquery. Then, you can check whether the TextBox.val() is empty or not using the javascript function.

redpajama_set_name: RedPajamaStackExchange


text:
Fabio Lacolla: "We have to think about a new mental health model"
When reviewing the sensations that were generated, the specialist remarked: "At first it was like a honeymoon, when the first restrictions began. But the house is to rest, not to be. Life is outside. Now that paradigm began to combine, because the private sphere began to be made public, because work meetings were done by zooming in a room ".
"Something interesting is that the future was annihilated for us. And life is for the future. We could not think about what is coming, so when the future disappears, what appears is the past. That is why we began to review photos, to connect with friends of the past. Because it was the only thing that gave us identity. Not who I am, but who I was. It was a reconstruction of identity in the midst of a pandemic. "
panel 2.jpg
In a one-on-one with Guillermo Capuya, Fabio Lacolla analyzed the psychological impact of the pandemic.
Mariano fuchila
Lacolla also analyzed how the confinement affected couples and families. "The pandemic was a discovery, because everything was discovered. Families that were strengthened became stronger. But the couples who had certain conflicts came to light. No more people were separated by the pandemic, it was because the bond was not strengthened. We discovered that we had other parents. Or the children, who had the father inside all day. There began to be a new form of coexistence ", he pointed out, adding:"When talking about the new normal, it seems to me that it is half dangerous: because normality also brought us the pandemic. Normality is situational, something that occurs in certain situations. What is normal?
The psychologist also maintained that depression grew, in these two years. Even through anxiety disorder. Depression is backwards, the depressive was hooked on a part of the past. And as the future disappeared, much of the past appeared. Depression, because it was not known what was going to happen. There were three restrictions: health, fear and economic. In that of fear, they were divided because some wanted to go out, but there are also those who many do not want to go out. They are afraid ", he remarked.
Finally, given the possible irruption of new waves and confinements, Lacolla pointed out that they may not have the same psychological impact as the first: "We are prepared. We have more tools to live in a pandemic. Some networks that began to be served. This pandemic without virtuality would have been a catastrophe. That brought people closer together. The word was not lost. We could talk to relatives. The communication, luckily, worked. And that was and is a great support ".
Another definition that Lacolla left behind is that, even when the pandemic passes, life will no longer be the same: "Habits have already changed. A two-year-old does not know a face without a mask. That generation, the newborns, are already growing up with that. After Cromañón, the musicians began to have criteria of the scenic risks on stage. This pandemic is going to teach us to take care of ourselves, "he said, and concluded:" One of the sequels left by the pandemic is irritability. There is more intolerance on the street. There is more selfishness too. In other cases, it was understood that nobody is saved alone, that we have to be more supportive. I think it changed now is the way to suffer. The pain that one felt, the anguish that one felt, was changing. We have to rewrite psychoanalysis books. There is going to be a paradigm shift. We are witnesses and we have a great responsibility to rethink mental health and health as a whole ".
PrevPolygon Price Analysis: MATIC Eyes $2 as Major Move Appears Imminent
NextIBM and Samsung work to make cell phone batteries last a week | Digital Trends Spanish
Apple Pay arrives in Argentina: how the virtual wallet works and what benefits it offers

redpajama_set_name: RedPajamaCommonCrawl


text:
Q: Dynamic policy and signature creation for browser uploads to Amazon S3 I would like to select a file in an HTML form and upload it directly to Amazon S3. I understand the principles of browser-based uploads to S3 using POST, but how can I create a policy and signature for the form if I haven't yet chosen a file to upload? Should I use AJAX to send the filename to my server and return an appropriate policy and signature based on the file extension?

A: You can use Ajax, Flash or any other method to upload the file to your server and then transfer it to the Amazon S3.


text:
Aus der Heimath, op. 347, är en polkamazurka av Johann Strauss den yngre. Den framfördes första gången den 2 juni 1871 i Volksgarten Wien.

Historia
Aus der Heimath var en av sex polkor som Johann Strauss arrangerade utifrån melodier tagna från hans operett Indigo und die 40 Räuber. Strauss var glad över att ha lämnat balsalarna för teaterscenen och överlät åt brodern Eduard att framföra de nio separata orkesterverk från Indigo för publiken. Polkan Aus der Heimath framfördes vid Eduards konsert i Volksgarten den 2 juni 1871. Titeln är hämtad från texten till operetten, närmare bestämt till Janios öppningssång i akt I ("Liebchen, ach, wie hab' ich dich gern, wär' ich mit dir doch in der schönen Heimat fern'"), och det är även "Andantino con moto"-delen av detta nummer som också har givit upphov till polkans huvudtema.

Om polkan
Speltiden är ca 3 minuter och 49 sekunder plus minus några sekunder beroende på dirigentens musikaliska tolkning.

Polkan var ett av flera verk där Strauss återanvände musik från operetten Indigo:

 Shawl-Polka, Polka-francaise, Opus 343
 Indigo-Quadrille, Opus 344
 Auf freiem Fusse, Polka francaise, Opus 345
 Tausend und Eine Nacht, vals, Opus 346
 Aus der Heimath, Polkamazurka, Opus 347
 Im Sturmschritt, Schnell-Polka, Opus 348
 Indigo-Marsch, Opus 349
 Lust'ger Rath, Polka-francaise, Opus 350
 Die Bajadere, Polka-schnell, Opus 351

Externa länkar
 Aus der Heimath i Naxos-utgåvan

Referenser

Noter

Polkor av Johann Strauss den yngre


text:
Q: title not displayed when using sigplanconf class I'm using sigplanconf document class. Title is not displayed even when \maketitle is specified
\documentclass{sigplanconf}
\begin{document}
\title{foo}
\authorinfo{bar}
\maketitle

hello world
\end{document}

A: Using the class provided at sigplan and reducing the template they provide, results in the requirement of defining the \doi{} field to get the file compiled showing the title.
The minimal code is then
\documentclass{sigplanconf}
\begin{document}
\doi{nnnnnnn.nnnnnnn}
\title{Title Text}
\authorinfo{Name1}
           {Affiliation1}
           {Email1}
\maketitle
\end{document}


If you're interested in learning more about Governor Romney's education proposals, take a look at this thoughtful analysis by Michael Petrilli, the director of the Fordham Institute.
Today, there's not a single Republican in the House of Representatives, in the Senate, or running for president willing to defend federal accountability mandates. The GOP conversation has shifted to transparency, in line with what we've called Reform Realism. What a difference a decade makes.
Every time I see the term "accountability" when discussing education, it only seems to refer to one part of tripod necessary for a students education. No accountability for parents and students in this proposal as usual.
This proposal is really nothing new and really does nothing to solve the problems of education. As long as traditional public schools are forced to take all students and private and charter schools are not, private and charter schools should perform at a much higher level. As long as private and charter schools are allowed to create a learning environment where discipline is allowed, expectations are put in place and must be met, then private and charter schools should outperform traditional public schools.
Lawmakers have done everything they can to handicap traditional public education, and then have complained about their lack of performance. If we are going to create a system of competition then lets allow everyone to compete, not just the private and charter schools. If we are going to allow the money to follow students, then lets allow traditional public schools to create rules that when students choose not follow them they are not allowed to attend school at the traditional public school. I teach in a traditional public high school, I have 15-20 kids that if they were placed in a charter school they would either change their ways or would be kicked out of the school. We don't have that ability in traditional schools, often times a judge will force the neighborhood school to take back students. That doesn't happen in private or charter schools.
Please don't tell me it doesn't happen because it does. In our school of over 1300 students we will have at least 50 students a year who are there under court order, and a 100 students a year who have been kicked out of private or charter schools for violations that wouldn't even get them suspended in a traditional public high school.
Competition implies a level playing field. In the world of education the playing field is not level, traditional public schools play at a disadvantage. Many will say they are not at a disadvantage because they have so much more money. If you look at where the largest sums of money is spent, it is spent on students who would not typically even get into a private or charter school. It is spent on the most incorrigible students, and the most severe special needs students. If you looked at the money a school district spends on the "good" kids who come to school work hard and try to learn, the actual money spent in their behalf is very low.
There is another article running in the Deseret News today asking if we could learn anything about what other countries do that could be applied in America. That would be better reading for Candidate Romney than his own plan if he truly desires to create a change in the American education system. If he simply wants to play to his base, then his plan is spot on.
The bigger question for Romney or Obama is what will be DONE to help struggling schools. Local control is the conservative mantra but all too often in inner-city schools it means dilapidated facilities with struggling young teachers looking for the quickest way out. It's great to have "transparency" and "report cards" but threatening schools with sanctions and withholding money seems to be a backward approach on many levels. These schools often need better and safer facilities, technology and better trained teachers and unfortunately with local and state shoe-string budgets, the only hope for these schools is some federal commitment of dollars. Vouchers and charters may seem like the answer but private schools will always remain exclusionary for all too many and charters, with their lotteries, will always have losers and many charters don't perform that well.
There are good things going on in many of our schools across the nation, however, there are many schools, usually in the tougher socio-economic locales, that are struggling. Also many of our higher end students are not getting challenged enough and perhaps we are falling behind in areas of math and science especially.
I think the conservative plan of vouchers, charters, and putting the hammer down on these schools with report cards and threats didn't work with NCLB and it won't work here either.
There is no federal or state law that I am aware of that requires public schools to allow students who refuse to participate in the programs or maintain standards of conduct to remain in them. District policies have been crafted to allow students to stay in schools even after they have demonstrated they are not going to attend class, do homework, or maintain behavior standards. The requirements of FAPE (a free and appropriate public education) have not, to my knowledge, been interpreted to mean that districts must allow students to remain at a particular school.
Districts could raise the standards for their schools quickly by making it a privilege to attend the best schools – a privilege maintained through effort, participation and attendance. If a student misses the mark, they can attend a different school in the district, which would likely have a different focus and less positive outcomes (because more time needs to be spent on remediating the students who have failed prior classes, or focusing on truancy etc). Parents could also lose their free transportation if the school the student qualifies for is not their boundary school. Being required to drive their student to school would increase the parents' participation. While this may be unpopular, I don't believe it is a violation of FAPE and would increase parent and student participation in schools.
We would be doing these families a big favor if there were some real consequences for failure to participate in school. It should be applied each quarter (not just at year's end) so students have LOTS of opportunities for "do-overs" which, I believe, is the key for many students during adolescence. If they fail a quarter, or don't meet attendance requirements, they transfer to a different school. They could earn their original placement back, but it would be difficult. As a parent I would have appreciated this, and I believe many parents would appreciate the support of the schools in not allowing their students to fail or misbehave without real, meaningful consequences.
So the fact that public schools are "required" to take all students is true, but HOW they determine they will serve students is completely up to them. At this point, they choose to ignore the failure to participate and bad behaviors and force teachers to try to deal with it, which greatly and negatively impacts the experience of all students.
If it is not a level playing field for public schools, it is due to the districts' lack of expertise in organizing their schools and implementing policies that effectively train students with real consequences. This is completely under the districts' control. Therefore I believe it is not a valid argument against holding public schools accountable. Just because they CHOOSE to disadvantage themselves through ineffective policies that set up a system that doesn't work very well, does not excuse them from being accountable. If the playing field isn't level, it is under their control to make it so.


Documentation that the performance of the alien or group is culturally unique.
Spouses and children of the P3 visa holder may obtain a P4 visa to enter and remain in the US. The P4 visa holder may attend school but if he or she wishes to work, he or she must change his or her status to obtain a work visa.
For Whom Is an P3 Visa Appropriate?
Artists and entertainers entering the US to perform in or coach a culturally unique program and their support personnel.
A P3 classification applies to an alien artist or entertainer who is coming temporarily to the United States, either individually or as part of a group, or as an integral part of the performance of the group, to perform, teach, or coach under a commercial or noncommercial program that is culturally unique.


text:
This Couple's Minimalist Brownstone Exudes the Control We All Crave
Maybe a sense of calm can be found in curation
By Kelly Dawson
Photography by Charlie Schuck
Minimalism requires meticulousness. It's a mindset photographer Charlie Schuck and prop stylist Natasha Felker have mastered in their Brooklyn brownstone, where their curated possessions meet a clean palette awash in natural light. Every room, including the backyard garden, is a reflection of their values—and that level of control feels worthwhile right about now.
"Photography and the way we view light has a massive impact on our design philosophy and how it relates to photography," Charlie says. "We purposefully looked for a brownstone that received light from three directions."
"We have taken some flak from friends on the sparseness of our walls," Charlie says. "Just because there is open space does not mean that it needs to be immediately filled with art. We feel that to live well in our home is to have a dialogue with space, light, and objects."
"Natural light in itself is a form of art and we view our place as a way to study it," adds Charlie. "In the winter, the light comes through the windows differently than it does during the summer. When you study light, you realize that there are many ways for light to get from point A to point B and each way has a subtly different outcome."
The confidence this couple has in personalizing their place started long before they moved in. They were looking for three bedrooms and a yard spread across multiple stories with windows on three sides. It had to be a rental, but it also needed to welcome a few projects. Charlie and Natasha weren't as concerned about its location as they were about the place itself—in fact, Charlie jokes that they liked to stay home long before the pandemic. Their search uncovered this 1912 property in Bay Ridge last fall, and they knew it was the one by how horrible it looked in the photos.
"That is always a good sign, because when something is nice and well photographed it creates competition," Charlie says, with an aside that speaks yet again to his penchant for details. "Our strategy is to prepare all of our paperwork and be ready to go when we find what we're looking for."
The decor is a perfect example of honoring the details and history, while also being complementary. Here, the classic Womb Chair blends seamlessly into the dark wood accents.
The property's layout has three common rooms along a hallway leading to a kitchen, and three bedrooms upstairs. Charlie and Natasha relished a century's worth of architectural flourishes as much as they appreciated an empty garden with no character at all. The way they saw it, this was their chance to make tarnished-yet-promising surroundings feel like home. They wanted to honor its history and speak to their personal tastes—all while sticking to a clear code of minimalist design. "Our philosophy is to create harmony and a sense of continuity by using like-minded furniture, textures, and colors," Charlie says. "Our personal space needs to nurture and inspire us."
Charlie and Natasha use one of the rooms as their "dressing room" for their clothes, and the third bedroom as their office. The pendant in their dressing room mimics the one in their dining room.
They replaced six light fixtures in the kitchen and dining room to make both spaces feel airier. "The main fixture over the dining table was swapped out for a simple circular rice paper lamp we obtained from the Finnish Design Shop," he says. "The dining room has a brick wall and dark wood floors, so we used lighter wood items and natural objects as a contrast."
Charlie Schuck
The couple brought in furniture they already owned, learning a while back that timeless pieces beat trendy ones when budgets and aesthetics are on the line. They changed light fixtures and hardware in just about all the rooms to make them feel more modern, and removed six doors so that it all came across as more spacious. They passed over anything that didn't conform to their rules, and found missing pieces thoughtfully. Figuring out "how to fill holes" was bound to happen, since the brownstone was 900 square feet bigger than their last apartment. "If there's something we fall in love with, we first look to see if it can be found used, at a discount, or for a trade," Charlie says. "If not, then we buy or commission the piece to be constructed."
"We never had an outdoor space, and a big part of the move was to be able to garden," Charlie says.
"There's a giant learning curve with greenery, and we are still taking notes on what's working and what's not," Charlie says. "It helped that we moved in the late fall last year. This allowed us to focus on the interior through the winter months and then work on the garden in the spring."
The garden may not look as composed as the rest of the home, but it still showcases the couple's foresight. "It was built around three main components, and the idea was to have the feeling of an overgrown formal garden," Charlie says. "We first set stone trails around a hedge square and then added boxes for vegetables. In the middle of the square is a lemon tree surrounded by a bunch of different plants, and in the far corner is an outdoor daybed for naps."
After so much careful planning, that corner seems only fair.
The office of a prop stylist is beautiful sight.
⚒ Do It Yourself
Swap out unsightly light fixtures and hardware "It's relatively easy to swap out light fixtures and hardware—especially doorknobs," Charlie says. "Or you can hire someone via TaskRabbit or Craigslist to do this for you."
You don't have to hate your bathroom fixtures "This is also surprisingly easy to swap out," he adds. "Make sure to save the old one and its parts in case you need to put it back when you move."
Cover surfaces with shoji paper "This takes a bit of patience, and make sure to order a bit more paper than you think you need," Charlie says. "Shoji paper can effectively cover strangely colored and patterned tile, walls, or cabinets without ruining the surfaces."
Spring for custom light shades "You can take an old lamp and make a new custom light shade in any color pattern or size you can dream up for only a few hundred dollars or less," he says.
Thankfully, most of the tiles in the bathrooms didn't need to be covered. In another bathroom, the tiles remained but Charlie and Natasha swapped out the hardware and lighting.
Charlie and Natasha covered mismatched tile in the master bathroom with Japanese shoji paper that they cut and adhered to the wall. "This paper is slightly transparent and easy to remove when needed," he says.
🛍 Shop It Out
All products featured on Architectural Digest are independently selected by our editors. However, when you buy something through our retail links, we may earn an affiliate commission.
Akari 3A light by Isamu Noguchi, $400, shop.noguchi.org
Womb Chair by Eero Saarinen, $4,620, dwr.com
Shoji Japanese paper roll, $24, eshoji.com
Palo floor lamp by Michael Raasch, $1,501, stillfried.com
Matin table lamp by HAY, $149, us.hay.com
"The bedroom is very minimal," Charlie says. "Lighting, color, and material are important to the design, and so is repetition. The Noguchi floor lamp ties in with our dining room lamp, and a black Skagerak stool repeats in a different color in the office. The dressing room is next door and has some repeating elements as well."
"Every time you move and work on a new project, it forces you to confront how you live," Charlie says. "It can give you clarity in many facets of life beyond simple design preferences."
Exploreinterior designClever Home Tourdecorationgarden
Subscribe to AD
Renew AD Print Subscription


text:
Helping less fortunate
Scroll down for all performers & their bios ↓
THE SKELETON CLUB
From its inception, Adelaide blues/folk 6-piece The Skeleton Club has been the makeshift home for a coming-together of storytellers.
Brothers and dual-frontmen Tom and Nick Vasic openly spill brutally cathartic anecdotes, shaped by the highs and lows of their contrasting vocal ranges.
Rhythm guitarist Toby Brandenburg and bassist Sammy Berry lead the sea of back-up harmonies, alongside lead guitarist Mel Pierides and together the band deliver a wall of sound that takes influence from genres such as country blues, folk, latin, gospel and soul.
The Skeleton Club have already shared the stage with an array of high class acts including Paul Kelly, The Black Sorrows, Tash Sultana, Marlon Williams, Jack Carty, Closure in Moscow, D.D Dumbo, Tkay Maidza, The Beards, Will Pugh(US) and Jonny Craig (US).
The band's eclectic troubadour-pop could find a comfortable home from Nashville to Cuba to Spain and almost anywhere in between but home is where the heart is, so TSC are romping and rocking their way across Australia, to a stage near you.
http://www.theskeletonclub.com.au/
https://www.facebook.com/theskeletonclub/
https://twitter.com/SkeletonClub
https://www.instagram.com/theskeletonclub/
For general enquiries please feel free to use the form on this page. For more information please go to our
Contact Page →
All rights reserved © Blenheimfest
Website Designed & Developed by Honeycomb Design
Send it off!