7 Overview
\[ \def\oi{{\omega_i}} \def\os{{\omega_s}} \def\Oi{{\Omega_i}} \def\Os{{\Omega_s}} \def\d{{\text{d}}} \def\D{{\Delta}} \def\do{{\d\omega}} \def\Do{{\Delta\omega}} \def\doi{{\d\omega_i}} \def\dos{{\d\omega_s}} \def\Doi{{\D\omega_i}} \def\Dos{{\D\omega_s}} \def\H{{\mathbf{H}}} \newcommand{\cM}{\mathcal{M}} \newcommand{\cL}{\mathcal{L}} \]
This chapter first lays the land for physically modeling light-matter interactions in rendering, discussing the different levels at which the modeling can occur (Section 7.1) and building intuitions using a toy example (Section 7.2). We will conclude the chapter by introducing an extremely high-level model that abstracts away almost all the underlying physics and models an object as its apparent reflection and transmission spectra (Section 7.3). This simple modeling is very commonly used in practice and serves as a curious teaser for the remaining rendering chapters: how and when can this simple model be a good approximation of the sophisticated light-matter interactions?
7.1 Light-Matter Interactions
By and large, we will treat light as a collection of photons, each traveling in a particular direction and carrying a specific amount of energy. When a beam of photons hits a material surface, some of the photons will be scattered directly back to your eyes, and others will penetrate into the material. These surface phenomena are governed by surface scattering. We use the word “scattering” here to generally refer to lights coming back from the surface. Depending on the material, some of the scattered photons are along the perfect mirror-reflection directions, and others might be more diffuse. You might sometimes see the word “reflection” used. Reflection is sometimes used in the same way as scattering, which will be our use, but other times is reserved for the perfect, mirror-like reflection. Usually what the word means is self-evident given the context, but we will err on the side of verbosity when we want to mean a specific form of reflection.
Photons that penetrate the surface will further interact with particles in the material, which absorb, scatter, or might even emit photons. This is called subsurface scattering (SSS) in computer graphics. Even though we use the term “scattering”, you should know that the actual SSS processes involve not only photon scattering but also absorption and emission. We will, however, generally ignore emission in our discussion unless otherwise noted, but just note that emission does happen and is correlated with absorption, since emission is the result of absorbed photons having (e.g., chemical) reactions with the particles.
It turns out that the principles that govern SSS are exactly the same as those that govern the interactions between photons and particles in the so-called participating media, such as clouds, fogs, and smokes. In computer graphics, light-matter interactions in participating media are called volume scattering1. SSS is, instead, more commonly used to refer to light-matter interactions in solids (which have a clear definition of “surface”) where subsurface-scattered photons contribute to their observed colors, such as skin, jade, and wax.
To model SSS/volume scattering, we no longer consider the light-matter interaction as photons bouncing off of the surface; instead, we break a material down into small particles and model how photons interact with individual particles. Very importantly, the difference in the modeling methodology does not imply that there somehow is a fundamental difference between surface scattering and volume scattering.
Ultimately, both are caused by the light, an oscillating electromagnetic field, exciting discrete electric charges. The differences lie in how the charges are arranged in space and in relation to one another. In the classical regime, the laws that govern how photons interact with the charges are described by the electromagnetic theory. In fact, using the electromagnetic theory, we can show that surface reflection/refraction is nothing more than the coherent scattering of incident light waves by the surface particles. In the quantum regime, light-matter interactions are governed by the quantum electrodynamics (QED)2.
Since there is no fundamental differences in the underlying physics, the only meaningful distinction is one between different phenomenological approximations, or “models”, of the same underlying physics. We totally could invoke the electromagnetic theories or QED, and if we did, we would have one single unified model that explains both surface scattering (reflection and refraction) and volume scattering. Doing so, however, is not only unnecessary (because many, not all, real-world material color phenomena could be modeled without them) and too computationally expensive, but also, perhaps more importantly, blinds us from the relatively simple intuitions in each scenario. Instead, each phenomenological model is based on a set of high-level guiding principles, which are approximations of the underlying physical process but are sufficient to quantitatively describe light-matter interactions in each scenario.
Johnsen (2012) is a great reference, which has some equations but generally focuses on building intuitions and mostly uses the electromagnetic language rather than the quantum language. If you want to get to the nuts and bolts of the mathematical modeling, Bohren and Clothiaux (2006) is a phenomenal text whose models are also built in the electromagnetic land. Feynman (1985) has an accessible and breathtaking introduction to QED that I highly recommend. Dorsey, Rushmeier, and Sillion (2010) is a classic text on material appearance modeling in graphics that covers a range of topics, including modeling, measurements, and various implementation issues in practice. Johnston-Feller (2001) is specifically concerned with paintings; it has many interesting discussions of pigments and pigment mixtures and has many real-world data and insights that are rarely found elsewhere.
7.2 Building Intuition
We will use Figure 7.1 as a running example to discuss the life of photons when interacting with materials/media and develop some key intuitions along the way.
Photons come from above the surface of the first material. At the Air-Material1 interface, photons are either reflected directly back or penetrate into the material through refraction. The exact form of reflection is governed by surface scattering. Here we have assumed a perfectly specular reflection and refraction that obey the Snell’s law (Section 11.1), where, for instance, the direction of reflection is mirror-symmetric to the incident direction. As we will see in Chapter 11, general material surfaces are more complicated than this.
Once inside the material, a photon roams about until it meets a particle. The interactions between photons and particles are governed by the subsurface scattering (SSS) or volume scattering processes. Subsurface scattering is so termed to distinguish itself from surface scattering, but what is beneath the surface is nothing more than a volume of particles. In fact, what is above the surface is also a volume of particles. Looking at Figure 7.1, the air, Material1, and Material2 can all be thought of as participating media. We usually model the air as a vacuum so photons traverse in straight lines undisturbedly, but if we were to be exact, we would want to model the particles in the air, which becomes a participating medium. So “above-surface scattering” is as different from surface scattering as is subsurface scattering.
When a photon meets a particle inside the material, the particle might absorb the photon or scatter it away. If absorbed, the photon is “dead” and can be removed from the discussion. If scattered, the photon might appear to change its direction and continue to travel on a straight line until it meets another particle, so in principle a photon can be scattered multiple times.
A photon has a certain probability of being absorbed when it meets a particle, so the longer it travels, the more likely it will be absorbed. If a photon survives the absorption of all the particles in the media, it would re-emerge from Material1 back to the air or they might emerge to the air from the bottom of Material2. Let’s examine cases where a photon escapes the media.
After multiple scattering events, some of the initial photons that enter Material1 from the air will reach the Material1-air boundary again, but this time from the material side. At that point, the photons necessarily go through another round of reflection-refraction governed by the surface scattering processes. The refracted photons will re-emerge from Material1.
This is called back-scattering, because these photons are scattered back to where they come. As a consequence, when we observe the material from the same side of the illumination, the lights that enter our eye come from two sources: the initial surface scattering and the back-scattering.
Some photons might leave Material1 from the other side and enter Material2, in which photons go through the same volume scattering processes, where some are absorbed, some can be turned back to Material1, and others, critically, can hit the Air-Material2 interface. Just like what happens at the Air-Material1 interface, some of the photons will eventually emerge from Material2. When you observe the material from the opposite side of the illumination, it is these transmitted photons that dictate the color of the material.
Sometimes people will also say, “subsurface scattering is caused by photons exiting at a point different from the incident point”. It points to the fact that a photon can re-emerge anywhere from the material after SSS, whereas surface scattering is modeled to be taking place only at the incident point. But this is just a useful macroscopic abstraction or, rather, modeling strategy. In reality, surface scattering results from, as discussed above, the superposition of scattering of the incident wave by all particles in the surface region, which is intrinsically non-local since wave interactions extend beyond a single point.
7.2.1 Material Types
We often hear materials being described as opaque, translucent, and transparent. We can now more scientifically approach these terms given the intuitions we have built so far.
Transparent Materials
Transparent materials either scatter light predominantly in forward directions or they scatter very little light (other than surface scattering). As a result, most photons traveling through the material are either absorbed or go through without changing much of their the directions. So if you hold a transparent material against a light source, you can clearly see through the material and see the light on the other side. This does not mean transparent materials always have the same color as the light source — absorption could be wavelength-selective. An example is aqueous/dye solutions where dye molecules are very small (\(\text{nm}\) range) and, thus, scatter little light so they look transparent, but depending on the absorption spectrum (which depends on how the dye molecules interact with molecules in the solvent), most dye solutions are not colorless.
Opaque Materials
In many materials, photons arriving at the material surface are either reflected right away at the surface or, for those that do penetrate into the materials, are all absorbed by the subsurface particles. Examples include conductors like metals or sufficiently thick dielectrics like bricks.
These materials are opaque in two senses. First, their transmittance is practically 0: because of strong absorption, no photon re-emerges at the other side of the material. If you hold, say, a brick (dielectric) against a light bulb, the brick would completely block the light. Second, their surface color is independent of the substrate or the material beneath them, so they completely hide the color of the substrate3. Using the example in Figure 7.1, if Material1 is sufficiently thick, few photons would reach Material2 and for those that do reach, they have little change of re-emerging from Material1. Painters know that if they want to cover a layer in their painting, they will need to apply a very thick layer of paint on top.
Translucent Materials
Translucent materials such as jade, wax, and skin are neither opaque nor transparent. If you hold a wax against a light bulb, the wax will not completely block the light, so you will see some light, but you will not be able to see clearly the other side through the wax, since photons from the light bulb are very much volume-scattered after passing through the wax. Clearly, modeling SSS is critical for accurately estimating the color of translucent materials. In fact, in graphics literature we sometimes see things like “modeling translucent material must consider subsurface scattering”.
It is not true that SSS is important only in modeling translucency. Modeling SSS can be important for opaque materials. Consider the wax case: what if we make the wax very thick? The thick wax will eventually become opaque in that it will completely hide the material behind it. But that does not mean volume scattering does not matter here; the back-scattered photons do contribute to the apparent color of a thick wax.
Oil Painting Example
To put things together, consider a painting. One way paintings are characterized is by how they were painted, and we might see things like “oil on canvas”. Oil means the paint is oil paint, where paint pigments are dispersed into (usually linseed) oil, which is usually called the binder or the vehicle. Canvas is the substrate, which is nothing more than another material that is right beneath the painting.
The oil itself is somewhat transparent, especially when you just apply a thin layer on the canvas. But with the paint pigments, the entire oil paint becomes a translucent material. When photons leave the oil paints, they immediately interact with the canvas. If the paint layer is thick enough, virtually no photon can ever reach the canvas. But if the paint is relatively thin, the property of the substrate will contribute to the overall color of the paint. For instance, if the canvas is white-ish, a good percentage of the photons will be reflected back. The same paint would look much darker if the canvas is black, which absorbs a lot of photons.
7.2.2 Equilibrium
We can view the light-material interaction as a dynamical system under an equilibrium. To appreciate this, consider again Figure 7.1. Some photons entering Material1 are back-scattered and hit the Air-Material1 interface and some of those photons will re-enter Material1 through internal reflection. Those photons will then go through multiple scattering, and as a result some will be back-scattered again and hit the Air-Material1 interface. The cycle goes on. The secondary back-scattering is weaker in power than the first back-scattering, and the third-order back-scattering is even weaker, and so on. So eventually you can imagine that the total number of photons back-scattered at the surface will reach a constant.
In fact, this sort of dynamics takes place everywhere inside the material along every direction. If you pick a point \(p\) in the material (or at the surface) and a direction \(\omega\) starting at the point, the radiance at (\(p, \omega\)) is a constant under equilibrium. In other words, the spatial radiance distribution is not changing over time.
The equilibrium is reached almost instantaneously, since light propagates incredibly fast. So the equilibrium discussion is probably of no practical impact in modeling or actual measurement, but it is still important to keep this in mind. Our discussion will assume equilibrium. For instance, later when we model volume scattering, we will set up the differential equations that describe the energy transfer assuming equilibrium.
7.3 Observed Reflection and Transmission
Regardless of the details of surface scattering and volume scattering, a material appears to have some color because some photons leaving the material enter our eye. If we observe the material from the same side of the light source, it is the lights reflected from the material that matter. If we observe the material from the other side of the light source, it is the light transmitted through the material that matter. At the highest level of abstraction, we can model the material color in the real world by modeling the observed reflection and transmission apparent to an outside observer: how much of the incident power is reflected/transmitted back to the eye?
We can quantify the observed reflection and transmission using the spectral reflectance function \(r(\lambda)\) and the spectral transmittance function \(t(\lambda)\), respectively. These two functions spare us the details of how lights interact with a material but describe, at each wavelength \(\lambda\), the percentage of optical power that is reflected back to the eye or transmitted through the material and enters the eye, respectively.
Figure 7.2 (a) illustrates this modeling at work using the famous The Astronomer by Johannes Vermeer. We will proceed with our discussion using reflectance, but the case idea can be easily extended to transmittance. Vermeer paints an astronomer looking at a globe. Given the illumination coming from the window \(\Phi(\lambda)\) and the spectral reflectance of the point on the globe under gaze \(r(\lambda)\), the light reflected toward the eye is then \(\Phi(\lambda)r(\lambda)\). We can then calculate the color of these lights using the cone fundamentals or some set of CMFs, the same way as if the lights were directly emitted from the globe.
As another example, Figure 7.2 (b) is a photo of Acadia Redfish I took when visiting the Ripley’s Aquarium of Canada. The fish ordinarily looks red-ish under a white-ish light, which suggests that its spectral reflectance \(r(\lambda)\) peaks at longer wavelengths: it scatters more long-wavelength, i.e., red-ish, lights than short-wavelength lights. But the fish appears colorless in the aquarium, which simulates the lighting environment in the deep sea where lights \(\Phi(\lambda)\) are predominately blue/violet4. As a result, the scattered lights have a rather uniform spectral power distribution, resulting in a gray-ish appearance.
Figure 7.2 makes an important simplification: the reflectance of a point \(p\) on the material is simplified to only a single spectrum. In reality, the reflectance of a point \(p\) depends on both \(\oi\), the direction of the light incident on \(p\), and \(\os\), the outgoing direction (leaving \(p\)) through which one observes the material. In certain materials where SSS contributes to the material appearance (e.g., translucent materials like jade), the reflectance can also depend on light incident on other points of the material surface. So when we use a single reflectance spectrum to model material colors, what we have implicitly assumed is that the reflectance spectrum has been calculated in such a way that when you multiply it with the incident illumination, you get the scattered light power that is actually observed.
How such a reflectance spectrum can be obtained in measurement (to the extent that it is a useful high-level abstraction) will be discussed in Section 11.3. The reflectance is a “quick-and-dirty” abstraction that we often use to give a rough estimation/explanation of a material’s color, but it is so high-level that it hides lots of the low-level details: what exactly are the light-matter interactions that cause the surface and subsurface scattering behaviors that eventually give rise to the apparent reflectance and transmittance spectra? The remaining chapters in this part essentially answer this question.
and again, even though we use the term “scattering”, absorption and emission are usually involved in the most general cases↩︎
The electromagnetic theory does not explain everything in light-matter interactions. Famously, they do not explain how the interference pattern in the double-slit experiment still arises even if the photons are delivered sequentially.↩︎
Technically speaking, having a zero transmittance requires the material to have a stronger absorption than hiding the substrate, because in the latter case photons have to make a round trip, so they have more opportunities to be absorbed.↩︎
which results from a combination of water selectively absorbing medium-to-long wavelengths of light and increasing scattering of short wavelengths in the Rayleigh regime (Section 12.2.4).↩︎