ARCore is back at Google I/O on May 10! Register now.

Realistically light virtual objects in a scene

Stay organized with collections Save and categorize content based on your preferences.

Learn how to use Lighting Estimation in your own apps.


Make sure that you understand fundamental AR concepts and how to configure an ARCore session before proceeding.

Configure the API once per session with the appropriate mode

Configure Lighting Estimation once per session for the mode you want to use.

Environmental HDR

// Configure the session's lighting estimation mode for
ArConfig* config = NULL;
ArConfig_create(session, &config);
ArSession_getConfig(session, config);
ArConfig_setLightEstimationMode(session, config,
ArSession_configure(session, config);

Ambient Intensity

// Configure the session's lighting estimation mode for
ArConfig* config = NULL;
ArConfig_create(session, &config);
ArSession_getConfig(session, config);
ArConfig_setLightEstimationMode(session, config,
ArSession_configure(session, config);


// Disable the session's lighting estimation mode.
ArConfig* config = NULL;
ArConfig_create(session, &config);
ArSession_getConfig(session, config);
ArConfig_setLightEstimationMode(session, config,
ArSession_configure(session, config);

Use values obtained from Lighting Estimation

To use values obtained from Lighting Estimation, get the light estimate for each frame.

// Get the current frame.
ArFrame* ar_frame = NULL;
if (ArSession_update(session, ar_frame) != AR_SUCCESS) {
  LOGE("ArSession_update error");

// Get the light estimate for the current frame.
ArLightEstimate* ar_light_estimate = NULL;
ArLightEstimate_create(session, &ar_light_estimate);
ArFrame_getLightEstimate(session, ar_frame, ar_light_estimate);

ArLightEstimateState ar_light_estimate_state;
ArLightEstimate_getState(session, ar_light_estimate,

// Check that the light estimate is valid before proceeding.
if (ar_light_estimate_state != AR_LIGHT_ESTIMATE_STATE_VALID) {
  LOGE("ArLightEstimateState is not valid.");

Then get the environmental HDR lighting components for the current configuration:

Environmental HDR

// Get intensity and direction of the main directional light from the current
// light estimate.
float direction[3];
    session, ar_light_estimate, direction);

float intensity[3];
    session, ar_light_estimate, intensity);

// Get ambient lighting as spherical harmonics coefficients.
float ambient_spherical_harmonics[27];
    session, ar_light_estimate, ambient_spherical_harmonics);

// Get HDR environmental lighting as a cubemap in linear color space.
ArImageCubemap cubemap_textures;
ArLightEstimate_acquireEnvironmentalHdrCubemap(session, ar_light_estimate,
int width = -1;
int height = -1;
int32_t format = -1;
for (int i = 0; i < 6; ++i) {
  ArImage* image_ptr = cubemap_textures[i];
  // We can access the cubemap texture data through ArImage APIs.
  ArImage_getWidth(session, image_ptr, &width);
  ArImage_getHeight(session, image_ptr, &height);
  ArImage_getFormat(session, image_ptr, &format);
  // Acquired image must be released with ArImage_release once it is no
  // longer needed.

Ambient Intensity

// Get the pixel intensity of AR_LIGHT_ESTIMATION_MODE_AMBIENT_INTENSITY mode.
float pixel_intensity;
ArLightEstimate_getPixelIntensity(session, ar_light_estimate,

// Get the pixel color correction of
float color_correction[4];
ArLightEstimate_getColorCorrection(session, ar_light_estimate,

Ensuring energy conservation with Environmental HDR APIs

Energy conservation is the principle that light reflected from a surface will never be more intense than it was before it hit the surface. This rule is enforced in physically-based rendering, but is usually omitted from legacy rendering pipelines used in video games and mobile apps.

If you're using a physically-based rendering pipeline with Environmental HDR light estimation, simply ensure physically-based materials are used in your virtual objects.

If you aren't using a physically-based pipeline, however, you have a couple of options:

  • The most ideal solution for this is to migrate to a physically-based pipeline.

  • If that isn't possible, however, a good workaround is to multiply the albedo value from a non-physically-based material by an energy conservation factor. This can make sure at least the BRDF shading model can be converted into physically-based. Each BRDF has a different factor -- for example, for a diffuse reflection it is 1/Pi.