Frame

Per-frame state.

Summary

Enumerations
`ArCoordinates2dType{ AR_COORDINATES_2D_TEXTURE_TEXELS = 0, AR_COORDINATES_2D_TEXTURE_NORMALIZED = 1, AR_COORDINATES_2D_IMAGE_PIXELS = 2, AR_COORDINATES_2D_IMAGE_NORMALIZED = 3, AR_COORDINATES_2D_OPENGL_NORMALIZED_DEVICE_COORDINATES = 6, AR_COORDINATES_2D_VIEW = 7, AR_COORDINATES_2D_VIEW_NORMALIZED = 8 }`	enum 2d coordinate systems supported by ARCore.

Typedefs
`ArFrame`	typedef `struct ArFrame_` The world state resulting from an update (value type).

Functions
`ArFrame_acquireCamera(const ArSession session, const ArFrame frame, ArCamera **out_camera)`	`void` Returns the camera object for the session.
`ArFrame_acquireCameraImage(ArSession session, ArFrame frame, ArImage **out_image)`	`ArStatus` Returns the CPU image for the current frame.
`ArFrame_acquireImageMetadata(const ArSession session, const ArFrame frame, ArImageMetadata **out_metadata)`	`ArStatus` Gets the camera metadata for the current camera image.
`ArFrame_acquirePointCloud(const ArSession session, const ArFrame frame, ArPointCloud **out_point_cloud)`	`ArStatus` Acquires the current set of estimated 3d points attached to real-world geometry.
`ArFrame_create(const ArSession session, ArFrame *out_frame)`	`void` Allocates a new ArFrame object, storing the pointer into `*out_frame`.
`ArFrame_destroy(ArFrame *frame)`	`void` Releases an ArFrame and any references it holds.
`ArFrame_getAndroidSensorPose(const ArSession session, const ArFrame frame, ArPose *out_pose)`	`void` Sets `out_pose` to the pose of the Android Sensor Coordinate System in the world coordinate space for this frame.
`ArFrame_getDisplayGeometryChanged(const ArSession session, const ArFrame frame, int32_t *out_geometry_changed)`	`void` Checks if the display rotation or viewport geometry changed since the previous call to ArSession_update().
`ArFrame_getLightEstimate(const ArSession session, const ArFrame frame, ArLightEstimate *out_light_estimate)`	`void` Gets the current ambient light estimate, if light estimation was enabled.
`ArFrame_getTimestamp(const ArSession session, const ArFrame frame, int64_t *out_timestamp_ns)`	`void` Returns the timestamp in nanoseconds when this image was captured.
`ArFrame_getUpdatedAnchors(const ArSession session, const ArFrame frame, ArAnchorList *out_anchor_list)`	`void` Gets the set of anchors that were changed by the ArSession_update() that produced this Frame.
`ArFrame_getUpdatedTrackables(const ArSession session, const ArFrame frame, ArTrackableType filter_type, ArTrackableList *out_trackable_list)`	`void` Gets the set of trackables of a particular type that were changed by the ArSession_update() call that produced this Frame.
`ArFrame_hitTest(const ArSession session, const ArFrame frame, float pixel_x, float pixel_y, ArHitResultList *hit_result_list)`	`void` Performs a ray cast from the user's device in the direction of the given location in the camera view.
`ArFrame_hitTestRay(const ArSession session, const ArFrame frame, const float ray_origin_3, const float ray_direction_3, ArHitResultList *hit_result_list)`	`void` Similar to ArFrame_hitTest(), but takes an arbitrary ray in world space coordinates instead of a screen space point.
`ArFrame_transformCoordinates2d(const ArSession session, const ArFrame frame, ArCoordinates2dType input_coordinates, int32_t number_of_vertices, const float vertices_2d, ArCoordinates2dType output_coordinates, float out_vertices_2d)`	`void` Transforms a list of 2D coordinates from one 2D coordinate system to another 2D coordinate system.
`ArFrame_transformDisplayUvCoords(const ArSession session, const ArFrame frame, int32_t num_elements, const float uvs_in, float uvs_out)`	`void` Deprecated. in release 1.7.0. Please use instead: ArFrame_transformCoordinates2d(session, frame, AR_COORDINATES_2D_VIEW_NORMALIZED, num_elements, uvs_in, AR_COORDINATES_2D_TEXTURE_NORMALIZED, uvs_out); Transform the given texture coordinates to correctly show the background image.

Enumerations

ArCoordinates2dType

 ArCoordinates2dType

2d coordinate systems supported by ARCore.

Properties
`AR_COORDINATES_2D_IMAGE_NORMALIZED`	CPU image, (x,y) normalized to [0.0f, 1.0f] range.
`AR_COORDINATES_2D_IMAGE_PIXELS`	CPU image, (x,y) in pixels.
`AR_COORDINATES_2D_OPENGL_NORMALIZED_DEVICE_COORDINATES`	OpenGL Normalized Device Coordinates, display-rotated, (x,y) normalized to [-1.0f, 1.0f] range.
`AR_COORDINATES_2D_TEXTURE_NORMALIZED`	GPU texture coordinates, (s,t) normalized to [0.0f, 1.0f] range.
`AR_COORDINATES_2D_TEXTURE_TEXELS`	GPU texture, (x,y) in pixels.
`AR_COORDINATES_2D_VIEW`	Android view, display-rotated, (x,y) in pixels.
`AR_COORDINATES_2D_VIEW_NORMALIZED`	Android view, display-rotated, (x,y) normalized to [0.0f, 1.0f] range.

Typedefs

ArFrame

struct ArFrame_ ArFrame

The world state resulting from an update (value type).

Allocate with ArFrame_create()
Populate with ArSession_update()
Release with ArFrame_destroy()

Functions

ArFrame_acquireCamera

void ArFrame_acquireCamera(
  const ArSession *session,
  const ArFrame *frame,
  ArCamera **out_camera
)

Returns the camera object for the session.

Note that this Camera instance is long-lived so the same instance is returned regardless of the frame object this method was called on.

ArFrame_acquireCameraImage

ArStatus ArFrame_acquireCameraImage(
  ArSession *session,
  ArFrame *frame,
  ArImage **out_image
)

Returns the CPU image for the current frame.

Caller is responsible for later releasing the image with ArImage_release. Not supported on all devices (see https://developers.google.com/ar/discover/supported-devices). Return values:

Details
Returns	AR_SUCCESS or any of: AR_ERROR_INVALID_ARGUMENT - one more input arguments are invalid. AR_ERROR_DEADLINE_EXCEEDED - the input frame is not the current frame. AR_ERROR_RESOURCE_EXHAUSTED - the caller app has exceeded maximum number of images that it can hold without releasing. AR_ERROR_NOT_YET_AVAILABLE - image with the timestamp of the input frame was not found within a bounded amount of time, or the camera failed to produce the image

Details

Returns

AR_SUCCESS or any of:

AR_ERROR_INVALID_ARGUMENT - one more input arguments are invalid.
AR_ERROR_DEADLINE_EXCEEDED - the input frame is not the current frame.
AR_ERROR_RESOURCE_EXHAUSTED - the caller app has exceeded maximum number of images that it can hold without releasing.
AR_ERROR_NOT_YET_AVAILABLE - image with the timestamp of the input frame was not found within a bounded amount of time, or the camera failed to produce the image

ArFrame_acquireImageMetadata

ArStatus ArFrame_acquireImageMetadata(
  const ArSession *session,
  const ArFrame *frame,
  ArImageMetadata **out_metadata
)

Gets the camera metadata for the current camera image.

Details
Returns	AR_SUCCESS or any of: AR_ERROR_DEADLINE_EXCEEDED if `frame` is not the latest frame from by ArSession_update(). AR_ERROR_RESOURCE_EXHAUSTED if too many metadata objects are currently held. AR_ERROR_NOT_YET_AVAILABLE if the camera failed to produce metadata for the given frame. Note: this will commonly happen for few frames right after `ArSession_resume()` due to the camera stack bringup.

ArFrame_acquirePointCloud

ArStatus ArFrame_acquirePointCloud(
  const ArSession *session,
  const ArFrame *frame,
  ArPointCloud **out_point_cloud
)

Acquires the current set of estimated 3d points attached to real-world geometry.

A matching call to PointCloud_release() must be made when the application is done accessing the point cloud.

Note: This information is for visualization and debugging purposes only. Its characteristics and format are subject to change in subsequent versions of the API.

Details

Parameters

`session`	The ARCore session.
`frame`	The current frame.
`out_point_cloud`	Pointer to an `ArPointCloud*` receive the address of the point cloud.

Returns

AR_SUCCESS or any of:

AR_ERROR_DEADLINE_EXCEEDED if frame is not the latest frame from by ArSession_update().
AR_ERROR_RESOURCE_EXHAUSTED if too many point clouds are currently held.

ArFrame_create

void ArFrame_create(
  const ArSession *session,
  ArFrame **out_frame
)

Allocates a new ArFrame object, storing the pointer into *out_frame.

Note: the same ArFrame can be used repeatedly when calling ArSession_update.

ArFrame_destroy

void ArFrame_destroy(
  ArFrame *frame
)

Releases an ArFrame and any references it holds.

ArFrame_getAndroidSensorPose

void ArFrame_getAndroidSensorPose(
  const ArSession *session,
  const ArFrame *frame,
  ArPose *out_pose
)

Sets out_pose to the pose of the Android Sensor Coordinate System in the world coordinate space for this frame.

The orientation follows the device's "native" orientation (it is not affected by display rotation) with all axes corresponding to those of the Android sensor coordinates.

ArFrame_getDisplayGeometryChanged

void ArFrame_getDisplayGeometryChanged(
  const ArSession *session,
  const ArFrame *frame,
  int32_t *out_geometry_changed
)

Checks if the display rotation or viewport geometry changed since the previous call to ArSession_update().

The application should re-query ArCamera_getProjectionMatrix() and ArFrame_transformCoordinates2d() whenever this emits non-zero.

ArFrame_getLightEstimate

void ArFrame_getLightEstimate(
  const ArSession *session,
  const ArFrame *frame,
  ArLightEstimate *out_light_estimate
)

Gets the current ambient light estimate, if light estimation was enabled.

Details

Parameters

`session`	The ARCore session.
`frame`	The current frame.
`out_light_estimate`	The light estimate to fill. This object must have been previously created with ArLightEstimate_create().

ArFrame_getTimestamp

void ArFrame_getTimestamp(
  const ArSession *session,
  const ArFrame *frame,
  int64_t *out_timestamp_ns
)

Returns the timestamp in nanoseconds when this image was captured.

This can be used to detect dropped frames or measure the camera frame rate. The time base of this value is specifically not defined, but it is likely similar to clock_gettime(CLOCK_BOOTTIME).

ArFrame_getUpdatedAnchors

void ArFrame_getUpdatedAnchors(
  const ArSession *session,
  const ArFrame *frame,
  ArAnchorList *out_anchor_list
)

Gets the set of anchors that were changed by the ArSession_update() that produced this Frame.

Details

Parameters

`session`	The ARCore session
`frame`	The current frame.
`out_anchor_list`	The list to fill. This list must have already been allocated with ArAnchorList_create(). If previously used, the list will first be cleared.

ArFrame_getUpdatedTrackables

void ArFrame_getUpdatedTrackables(
  const ArSession *session,
  const ArFrame *frame,
  ArTrackableType filter_type,
  ArTrackableList *out_trackable_list
)

Gets the set of trackables of a particular type that were changed by the ArSession_update() call that produced this Frame.

Details

Parameters

`session`	The ARCore session
`frame`	The current frame.
`filter_type`	The type(s) of trackables to return. See ArTrackableType for legal values.
`out_trackable_list`	The list to fill. This list must have already been allocated with ArTrackableList_create(). If previously used, the list will first be cleared.

ArFrame_hitTest

void ArFrame_hitTest(
  const ArSession *session,
  const ArFrame *frame,
  float pixel_x,
  float pixel_y,
  ArHitResultList *hit_result_list
)

Performs a ray cast from the user's device in the direction of the given location in the camera view.

Intersections with detected scene geometry are returned, sorted by distance from the device; the nearest intersection is returned first.

Note: Significant geometric leeway is given when returning hit results. For example, a plane hit may be generated if the ray came close, but did not actually hit within the plane extents or plane bounds (ArPlane_isPoseInExtents() and ArPlane_isPoseInPolygon() can be used to determine these cases). A point (point cloud) hit is generated when a point is roughly within one finger-width of the provided screen coordinates.

The resulting list is ordered by distance, with the nearest hit first

Note: If not tracking, the hit_result_list will be empty.
Note: If called on an old frame (not the latest produced by ArSession_update() the hit_result_list will be empty). Note: When using AR_SESSION_FEATURE_FRONT_CAMERA, the returned hit result list will always be empty, as the camera is not AR_TRACKING_STATE_TRACKING}. Hit testing against tracked faces is not currently supported.

Details

Parameters

`session`	The ARCore session.
`frame`	The current frame.
`pixel_x`	Logical X position within the view, as from an Android UI event.
`pixel_y`	Logical X position within the view.
`hit_result_list`	The list to fill. This list must have been previously allocated using ArHitResultList_create(). If the list has been previously used, it will first be cleared.

ArFrame_hitTestRay

void ArFrame_hitTestRay(
  const ArSession *session,
  const ArFrame *frame,
  const float *ray_origin_3,
  const float *ray_direction_3,
  ArHitResultList *hit_result_list
)

Similar to ArFrame_hitTest(), but takes an arbitrary ray in world space coordinates instead of a screen space point.

Details

Parameters

`session`	The ARCore session.
`frame`	The current frame.
`ray_origin_3`	A pointer to float[3] array containing ray origin in world space coordinates.
`ray_direction_3`	A pointer to float[3] array containing ray direction in world space coordinates. Does not have to be normalized.
`hit_result_list`	The list to fill. This list must have been previously allocated using ArHitResultList_create(). If the list has been previously used, it will first be cleared.

ArFrame_transformCoordinates2d

void ArFrame_transformCoordinates2d(
  const ArSession *session,
  const ArFrame *frame,
  ArCoordinates2dType input_coordinates,
  int32_t number_of_vertices,
  const float *vertices_2d,
  ArCoordinates2dType output_coordinates,
  float *out_vertices_2d
)

Transforms a list of 2D coordinates from one 2D coordinate system to another 2D coordinate system.

For Android view coordinates (VIEW, VIEW_NORMALIZED), the view information is taken from the most recent call to ArSession_setDisplayGeometry.

Must be called on the most recently obtained ArFrame object. If this function is called on an older frame, a log message will be printed and out_vertices_2d will remain unchanged.

Some examples of useful conversions:

To transform from [0,1] range to screen-quad coordinates for rendering: VIEW_NORMALIZED -> TEXTURE_NORMALIZED
To transform from [-1,1] range to screen-quad coordinates for rendering: OPENGL_NORMALIZED_DEVICE_COORDINATES -> TEXTURE_NORMALIZED
To transform a point found by a computer vision algorithm in a cpu image into a point on the screen that can be used to place an Android View (e.g. Button) at that location: IMAGE_PIXELS -> VIEW
To transform a point found by a computer vision algorithm in a CPU image into a point to be rendered using GL in clip-space ([-1,1] range): IMAGE_PIXELS -> OPENGL_NORMALIZED_DEVICE_COORDINATES

If inputCoordinates is same as outputCoordinates, the input vertices will be copied to the output vertices unmodified.

Details

Parameters

`session`	The ARCore session.
`frame`	The current frame.
`input_coordinates`	The coordinate system used by `vectors2d_in`.
`number_of_vertices`	The number of 2D vertices to transform. `vertices_2d` and `out_vertices_2d` must point to arrays of size at least num_vertices*2.
`vertices_2d`	Input 2D vertices to transform.
`output_coordinates`	The coordinate system to convert to.
`out_vertices_2d`	Transformed 2d vertices, can be the same array as vertices_2d for in-place transform.

ArFrame_transformDisplayUvCoords

void ArFrame_transformDisplayUvCoords(
  const ArSession *session,
  const ArFrame *frame,
  int32_t num_elements,
  const float *uvs_in,
  float *uvs_out
)

Transform the given texture coordinates to correctly show the background image.

This will account for the display rotation, and any additional required adjustment. For performance, this function should be called only if ArFrame_hasDisplayGeometryChanged() emits true.

Deprecated. in release 1.7.0. Please use instead:

ArFrame_transformCoordinates2d(session, frame,
  AR_COORDINATES_2D_VIEW_NORMALIZED, num_elements, uvs_in,
  AR_COORDINATES_2D_TEXTURE_NORMALIZED, uvs_out);

Details

Parameters

`session`	The ARCore session
`frame`	The current frame.
`num_elements`	The number of floats to transform. Must be a multiple of 2. `uvs_in` and `uvs_out` must point to arrays of at least this many floats.
`uvs_in`	Input UV coordinates in normalized screen space.
`uvs_out`	Output UV coordinates in texture coordinates.