Frame

public class Frame

Captures the state and changes to the AR system from a call to update().

Protected Constructors

Frame() (FOR TESTING) Constructs a new instance for use as a mock.

Public Methods

Image	acquireCameraImage() Acquires an image from the tracking camera that corresponds to the current frame.
PointCloud	acquirePointCloud() Acquires the current set of estimated 3d points attached to real-world geometry.
Pose	getAndroidSensorPose() Returns the pose of the Android Sensor Coordinate System in the world coordinate space at the time of capture of the current camera texture.
Camera	getCamera() Returns the Camera object for the session.
ImageMetadata	getImageMetadata() Return the camera metadata for the current camera image.
LightEstimate	getLightEstimate() Returns the current ambient light estimate, if light estimation was enabled.
long	getTimestamp() Returns the timestamp in nanoseconds when this image was captured.
Collection<Anchor>	getUpdatedAnchors() Returns the anchors that were changed by the update() that returned this Frame.
<T extends Trackable> Collection<T>	getUpdatedTrackables(Class<T> filterType) Returns the trackables of a particular type that were changed by the update() that returned this Frame.
boolean	hasDisplayGeometryChanged() Checks if the display rotation or viewport geometry changed since the previous Frame.
List<HitResult>	hitTest(MotionEvent motionEvent) Similar to hitTest(float, float), but will take values from Android MotionEvent.
List<HitResult>	hitTest(float[] origin3, int originOffset, float[] direction3, int directionOffset) Similar to hitTest(float, float), but takes an arbitrary ray in world space coordinates instead of a screen-space point.
List<HitResult>	hitTest(float xPx, float yPx) Performs a ray cast from the user's device in the direction of the given location in the camera view.
void	transformDisplayUvCoords(FloatBuffer uvCoords, FloatBuffer outUvCoords) Transform the given texture coordinates to correctly show the background image.

Inherited Methods

From class java.lang.Object

Object	clone()
boolean	equals(Object arg0)
void	finalize()
final Class<?>	getClass()
int	hashCode()
final void	notify()
final void	notifyAll()
String	toString()
final void	wait(long arg0, int arg1)
final void	wait(long arg0)
final void	wait()

Protected Constructors

protected Frame ()

(FOR TESTING) Constructs a new instance for use as a mock. Calling any base method implementation on this instance may return unexpected results, throw an exception, or even crash.

To obtain a Frame normally, use update().

Public Methods

public Image acquireCameraImage ()

Acquires an image from the tracking camera that corresponds to the current frame.

Returns

• an Android image object that contains the image data from the tracking camera. The returned image object format is AIMAGE_FORMAT_YUV_420_888.

Throws

NullPointerException	if session or frame is null.
DeadlineExceededException	if the input frame is not the current frame.
ResourceExhaustedException	if the caller app has exceeded maximum number of images that it can hold without releasing.
NotYetAvailableException	if the image with the timestamp of the input frame was not found within a bounded amount of time, or the camera failed to produce the image

public PointCloud acquirePointCloud ()

Acquires the current set of estimated 3d points attached to real-world geometry. release() must be called after application is done using the PointCloud object.

Note: This information is for visualization and debugging purposes only. Its characteristics and format are subject to change in subsequent versions of the API.

Throws

ResourceExhaustedException	if too many point clouds are acquired without being released.

public Pose getAndroidSensorPose ()

Returns the pose of the Android Sensor Coordinate System in the world coordinate space at the time of capture of the current camera texture. The orientation follows the device's "native" orientation (it is not affected by display orientation).

Note: This pose is only useful when getTrackingState() returns TRACKING and otherwise should not be used.

public Camera getCamera ()

Returns the Camera object for the session. Note that this Camera instance is long-lived so the same instance is returned regardless of the frame object this method was called on.

public ImageMetadata getImageMetadata ()

Return the camera metadata for the current camera image.

Throws

NotYetAvailableException

public LightEstimate getLightEstimate ()

Returns the current ambient light estimate, if light estimation was enabled.

If lighting estimation is not enabled in the session configuration, the returned LightingEstimate will always return NOT_VALID from getState().

public long getTimestamp ()

Returns the timestamp in nanoseconds when this image was captured. This can be used to detect dropped frames or measure the camera frame rate. The time base of this value is specifically not defined, but it is likely similar to System.nanoTime().

public Collection<Anchor> getUpdatedAnchors ()

Returns the anchors that were changed by the update() that returned this Frame.

public Collection<T> getUpdatedTrackables (Class<T> filterType)

Returns the trackables of a particular type that were changed by the update() that returned this Frame. filterType may be Plane.class or Point.class, or Trackable.class to retrieve all changed trackables.

Parameters

filterType

public boolean hasDisplayGeometryChanged ()

Checks if the display rotation or viewport geometry changed since the previous Frame. The application should re-query getProjectionMatrix(float[], int, float, float) and transformDisplayUvCoords(FloatBuffer, FloatBuffer) whenever this is true.

public List<HitResult> hitTest (MotionEvent motionEvent)

Similar to hitTest(float, float), but will take values from Android MotionEvent. It is assumed that the MotionEvent is received from the same view that was used as the size for setDisplayGeometry(int, int, int).

Note: this method does not consider the action of the MotionEvent. The caller must check for appropriate action, if needed, before calling this method.

Parameters

motionEvent	an event containing the x,y coordinates to hit test

public List<HitResult> hitTest (float[] origin3, int originOffset, float[] direction3, int directionOffset)

Similar to hitTest(float, float), but takes an arbitrary ray in world space coordinates instead of a screen-space point.

Parameters

origin3	an array of 3 floats containing ray origin in world space coordinates.
originOffset	the offset into origin3 array.
direction3	an array of 3 floats containing ray direction in world space coordinates. Does not have to be normalized.
directionOffset	the offset into direction3 array.

Returns

• an ordered list of intersections with scene geometry, nearest hit first.

public List<HitResult> hitTest (float xPx, float yPx)

Performs a ray cast from the user's device in the direction of the given location in the camera view. Intersections with detected scene geometry are returned, sorted by distance from the device; the nearest intersection is returned first.

Note: Significant geometric leeway is given when returning hit results. For example, a plane hit may be generated if the ray came close, but did not actually hit within the plane extents or plane bounds (isPoseInExtents(Pose) and isPoseInPolygon(Pose) can be used to determine these cases). A point (point cloud) hit is generated when a point is roughly within one finger-width of the provided screen coordinates.

Parameters

xPx	x coordinate in pixels
yPx	y coordinate in pixels

Returns

• an ordered list of intersections with scene geometry, nearest hit first

public void transformDisplayUvCoords (FloatBuffer uvCoords, FloatBuffer outUvCoords)

Transform the given texture coordinates to correctly show the background image. This will account for the display rotation, and any additional required adjustment. For performance, this function should be called only if hasDisplayGeometryChanged() returns true.

Usage Notes / Bugs:

• Both input and output buffers must be direct and native byte order.
• Position and limit of buffers is ignored.
• Capacity of both buffers must be identical.
• Capacity of both buffers must be a multiple of 2.

Note: both buffer positions will remain unmodified after this call.

Parameters

uvCoords	The uv coordinates to transform.
outUvCoords	The buffer to hold the transformed uv coordinates. Must have enough remaining elements to fit the input uvCoords.