GCKOpenGLESVideoFrameInput Class

GCKOpenGLESVideoFrameInput Class Reference

Overview

Specialization of GCKVideoFrameInput for OpenGL ES.

This video frame input makes it easier to integrate an OpenGL ES renderer with Remote Display by providing a fast (but somewhat inaccurate) color space conversion pipeline. Apps that wish to have full control over this process must use the session object directly to dequeue video frame buffers and enqueue them for encoding and transmission once rendered.

Both OpenGL ES 2 and OpenGL ES 3 contexts are supported. OpenGL ES 1 contexts are not.

See also
GCKRemoteDisplaySession

Initialization

Once a Remote Display session has been negotiated, initialize a frame input with it and set the OpenGL ES context property. The frame input will synchronously create all of its required resources in the property setter. Finally, set up the OpenGL ES rendering code for Remote Display, creating a 2D texture matching the size of the Remote Display video stream as the final render target. The texture format must be color renderable and must support bilinear filtering. The texture must use the GL_TEXTURE_2D target.

Producing a frame

To produce a Remote Display frame, first render the frame then call encodeFrame: (GCKOpenGLESVideoFrameInput) with the render target texture. The frame input will encode 2 render passes to perform color space conversion. Because OpenGL ES does not provide a callback mechanism for command buffer completion, syncAndSubmitFrame (GCKOpenGLESVideoFrameInput) must eventually be called to synchronously wait for the frame to finish processing.

If the app has no further work to do after encoding a Remote Display frame, it may call syncAndSubmitFrame (GCKOpenGLESVideoFrameInput) right away and block until the frame is ready. Otherwise, it is generally safe to call syncAndSubmitFrame (GCKOpenGLESVideoFrameInput) at the beginning of the next frame. By that time, the previously encoded frame will almost certainly be ready and the app will not block.

Thread safety

Because this frame input calls into OpenGL, it must be used on the same thread using its current OpenGL ES context, or at least serially with any other thread using the context. To avoid race conditions when objects of this class are deallocated, set the context property to nil on the thread that owns the OpenGL ES context. This applies to every method and property get & set.

State mutation

When setting a context, encoding a frame or syncing and submitting a frame, OpenGL ES state must be mutated. Whenever possible, the frame input saves and restores the global context state it mutates as well as the thread's current context.

Inherits GCKVideoFrameInput.

Instance Method Summary

(void) - encodeFrame:
 Encode commands to process and transmit a frame. More...
 
(void) - syncAndSubmitFrame
 Wait for the previously encoded frame to be ready and submit it to the Remote Display session. More...
 
(instancetype) - initWithSession:
 Initialize a new video frame input bound to the specified session. More...
 
(instancetype) - NS_UNAVAILABLE
 

Properties

EAGLContext * context
 The OpenGL ES context to use for resource allocations. More...
 
matrix_float3x3 textureCoordinateMatrix
 Transform matrix applied to the texture coordinates of enqueued frames. More...
 
id< GCKRemoteDisplaySessionsession
 Remote Display session the input is bound to. More...
 
uint32_t width
 Width of the session's video stream. More...
 
uint32_t height
 Height of the session's video stream. More...
 

Method Detail

- (void) encodeFrame: (GLuint)  texture

Encode commands to process and transmit a frame.

The frame input will encode 2 render passes to convert the frame to Y'CbCr with 4:2:0 chroma subsampling. It will also encode a command buffer fence to test if the frame has finished processing. This fence must eventually be tested and potentially waited on by calling syncAndSubmitFrame.

The current OpenGL ES context is used and is assumed to be the frame input's context. If not, behavior is undefined.

If syncAndSubmitFrame is not called before the next call to encodeFrame:, the previously encoded frame is dropped.

If anything goes wrong, such as if the session runs out of video frame buffers or the texture cannot be sampled, the frame is dropped.

The texture's size must match the drawable size, must support bilinear filtering and must be a 2D texture (using the GL_TEXTURE_2D target). The same texture can be used each frame since its content does not need to remain unchanged.

- (void) syncAndSubmitFrame

Wait for the previously encoded frame to be ready and submit it to the Remote Display session.

Waits on the command buffer fence encoded by encodeFrame: for up to 100 ms. If the fence is signalled before this timeout, the frame is submitted to the session to be compressed and transmitted. The presentation timestamp of the frame will be the result of CACurrentMediaTime when the fence is signalled.

If no frame is pending, returns immediately. If the fence is not signalled in time, the frame is dropped.

The current OpenGL ES context is used and is assumed to be the frame input's context. If not, behavior is undefined.

- (instancetype) initWithSession: (id< GCKRemoteDisplaySession >)  session

Initialize a new video frame input bound to the specified session.

Property Documentation

- (EAGLContext*) context
readwritenonatomicassign

The OpenGL ES context to use for resource allocations.

Changing this property reallocates resources synchronously and can take some amount of time. Setting the context to nil will immediately free all allocated resources.

- (matrix_float3x3) textureCoordinateMatrix
readwritenonatomicassign

Transform matrix applied to the texture coordinates of enqueued frames.

This matrix is applied to the normalized texture coordinates used to sample enqueued frames during the color space conversion rendering passes encoded by this frame input. It can be more efficient to use this property than to perform the transform as a separate rendering pass.

For example, to perform a y-flip, set the following matrix (here as an array of column vectors): { {1.0, 0.0, 0.0}, {0.0, -1.0, 0.0}, {0.0, 1.0, 0.0} }

Defaults to the identity matrix.

- (id<GCKRemoteDisplaySession>) session
readnonatomicweakinherited

Remote Display session the input is bound to.

- (uint32_t) width
readnonatomicassigninherited

Width of the session's video stream.

- (uint32_t) height
readnonatomicassigninherited

Height of the session's video stream.

Google Cast iOS Sender API Reference v 2.10.1 4691