Defines an interface for controlling playback of media elementary streams. ## Macros ### SB_PLAYER_INITIAL_TICKET The value of the initial ticket held by the player before the first seek. The player will use this ticket value to make the first call to SbPlayerStatusFunc with kSbPlayerStateInitialized. ### SB_PLAYER_NO_DURATION The value to pass into SbPlayerCreate's `duration_pts` argument for cases where the duration is unknown, such as for live streams. ### kSbPlayerInvalid Well-defined value for an invalid player. ### kSbPlayerWriteDurationLocal The audio write duration when all the audio connectors are local. ### kSbPlayerWriteDurationRemote The audio write duration when at least one of the audio connectors are remote. ## Enums ### SbPlayerDecoderState An indicator of whether the decoder can accept more samples. #### Values - `kSbPlayerDecoderStateNeedsData` The decoder is asking for one more sample. ### SbPlayerSampleSideDataType Identify the type of side data accompanied with `SbPlayerSampleInfo`, as side data may come from multiple sources. #### Values - `kMatroskaBlockAdditional` The side data comes from the BlockAdditional data in the Matroska/Webm container. The first 8 bytes of the data contains the value of BlockAddID in big endian format, followed by the content of BlockAdditional. See: [https://datatracker.ietf.org/doc/draft-lhomme-cellar-matroska/03](https://datatracker.ietf.org/doc/draft-lhomme-cellar-matroska/03) [https://www.webmproject.org/docs/container/#BlockAdditional](https://www.webmproject.org/docs/container/#BlockAdditional) ### SbPlayerState An indicator of the general playback state. #### Values - `kSbPlayerStateInitialized` The player has just been initialized. It is expecting an SbPlayerSeek() call to enter the prerolling state. - `kSbPlayerStatePrerolling` The player is prerolling, collecting enough data to fill the pipeline before presentation starts. After the first preroll is completed, there should always be a video frame to render, even if the player goes back to Prerolling after a Seek. - `kSbPlayerStatePresenting` The player is presenting media, and it is either paused or actively playing in real-time. Note that the implementation should use this state to signal that the preroll has been finished. - `kSbPlayerStateEndOfStream` The player is presenting media, but it is paused at the end of the stream. - `kSbPlayerStateDestroyed` The player has been destroyed, and will send no more callbacks. ## Typedefs ### SbPlayer An opaque handle to an implementation-private structure representing a player. #### Definition typedef struct SbPlayerPrivate* SbPlayer ### SbPlayerDeallocateSampleFunc Callback to free the given sample buffer data. When more than one buffer are sent in SbPlayerWriteSample(), the implementation only has to call this callback with `sample_buffer` points to the first buffer. #### Definition typedef void(* SbPlayerDeallocateSampleFunc) (SbPlayer player, void *context, const void *sample_buffer) ### SbPlayerDecoderStatusFunc Callback for decoder status updates, called in response to a call to SbPlayerSeek() or SbPlayerWriteSample(). This callback will never be called until at least one call to SbPlayerSeek has occurred. `ticket` will be set to the ticket passed into the last received call to SbPlayerSeek() at the time this callback was dispatched. This is to distinguish status callbacks for interrupting seeks. These callbacks will happen on a different thread than the calling thread, and it is not valid to call SbPlayer functions from within this callback. After an update with kSbPlayerDecoderStateNeedsData, the user of the player will make at most one call to SbPlayerWriteSample() or SbPlayerWriteEndOfStream(). The player implementation should update the decoder status again after such call to notify its user to continue writing more frames. #### Definition typedef void(* SbPlayerDecoderStatusFunc) (SbPlayer player, void *context, SbMediaType type, SbPlayerDecoderState state, int ticket) ### SbPlayerErrorFunc Callback for player errors, that may set a `message`. `error`: indicates the error code. `message`: provides specific informative diagnostic message about the error condition encountered. It is ok for the message to be an empty string or NULL if no information is available. #### Definition typedef void(* SbPlayerErrorFunc) (SbPlayer player, void *context, SbPlayerError error, const char *message) ### SbPlayerStatusFunc Callback for player status updates. These callbacks will happen on a different thread than the calling thread, and it is not valid to call SbPlayer functions from within this callback. #### Definition typedef void(* SbPlayerStatusFunc) (SbPlayer player, void *context, SbPlayerState state, int ticket) ## Structs ### SbPlayerCreationParam The playback related parameters to pass into SbPlayerCreate() and SbPlayerGetPreferredOutputMode(). #### Members - `SbDrmSystem drm_system` Provides an appropriate DRM system if the media stream has encrypted portions. It will be `kSbDrmSystemInvalid` if the stream does not have encrypted portions. - `SbMediaAudioStreamInfo audio_stream_info` Contains a populated SbMediaAudioStreamInfo if `audio_stream_info.codec` isn't `kSbMediaAudioCodecNone`. When `audio_stream_info.codec` is `kSbMediaAudioCodecNone`, the video doesn't have an audio track. - `SbMediaVideoStreamInfo video_stream_info` Contains a populated SbMediaVideoStreamInfo if `video_stream_info.codec` isn't `kSbMediaVideoCodecNone`. When `video_stream_info.codec` is `kSbMediaVideoCodecNone`, the video is audio only. - `SbPlayerOutputMode output_mode` Selects how the decoded video frames will be output. For example, `kSbPlayerOutputModePunchOut` indicates that the decoded video frames will be output to a background video layer by the platform, and `kSbPlayerOutputDecodeToTexture` indicates that the decoded video frames should be made available for the application to pull via calls to SbPlayerGetCurrentFrame(). ### SbPlayerInfo Information about the current media playback state. #### Members - `int64_t current_media_timestamp` The position of the playback head, as precisely as possible, in microseconds. - `int64_t duration` The known duration of the currently playing media stream, in microseconds. - `int64_t start_date` The result of getStartDate for the currently playing media stream, in microseconds since the epoch of January 1, 1601 UTC. - `int frame_width` The width of the currently displayed frame, in pixels, or 0 if not provided by this player. - `int frame_height` The height of the currently displayed frame, in pixels, or 0 if not provided by this player. - `bool is_paused` Whether playback is currently paused. - `double volume` The current player volume in \[0, 1\]. - `int total_video_frames` The number of video frames sent to the player since the creation of the player. - `int dropped_video_frames` The number of video frames decoded but not displayed since the creation of the player. - `int corrupted_video_frames` The number of video frames that failed to be decoded since the creation of the player. - `double playback_rate` The rate of playback. The video is played back in a speed that is proportional to this. By default it is 1.0 which indicates that the playback is at normal speed. When it is greater than one, the video is played in a faster than normal speed. When it is less than one, the video is played in a slower than normal speed. Negative speeds are not supported. ### SbPlayerSampleInfo Information about the samples to be written into SbPlayerWriteSamples(). #### Members - `SbMediaType type` - `const void* buffer` Points to the buffer containing the sample data. - `int buffer_size` Size of the data pointed to by `buffer`. - `int64_t timestamp` The timestamp of the sample in microseconds since Windows epoch UTC. - `SbPlayerSampleSideData* side_data` Points to an array of side data for the input, when available. - `int side_data_count` The number of side data pointed by `side_data`. It should be set to 0 if there is no side data for the input. - `SbMediaAudioSampleInfo audio_sample_info` Information about an audio sample. This value can only be used when `type` is kSbMediaTypeAudio. - `SbMediaVideoSampleInfo video_sample_info` Information about a video sample. This value can only be used when `type` is kSbMediaTypeVideo. - `union { SbMediaAudioSampleInfo, SbMediaVideoSampleInfo }` - `const SbDrmSampleInfo* drm_info` The DRM system related info for the media sample. This value is required for encrypted samples. Otherwise, it must be `NULL`. ### SbPlayerSampleSideData Side data accompanied with `SbPlayerSampleInfo`, it can be arbitrary binary data coming from multiple sources. #### Members - `SbPlayerSampleSideDataType type` - `const uint8_t* data` `data` will remain valid until SbPlayerDeallocateSampleFunc() is called on the `SbPlayerSampleInfo::buffer` the data is associated with. - `size_t size` The size of the data pointed by `data`, in bytes. ## Functions ### SbPlayerCreate Creates a player that will be displayed on `window` for the specified `video_codec` and `audio_codec`, acquiring all resources needed to operate it, and returning an opaque handle to it. The expectation is that a new player will be created and destroyed for every playback. This function returns the created player. Note the following: - The associated decoder of the returned player should be assumed to not be in `kSbPlayerDecoderStateNeedsData` until SbPlayerSeek() has been called on it. - If there is a platform limitation on how many players can coexist simultaneously, then calls made to this function that attempt to exceed that limit must return `kSbPlayerInvalid`. Multiple calls to SbPlayerCreate must not cause a crash. `window`: The window that will display the player. `window` can be `kSbWindowInvalid` for platforms where video is only displayed on a particular window that the underlying implementation already has access to. `video_codec`: The video codec used for the player. If `video_codec` is `kSbMediaVideoCodecNone`, the player is an audio-only player. If `video_codec` is any other value, the player is an audio/video decoder. This can be set to `kSbMediaVideoCodecNone` to play a video with only an audio track. `audio_codec`: The audio codec used for the player. The caller must provide a populated `audio_sample_info` if audio codec is `kSbMediaAudioCodecAac`. Can be set to `kSbMediaAudioCodecNone` to play a video without any audio track. In such case `audio_sample_info` must be NULL. `drm_system`: If the media stream has encrypted portions, then this parameter provides an appropriate DRM system, created with `SbDrmCreateSystem()`. If the stream does not have encrypted portions, then `drm_system` may be `kSbDrmSystemInvalid`. `audio_sample_info`: Note that the caller must provide a populated `audio_sample_info` if the audio codec is `kSbMediaAudioCodecAac`. Otherwise, `audio_sample_info` can be NULL. See media.h for the format of the `SbMediaAudioSampleInfo` struct. Note that `audio_specific_config` is a pointer and the content it points to is no longer valid after this function returns. The implementation has to make a copy of the content if it is needed after the function returns. `max_video_capabilities`: This string communicates the max video capabilities required to the platform. The web app will not provide a video stream exceeding the maximums described by this parameter. Allows the platform to optimize playback pipeline for low quality video streams if it knows that it will never adapt to higher quality streams. The string uses the same format as the string passed in to SbMediaCanPlayMimeAndKeySystem(), for example, when it is set to "width=1920; height=1080; framerate=15;", the video will never adapt to resolution higher than 1920x1080 or frame per second higher than 15 fps. When the maximums are unknown, this will be set to NULL. `sample_deallocator_func`: If not `NULL`, the player calls this function on an internal thread to free the sample buffers passed into SbPlayerWriteSample(). `decoder_status_func`: If not `NULL`, the decoder calls this function on an internal thread to provide an update on the decoder's status. No work should be done on this thread. Rather, it should just signal the client thread interacting with the decoder. `player_status_func`: If not `NULL`, the player calls this function on an internal thread to provide an update on the playback status. No work should be done on this thread. Rather, it should just signal the client thread interacting with the decoder. `player_error_func`: If not `NULL`, the player calls this function on an internal thread to provide an update on the error status. This callback is responsible for setting the media error message. `context`: This is passed to all callbacks and is generally used to point at a class or struct that contains state associated with the player. `output_mode`: Selects how the decoded video frames will be output. For example, kSbPlayerOutputModePunchOut indicates that the decoded video frames will be output to a background video layer by the platform, and kSbPlayerOutputDecodeToTexture indicates that the decoded video frames should be made available for the application to pull via calls to SbPlayerGetCurrentFrame(). `provider`: Only present in Starboard version 3 and up. If not `NULL`, then when output_mode == kSbPlayerOutputModeDecodeToTexture, the player MAY use the provider to create SbDecodeTargets on the renderer thread. A provider may not always be needed by the player, but if it is needed, and the provider is not given, the player will fail by returning `kSbPlayerInvalid`. If `NULL` is passed to any of the callbacks (`sample_deallocator_func`, `decoder_status_func`, `player_status_func`, or `player_error_func` if it applies), then `kSbPlayerInvalid` must be returned. #### Declaration SbPlayer SbPlayerCreate(SbWindow window, const SbPlayerCreationParam *creation_param, SbPlayerDeallocateSampleFunc sample_deallocate_func, SbPlayerDecoderStatusFunc decoder_status_func, SbPlayerStatusFunc player_status_func, SbPlayerErrorFunc player_error_func, void *context, SbDecodeTargetGraphicsContextProvider *context_provider) ### SbPlayerDestroy Destroys `player`, freeing all associated resources. - Upon calling this method, there should be one call to the player status callback (i.e. `player_status_func` used in the creation of the player) which indicates the player is destroyed. Note, the callback has to be inflight when SbPlayerDestroyed is called. - No more other callbacks should be issued after this function returns. - It is not allowed to pass `player` into any other `SbPlayer` function once SbPlayerDestroy has been called on that player. `player`: The player to be destroyed. Must not be `kSbPlayerInvalid`. #### Declaration void SbPlayerDestroy(SbPlayer player) ### SbPlayerGetAudioConfiguration Returns the audio configurations used by `player`. Returns true when `out_audio_configuration` is filled with the information of the configuration of the audio output devices used by `player`. Returns false for `index` 0 to indicate that there is no audio output for this `player`. Returns false for `index` greater than 0 to indicate that there are no more audio output configurations other than the ones already returned. The app will use the information returned to determine audio related behaviors, like: - Audio Write Duration: Audio write duration is how far past the current playback position the app will write audio samples. The app will write all samples between `current_playback_position` and `current_playback_position` - `audio_write_duration`, as soon as they are available. - `audio_write_duration` will be to `kSbPlayerWriteDurationLocal` when all audio configurations linked to `player` is local, or if there isn't any audio output. It will be set to `kSbPlayerWriteDurationRemote` for remote or wireless audio outputs, i.e. one of `kSbMediaAudioConnectorBluetooth` or `kSbMediaAudioConnectorRemote*`. - The app only guarantees to write `audio_write_duration` past `current_playback_position`, but the app is free to write more samples than that. So the platform shouldn't rely on this for flow control. The platform should achieve flow control by sending `kSbPlayerDecoderStateNeedsData` less frequently. - The platform is responsible for guaranteeing that when only `audio_write_duration` audio samples are written at a time, no playback issues occur (such as transient or indefinite hanging). The audio configurations should be available as soon as possible, and they have to be available when the `player` is at `kSbPlayerStatePresenting`, unless the audio codec is `kSbMediaAudioCodecNone` or there's no written audio inputs. The app will set `audio_write_duration` to `kSbPlayerWriteDurationLocal` when the audio configuration isn't available (i.e. the function returns false when index is 0). The platform has to make the audio configuration available immediately after the SbPlayer is created, if it expects the app to treat the platform as using wireless audio outputs. Once at least one audio configurations are returned, the return values and their orders shouldn't change during the life time of `player`. The platform may inform the app of any changes by sending `kSbPlayerErrorCapabilityChanged` to request a playback restart. `player`: The player about which information is being retrieved. Must not be `kSbPlayerInvalid`. `index`: The index of the audio output configuration. Must be greater than or equal to 0. `out_audio_configuration`: The information about the audio output, refer to `SbMediaAudioConfiguration` for more details. Must not be NULL. #### Declaration bool SbPlayerGetAudioConfiguration(SbPlayer player, int index, SbMediaAudioConfiguration *out_audio_configuration) ### SbPlayerGetCurrentFrame Given a player created with the kSbPlayerOutputModeDecodeToTexture output mode, it will return a SbDecodeTarget representing the current frame to be rasterized. On GLES systems, this function must be called on a thread with an EGLContext current, and specifically the EGLContext that will be used to eventually render the frame. If this function is called with a `player` object that was created with an output mode other than kSbPlayerOutputModeDecodeToTexture, kSbDecodeTargetInvalid is returned. `player` must not be `kSbPlayerInvalid`. #### Declaration SbDecodeTarget SbPlayerGetCurrentFrame(SbPlayer player) ### SbPlayerGetInfo Gets a snapshot of the current player state and writes it to `out_player_info`. This function may be called very frequently from arbitrary threads and is expected to be inexpensive and thread safe. `player`: The player about which information is being retrieved. Must not be `kSbPlayerInvalid`. `out_player_info`: The information retrieved for the player. #### Declaration void SbPlayerGetInfo(SbPlayer player, SbPlayerInfo *out_player_info) ### SbPlayerGetMaximumNumberOfSamplesPerWrite Returns the maximum number of samples that can be written in a single call to SbPlayerWriteSamples(). Returning a value greater than one can improve performance by allowing SbPlayerWriteSamples() to write multiple samples in one call. Note that this feature is currently disabled in Cobalt where SbPlayerWriteSamples() will always be called with one sample. `player`: The player for which the number is retrieved. `sample_type`: The type of sample for which the number is retrieved. See the `SbMediaType` enum in media.h. #### Declaration int SbPlayerGetMaximumNumberOfSamplesPerWrite(SbPlayer player, SbMediaType sample_type) ### SbPlayerGetPreferredOutputMode Returns the preferred output mode of the implementation when a video described by `creation_param` is played. It is assumed that it is okay to call SbPlayerCreate() with the same video described by `creation_param`, with its `output_mode` member replaced by the returned output mode. When the caller has no preference on the output mode, it will set `creation_param->output_mode` to `kSbPlayerOutputModeInvalid`, and the implementation can return its preferred output mode based on the information contained in `creation_param`. The caller can also set `creation_param->output_mode` to its preferred output mode, and the implementation should return the same output mode if it is supported, otherwise the implementation should return an output mode that it is supported, as if `creation_param->output_mode` is set to `kSbPlayerOutputModeInvalid` prior to the call. Note that it is not the responsibility of this function to verify whether the video described by `creation_param` can be played on the platform, and the implementation should try its best effort to return a valid output mode. `creation_param` must not be NULL. #### Declaration SbPlayerOutputMode SbPlayerGetPreferredOutputMode(const SbPlayerCreationParam *creation_param) ### SbPlayerIsValid Returns whether the given player handle is valid. #### Declaration static bool SbPlayerIsValid(SbPlayer player) ### SbPlayerSeek Tells the player to freeze playback (if playback has already started), reset or flush the decoder pipeline, and go back to the Prerolling state. The player should restart playback once it can display the frame at `seek_to_timestamp`, or the closest it can get. (Some players can only seek to I-Frames, for example.) - Seek must be called before samples are sent when starting playback for the first time, or the client never receives the `kSbPlayerDecoderStateNeedsData` signal. - A call to seek may interrupt another seek. - After this function is called, the client should not send any more audio or video samples until `SbPlayerDecoderStatusFunc` is called back with `kSbPlayerDecoderStateNeedsData` for each required media type. `SbPlayerDecoderStatusFunc` is the `decoder_status_func` callback function that was specified when the player was created (SbPlayerCreate). `player`: The SbPlayer in which the seek operation is being performed. Must not be `kSbPlayerInvalid`. `seek_to_timestamp`: The frame at which playback should begin. `ticket`: A user-supplied unique ID to be passed to all subsequent `SbPlayerDecoderStatusFunc` calls. (That is the `decoder_status_func` callback function specified when calling SbPlayerCreate.) The `ticket` value is used to filter calls that may have been in flight when SbPlayerSeek was called. To be very specific, once SbPlayerSeek has been called with ticket X, a client should ignore all `SbPlayerDecoderStatusFunc` calls that do not pass in ticket X. #### Declaration void SbPlayerSeek(SbPlayer player, int64_t seek_to_timestamp, int ticket) ### SbPlayerSetBounds Sets the player bounds to the given graphics plane coordinates. The changes do not take effect until the next graphics frame buffer swap. The default bounds for a player is the full screen. This function is only relevant when the `player` is created with the kSbPlayerOutputModePunchOut output mode, and if this is not the case then this function call can be ignored. This function is called on every graphics frame that changes the video bounds. For example, if the video bounds are being animated, then this will be called at up to 60 Hz. Since the function could be called up to once per frame, implementers should take care to avoid related performance concerns with such frequent calls. `player`: The player that is being resized. Must not be `kSbPlayerInvalid`. `z_index`: The z-index of the player. When the bounds of multiple players are overlapped, the one with larger z-index will be rendered on top of the ones with smaller z-index. `x`: The x-coordinate of the upper-left corner of the player. `y`: The y-coordinate of the upper-left corner of the player. `width`: The width of the player, in pixels. `height`: The height of the player, in pixels. #### Declaration void SbPlayerSetBounds(SbPlayer player, int z_index, int x, int y, int width, int height) ### SbPlayerSetPlaybackRate Set the playback rate of the `player`. `rate` is default to 1.0 which indicates the playback is at its original speed. A `rate` greater than one will make the playback faster than its original speed. For example, when `rate` is 2, the video will be played at twice the speed as its original speed. A `rate` less than 1.0 will make the playback slower than its original speed. When `rate` is 0, the playback will be paused. The function returns true when the playback rate is set to `playback_rate` or to a rate that is close to `playback_rate` which the implementation supports. It returns false when the playback rate is unchanged, this can happen when `playback_rate` is negative or if it is too high to support. `player` must not be `kSbPlayerInvalid`. #### Declaration bool SbPlayerSetPlaybackRate(SbPlayer player, double playback_rate) ### SbPlayerSetVolume Sets the player's volume. `player`: The player in which the volume is being adjusted. Must not be `kSbPlayerInvalid`. `volume`: The new player volume. The value must be between `0.0` and `1.0`, inclusive. A value of `0.0` means that the audio should be muted, and a value of `1.0` means that it should be played at full volume. #### Declaration void SbPlayerSetVolume(SbPlayer player, double volume) ### SbPlayerWriteEndOfStream Writes a marker to `player`'s input stream of `stream_type` indicating that there are no more samples for that media type for the remainder of this media stream. This marker is invalidated, along with the rest of the stream's contents, after a call to SbPlayerSeek. `player`: The player to which the marker is written. `stream_type`: The type of stream for which the marker is written. #### Declaration void SbPlayerWriteEndOfStream(SbPlayer player, SbMediaType stream_type) ### SbPlayerWriteSamples Writes samples of the given media type to `player`'s input stream. The lifetime of `sample_infos`, and the members of its elements like `buffer`, `video_sample_info`, and `drm_info` (as well as member `subsample_mapping` contained inside it) are not guaranteed past the call to SbPlayerWriteSamples(). That means that before returning, the implementation must synchronously copy any information it wants to retain from those structures. SbPlayerWriteSamples() allows writing of multiple samples in one call. `player`: The player to which the sample is written. Must not be `kSbPlayerInvalid`. `sample_type`: The type of sample being written. See the `SbMediaType` enum in media.h. `sample_infos`: A pointer to an array of SbPlayerSampleInfo with `number_of_sample_infos` elements, each holds the data for an sample, i.e. a sequence of whole NAL Units for video, or a complete audio frame. `sample_infos` cannot be assumed to live past the call into SbPlayerWriteSamples(), so it must be copied if its content will be used after SbPlayerWriteSamples() returns. Subsequent calls to SbPlayerWriteSamples() may pass samples with a media configuration (such as codec or MIME type) that differs from the player's current active configuration. When this transition is supported (see SbMediaCanChangeType()), the player is expected to handle the configuration switch on the active SbPlayer instance, allowing for the incoming samples to be written. The caller is guaranteed not to attempt dynamic configuration changes for a transition if SbMediaCanChangeType() returns false. `number_of_sample_infos`: Specify the number of samples contained inside `sample_infos`. It has to be at least one, and at most the return value of SbPlayerGetMaximumNumberOfSamplesPerWrite(). #### Declaration void SbPlayerWriteSamples(SbPlayer player, SbMediaType sample_type, const SbPlayerSampleInfo *sample_infos, int number_of_sample_infos)