Daydream Media App Template

The Media App Template demonstrates best practices for building media-viewing apps on Daydream. The template can be used out-of-the-box to view media content that is stored locally on a user's device. The template can also be customized to fit your desired style or adapted to view streamed content from a server.

Download on GitHub

This template includes the following features, all of which are fully customizable or removable depending on your specific use case:

  • Permissions Flow: This feature lets users leave VR temporarily to grant required permissions.
  • File Selector: This feature uses paginated scrolling to allow users to navigate their file system to select media files.
  • Media Player: The player or set of players used to display selected media.
  • Playback Controls: The set of controls used to manipulate the media player.
  • Storyboard: The preview window that shows a given frame when the user hovers over the video scrubber.
  • Screen Manipulation: This feature lets users move the media player to allow for various viewing positions.
  • Screen Lighting: This feature lights the screen, increasing the sense of realism.
  • Controller Idling: This feature hides the controller when it's idling.
  • Media Format Detection: The feature that automatically detects the stereoscopic and projection format of a selected media file and lets you match it to a proper media player.

Permissions Flow

To view sideloaded content the user must grant the app READ_EXTERNAL_STORAGE permission, which requires the user to exit VR.

The media template app handles this by using PermissionsFlowController, a script attached to the PermissionsFlow prefab.

The flow works as follows:

  • Upon launching, PermissionsFlowController checks if READ_EXTERNAL_STORAGE permissions need to be granted by using GvrPermissionsRequestor, included with the Google VR SDK for Unity.
  • If permissions are required, a message is displayed in VR informing the user that they need to grant permissions to continue. The message provides a button that the user needs to press to proceed. Both these UI elements are child objects of the PermissionsFlow prefab.
  • Upon pressing the button, the user is instructed to remove their phone from the headset. After removing the phone, the user is prompted to grant the needed permission. After permissions are granted, the user is returned to VR.
  • When PermissionsFlowController detects that permissions have been granted, it destroys itself and instatiates a prefab specifed through serialization to kickoff the next flow. In this case, that's File Selector.

File Selector

The File Selector is a UI that uses paginated scrolling to navigate the file system to select files (In this case, videos and photos). It can also be modified to navigate videos and photos coming from sources other than the file system. The UI includes a breadcrumb trail used to show where the user is in the folder hierarchy and allow navigation back up the file hierarchy.

The File Selector makes use of the paged scrolling sample included with the Google VR SDK for Unity. You can try out the sample by running the scene ScrollingUIDemo. The class PagedScrollRect is used to control navigation between pages.

The UI is managed by the class FileSelectorPageProvider. This class is also responsible for setting the breadcrumb trail. A Breadcrumb is a clickable UI element for a particular piece of BreadcrumbData. When clicked, FileSelectorPageProvider registers the event and updates the UI and the BreadcrumbTrail accordingly.

Media Player

The Media App Template supports having a variety of media players that play a variety of types of media. The out-of-the-box media player is a basic implementation that will display videos and photos. The player also detects whether a video or photo is spherical or stereoscopic and display it in the proper format. it will

The basic media player is managed by the class BaseMediaPlayer. The concrete implementation of the media players is hidden behind abstract classes. You can easily replace the implementation with custom media players by creating custom subclasses of BaseMediaPlayer. When a file is selected by the File Selector, the MediaSelector class will create the correct type of media player based on the file that was selected.

Playback Controls

Playback controls are the UI elements used to control the media player, such as the skip forward, skip backward, pause, and restart buttons. Some of them work with all types of media while some of them only work with videos or photos. Playback controls that are not compatible with the current media player are automatically hidden and the layout is adjusted accordingly.

PlaybackControlsManager is responsible for toggling the playback controls on/off when a click occurs and when the controls have been idle for several seconds. PlaybackControls is responsible for setting up all of the IPlaybackControl implementations that are on children of the PlaybackControls.

Storyboard

When the user hovers over the video scrubber, a small preview window pops up above the pointer, displaying a frame close to that point in the video. We call that preview window the Storyboard and the individual frames Storyboard frames.

The class PlaybackScrubber instantiates a prefab of type BaseVideoPlayerStoryBoard when the pointer begins hovering over the PlaybackScrubber. The Media App Template comes with one implementation of BaseVideoPlayerStoryboard called LocalVideoPlayerStoryboard. This implementation uses an android plugin to grab keyframes of local video files and stores them in an LRU cache. It divides the video into 50 segments and displays the closest keyframe for each segment.

Screen Manipulation

Screen manipulation functionality allows the user to reposition and resize the screen. This makes the media app usable in multiple positions, such as lying in bed When switching between videos and photos, the screen’s position and scale are maintained so that you don’t need to re-adjust it every time you enter a new video. The file selector can be similarly repositioned.

The class MediaScreenController is used to reposition the screen when drag events occur and to resize the screen when scroll events occur. MediaScreenController must be placed on the same game object as the screen itself.

Screen Light

Real screens emit light which in turn affects the ambient light around them. The media app template mimics this effect by using the class AverageColorCalculator to determine the average color emitted by the media. The screen light is then used in conjuction with a Unity light to render the light with the correct intensity and color from th e screen. These scripts are used in the FlatVideoScreen and FlatPhotoScreen prefabs.

Controller Idling

To prevent the controller UI from distracting the user while they are watching a video, the controller goes into an idle state when it hasn't been moved for a few seconds as determined by the ControllerIdleCondition class.

The class HideOnConditions is used to hide the laser and the reticle after the appropriate idle time. This class can react to a number of conditions using the ConditionsManager singleton. Each condition is an implementation of BaseCondition, which registers itself with ConditionsManager when enabled. Example implementation of BaseCondition include PointerHoveringCondition, ControllerIdleCondition, and PlaybackControlsOpenCondition.

Media Format Detection

There is a current lack of standardization for detecting the stereo or projection format of videos and photos. Media App Template can detect format in three ways:

  • By checking what format was used the last time the file was played.
  • By analyzing the video metadata.
  • By analyzing the image itself.

Additionally there is a menu accessible when the playback controls are open that allows the user to manually change the format in case the detected format is incorrect.

Each manner of detection is handled by a projection detector that inherits from the class BaseStereoProjectionDetector. The drojection detectors are stored in a prioritized list by the media player, and the media file will play in the format detected by the highest priority detector that was able to successfully find a format. If no format is detected, it will default to flat and mono.

This is the list of project detectors in order of priority:

  • SavedStereoProjectionDetector
    • Most recently used format the last time the file was used.
    • Stores the format in-between sessions in the persistentDataPath.
  • VideoMetadataProjectionDetector
  • ImageBasedProjectionDetector
    • Attempts to detect the format by analyzing an image. For videos, takes a frame one quarter of the way through the video.