Implement the Co-Doing API

This page describes how to use the Google Meet Live Sharing Co-Doing API to support a co-watching scenario.

Note: To do this properly requires effort to address several edge cases around latency and other delays. The following examples address some use cases in the co-watching scenario.

Initial setup

Similar to the Co-Watching initial setup, the following example shows a basic initialization use case:

Java

class AwesomeVideoMeetingDisconnectHandler implements MeetingDisconnectHandler {
};

class AwesomeVideoCoDoingSessionDelegate implements CoDoingSessionDelegate {
  // For example implementation, see the "Manage current state" section below.
};

public ListenableFuture<CoDoingSession> initialSetup() {
  LiveSharingClient meetClient = LiveSharingClientFactory.getClient();

  ListenableFuture<LiveSharingMeetingInfo> meetingInfoFuture =
      meetClient.connectMeeting(
          appContext,
          /* liveSharingApplicationName= */ "AwesomeVideoApp",
          new AwesomeVideoMeetingDisconnectHandler());

  return Futures.transformAsync(
      meetingInfoFuture,
      (LiveSharingMeetingInfo meetingInfo) -> {
        this.meetingInfo = meetingInfo;
        return meetClient.beginCoDoing(new AwesomeVideoCoDoingSessionDelegate());
      },
      executor);
}

Manage current state

At various points during live sharing, Meet might need to retrieve the current state of the local activity. It does so by calling the CoDoingSessionDelegate.onCoDoingStateQuery() callback. This might be called regularly, so it should be written to be performant (for example, <100 ms).

The following example shows an implementation of the CoDoingSessionDelegate:

Java

class AwesomeVideoState {
  public String videoUrl;
  public Duration videoTimestamp;
  public bool isPaused;
}

class AwesomeVideoCoDoingSessionDelegate implements CoDoingSessionDelegate {
  public CoDoingState onCoDoingStateQuery() {
    AwesomeVideoState videoState = new AwesomeVideoState();
    videoState.videoUrl = this.currentVideo.Url;
    videoState.videoTimestamp = this.currentVideo.Timestamp;
    videoState.isPaused = this.currentVideo.IsPaused;

    CoDoingState coDoingState = new CoDoingState();
    coDoingState.state = SerializationUtils.serialize(videoState);

    return coDoingState;
  }
};

Pause video

When participating in a live sharing experience, if a user pauses the playback on their local video app then presumably you'll likely want to make sure all participants in the live sharing experience also pause their video.

To do this, craft a CoDoingState message showing the video is paused, and tell Meet to broadcast it all other live sharing participants:

Java

public void onVideoPaused(String videoUrl, Instant currentTimestamp) {
  // Create an internal state object to share with other participants. Note: it's
  // good practice to encode all metadata - even seemingly irrelevant data - into
  // ActivityState updates to guard against race conditions and other subtle
  // failures.
  AwesomeVideoState videoState = new AwesomeVideoState();
  videoState.videoUrl = videoUrl;
  videoState.videoTimestamp = currentTimestamp;
  videoState.isPaused = true;

  // Create the CoDoingState object to wrap the internal state
  CoDoingState coDoingState = new CoDoingState();
  coDoingState.state = SerializationUtils.serialize(videoState);

  // Use Meet to broadcast internal state update to all other participants
  this.meetCoDoing.broadcastStateUpdate(coDoingState);
};

The above code triggers the serialized VideoState object to be broadcast to all other instances of Meet currently participating in the live sharing experience. For details on how to receive broadcast updates from other participants, see the Handle incoming updates section below.

The following sequence diagram describes the sequence of events after the pause action is triggered:

Unpause video

Similar to pause, if a user unpauses the video on their local app Meet needs to broadcast this operation to other live sharing participants.

On the sender side (the user who unpauses the video) the only difference from the pause example is the isPaused status is updated. The sender side code should look like this:

Java

public void onVideoUnpaused(String videoUrl, Instant currentTimestamp) {
  AwesomeVideoState videoState = new AwesomeVideoState();
  videoState.videoUrl = videoUrl;
  videoState.videoTimestamp = currentTimestamp;
  videoState.isPaused = false;

  CoDoingState coDoingState = new CoDoingState();
  coDoingState.state = SerializationUtils.serialize(videoState);

  this.meetCoDoing.broadcastStateUpdate(coDoingState);
}

Seek video

When a user drags the timeline on the local app to a new timestamp, we want to broadcast this operation to all participants similar to the pause and unpause sections. The sender side code looks like this:

Java

public void onVideoSeeked(String videoUrl, Instant currentTimestamp, bool isPaused) {
  AwesomeVideoState videoState = new AwesomeVideoState();
  videoState.videoUrl = videoUrl;
  videoState.videoTimestamp = currentTimestamp;
  videoState.isPaused = isPaused;

  CoDoingState coDoingState = new CoDoingState();
  coDoingState.state = SerializationUtils.serialize(videoState);

  this.meetCoDoing.broadcastStateUpdate(coDoingState);
}

Play a different video

If the user also changes the video being watched by selecting another video on the local app, we need to play the new video for all live sharing participants. The change in video is held in VideoState.videoUrl:

Java

public void onVideoChanged(String videoUrl, Duration currentTimestamp,
                           bool isPaused) {
  AwesomeVideoState videoState = new AwesomeVideoState();
  videoState.videoUrl = videoUrl;
  videoState.videoTimestamp = currentTimestamp;
  videoState.isPaused = isPaused;

  CoDoingState coDoingState = new CoDoingState();
  coDoingState.state = SerializationUtils.serialize(videoState);

  this.meetCoDoing.broadcastStateUpdate(coDoingState);
}

End co-watching

When the user selects “end co-watching,” we need to end their live sharing experience and possibly remove them from the meeting.

After the library is called to end the meeting, the live sharing app can quit or go back to its homepage (depending on the product’s preference).

Java

public void endCoWatching() {
  this.meetClient.disconnectMeeting();
}

Handle incoming updates

When another participant’s Meet app receives a broadcast, the onCoDoingStateChanged() callback is triggered. Usually, it's important to make good decisions on what action to take in response to incoming updates such as only matching incoming video timestamps if they're sufficiently different from the local timestamp.

Java

class AwesomeVideoCoDoingSessionDelegate implements CoDoingSessionDelegate {
  public void onCoDoingStateChanged(CoDoingState update) {
    AwesomeVideoState videoState = SerializationUtils.deserialize(update.state());

    // Handle transition to new video.
    if (videoState.videoUrl != this.videoPlayer.videoUrl) {
      this.videoPlayer.loadVideo(videoState.videoUrl);
    }

    // If the timestamp in the arriving update has sufficiently diverged, adjust
    // the local video playout.
    if (videoState.videoTimestamp.minus(this.videoPlayer.videoTimestamp).abs() >
                                        Duration.ofSeconds(2)) {
      this.videoPlayer.seek(videoState.videoTimestamp);
    }

    // Update pause state if necessary.
    if (!videoState.isPaused && this.videoPlayer.isPaused) {
      this.videoPlayer.unpause();
    } else if (videoState.isPaused && !this.videoPlayer.isPaused) {
      this.videoPlayer.pause();
    }
  }
}