Google Drive SDK

Example Drive App: DrEdit for Python

DrEdit is a sample Google Drive app written in Python. It is a text editor capable of editing files with the MIME types text/* and extensions such as .txt, .html etc. that are stored in a user's Google Drive. This article describes the complete application to help you in your integrations.

Setting up this sample application

Setting up DrEdit requires using Google App Engine and performing some configuration. Follow the instructions in the repository's README to set up the application.

Architecture

DrEdit is a web application written on Google App Engine in Python. With two types of endpoints:

  • User interface at /

    The user interface is rendered including HTML, JavaScript, and CSS.

  • Ajax services to communicate with Drive at /svc, /about and /user

    Drive API requests are made in response to GET, PUT, and POST requests and return JSON data.

DrEdit implements two Google Drive use cases:

  1. Create a new file from the Google Drive UI
  2. Open a file from the Google Drive UI

The flow for both use cases is similar. A user is redirected to DrEdit after selecting DrEdit from the Create menu or context menu of a file with a registered MIME type.

This redirect takes place with two important parts of data as query parameters:

  1. An authorization code, capable of being exchanged for an access token to access the user's data.
  2. A state object describing the user's action, i.e. which file(s) and which action (open, create) to perform.

DrEdit responds by performing the required action on the passed files using the authorization credentials. The authorization credentials are stored in a session for future use by Ajax requests from the user interface.

Authorization

In order for DrEdit to be able to make calls to the Drive API, an authorized API client must be created. The first requirement, using the Google API Python Client is to get the credentials of the user to authorize the client.

These credentials can be loaded from:

  1. A code passed from the Google Drive UI
  2. The user's session

The choice of credential source is determined by the action that DrEdit is performing. In cases where the user has been redirected from the Drive user interface, an authorization code is always provided and DrEdit always uses it. In cases where the user is making Ajax calls from the user interface, the credentials are loaded from a session.

Setting up the client ID, client secret, and other OAuth 2.0 parameters

Google Drive applications need these three values for authorization:

  1. Client ID
  2. Client secret
  3. List of valid redirect URIs

The client ID and client secret for an application are created when an application is registered in the Google Developers Console and the OAuth 2.0 client IDs are generated.

To register a new application, do the following:

  1. Go to the Google Developers Console.
  2. Select a project, or create a new one.
  3. In the sidebar on the left, select APIs & auth. In the list of APIs, make sure the status is ON for the Drive API.
  4. In the sidebar on the left, select Registered apps.
  5. At the top of the page, select Register App.
  6. Fill out the form and select Register.

If you have already registered your application, do the following to retrieve your client ID and secret:

  1. Go to the Google Developers Console.
  2. Select a project.
  3. In the sidebar on the left, select APIs & auth. In the list of APIs, make sure the status is ON for the Drive API.
  4. In the sidebar on the left, select Credentials.

In accordance with the the Client Secrets format proposed by the Google API Python library, DrEdit stores these settings in the file client_secrets.json. This file must be modified to reflect an application's own client ID, client secret, and list of valid redirect URIs. Developers normally do not need to change the auth_uri or token_uri parameters.

{
  "web": {
    "client_id": "abc123456789.apps.googleusercontent.com",
    "client_secret": "def987654321",
    "redirect_uris": ["https://my-dredit-instance.appspot.com"],
    "auth_uri": "https://accounts.google.com/o/oauth2/auth",
    "token_uri": "https://accounts.google.com/o/oauth2/token"
  }
}

Authorizing a code passed from the Google Drive UI

After a user chooses DrEdit to open or create a file, the user's browser is redirected to DrEdit's registered redirect URI (set in the Google Developers Console), with an OAuth 2.0 authorization code attached as the code query parameter in the URI. The given authorization code is scoped specifically to open the single file (Open With) or to create any file (Create New), and must be exchanged for an access token with the OAuth 2.0 web server flow.

Once this token is obtained, it is used to fetch the user profile information from the Userinfo service. This profile information, along with the Credentials instance are stored in the session for later use.

def GetCodeCredentials(self):
  """Create OAuth 2.0 credentials by extracting a code and performing OAuth2.0.

  The authorization code is extracted form the URI parameters. If it is absent,
  None is returned immediately. Otherwise, if it is present, it is used to
  perform step 2 of the OAuth 2.0 web server flow.

  Once a token is received, the user information is fetched from the userinfo
  service and stored in the session. The token is saved in the datastore against
  the user ID received from the userinfo service.

  Args:
    request: HTTP request used for extracting an authorization code and the
             session information.
  Returns:
    OAuth2.0 credentials suitable for authorizing clients or None if
    Authorization could not take place.
  """
  # Other frameworks use different API to get a query parameter.
  code = self.request.get('code')
  if not code:
    # returns None to indicate that no code was passed from Google Drive.
    return None

  # Auth flow is a controller that is loaded with the client information,
  # including client_id, client_secret, redirect_uri etc
  oauth_flow = self.CreateOAuthFlow()

  # Perform the exchange of the code. If there is a failure with exchanging
  # the code, return None.
  try:
    creds = oauth_flow.step2_exchange(code)
  except FlowExchangeError:
    return None

  # Create an API service that can use the userinfo API. Authorize it with our
  # credentials that we gained from the code exchange.
  users_service = CreateService('oauth2', 'v2', creds)

  # Make a call against the userinfo service to retrieve the user's information.
  # In this case we are interested in the user's "id" field.
  userid = users_service.userinfo().get().execute().get('id')

  # Store the user id in the user's cookie-based session.
  session = sessions.LilCookies(self, SESSION_SECRET)
  session.set_secure_cookie(name='userid', value=userid)

  # Store the credentials in the data store using the userid as the key.
  StorageByKeyName(Credentials, userid, 'credentials').put(creds)
  return creds

Loading credentials from the session

Once the user interface has loaded, it communicates with DrEdit over the /svc endpoint. These requests use Credentials that were earlier stored against the session. The userid is taken from the session, and used to load the Credentials instance from the data store.

def GetSessionCredentials(self):
  """Get OAuth 2.0 credentials for an HTTP session.

  If the user has a user id stored in their cookie session, extract that value
  and use it to load that user's credentials from the data store.

  Args:
    request: HTTP request to use session from.
  Returns:
    OAuth2.0 credentials suitable for authorizing clients.
  """
  # Try to load  the user id from the session
  session = sessions.LilCookies(self, SESSION_SECRET)
  userid = session.get_secure_cookie(name='userid')
  if not userid:
    # return None to indicate that no credentials could be loaded from the
    # session.
    return None

  # Load the credentials from the data store, using the userid as a key.
  creds = StorageByKeyName(Credentials, userid, 'credentials').get()

  # if the credentials are invalid, return None to indicate that the credentials
  # cannot be used.
  if creds and creds.invalid:
    return None

  return creds

Failing to authorize

There are a number of points at which the authorization process can fail. In all these cases, the user is redirected to the OAuth 2.0 authorization endpoint so that they can reauthorize and return to DrEdit.

ALL_SCOPES = (
    'https://www.googleapis.com/auth/drive.file '
    'https://www.googleapis.com/auth/userinfo.email '
    'https://www.googleapis.com/auth/userinfo.profile'
)

def RedirectAuth(self):
  """Redirect a handler to an authorization page.

  Used when a handler fails to fetch credentials suitable for making Drive API
  requests. The request is redirected to an OAuth 2.0 authorization approval
  page and on approval, are returned to application.

  Args:
    handler: webapp.RequestHandler to redirect.
  """
  flow = self.CreateOAuthFlow()

  # Manually add the required scopes. Since this redirect does not originate
  # from the Google Drive UI, which authomatically sets the scopes that are
  # listed in the Developers Console.
  flow.scope = ALL_SCOPES

  # Create the redirect URI by performing step 1 of the OAuth 2.0 web server
  # flow.
  uri = flow.step1_get_authorize_url(flow.redirect_uri)

  # Perform the redirect.
  self.redirect(uri)

Making authorized API requests

Once the access token has been retrieved, it is used to authorize the HTTP client that will be used for making API requests. The HTTP client is used to create an API client, along with the discovery document for the required service.

def CreateService(service, version, creds):
  """Create a Google API service.

  Load an API service from the Discovery API and authorize it with the
  provided credentials.

  Args:
    service: Request service (e.g 'drive', 'oauth2').
    version: Version of the service (e.g 'v2').
    creds: Credentials used to authorize service.
  Returns:
    Authorized Google API service.
  """
  # Instantiate an Http instance
  http = httplib2.Http()

  # Authorize the Http instance with the passed credentials
  creds.authorize(http)

  # Build a named service from the Discovery API
  return build(service, version, http=http)

Creating an Authorization flow instance

All the above methods use an OAuth 2.0 flow instance which behaves as a controller for performing Authorization actions. This is provided by the Python client library, and placed into a convenience method.

def CreateOAuthFlow(self):
  """Create OAuth2.0 flow controller

  This controller can be used to perform all parts of the OAuth 2.0 dance
  including exchanging an Authorization code.

  Args:
    request: HTTP request to create OAuth2.0 flow for
  Returns:
    OAuth2.0 Flow instance suitable for performing OAuth2.0.
  """
  flow = flow_from_clientsecrets('client-debug.json', scope='')
  # Dynamically set the redirect_uri based on the request URL. This is extremely
  # convenient for debugging to an alternative host without manually setting the
  # redirect URI.
  flow.redirect_uri = self.request.url.split('?', 1)[0].rsplit('/', 1)[0]
  return flow

Handling HTTP requests

This section describes how DrEdit handles HTTP requests for operations such as creating and opening new files and rendering the user interface.

Requests to the user interface

The user interface is provided as a single HTML template which is used for both opening (Open with) and creating new files. Once loaded, the user interface loads any data required.

The text editing component uses the Ace Editor, a text-editing component with syntax highlighting and configurable key bindings. It is used in DrEdit as a form field with advanced editing features.

Users may arrive at the / endpoint from two separate sources:

  1. Drive User interface, along with Drive State
  2. By directly accessing the URL, with no Drive State

Loading the Drive state

DrEdit contains a helper class for parsing and loading the Drive state. The state parameter is a JSON string containing two important properties:

  1. The action to be performed, create or open in the action field
  2. The file IDs (if any) to perform the action on in the ids field

DrEdit uses this state to determine how it should behave. If file ids are provided they are loaded immediately after the user interface is loaded.

class DriveState(object):
  """Store state provided by Drive."""

  def __init__(self, state):
    """Create a new instance of drive state.

    Parse and load the JSON state parameter.

    Args:
      state: State query parameter as a string.
    """
    state_data = json.loads(state)
    self.action = state_data['action']
    self.ids = map(str, state_data.get('ids', []))

  @classmethod
  def FromRequest(cls, request):
    """Create a Drive State instance from an HTTP request.

    Args:
      cls: Type this class method is called against.
      request: HTTP request.
    """
    return DriveState(request.get('state'))

Rendering the user interface

The user interface is rendered in response to GET requests. The Drive state is loaded and the template is rendered. These is no communication with the Drive API at this stage.

def get(self):
  """Handle GET for Create New and Open With.

  This creates an authorized client, and checks whether a resource id has
  been passed or not. If a resource ID has been passed, this is the Open
  With use-case, otherwise it is the Create New use-case.
  """
  # Generate a state instance for the request, this includes the action, and
  # the file id(s) that have been sent from the Drive user interface.
  drive_state = DriveState.FromRequest(self.request)
  if drive_state.action == 'open' and len(drive_state.ids) > 0:
    code = self.request.get('code')
    if code:
      code = '?code=%s' % code
    self.redirect('/#edit/%s%s' % (drive_state.ids[0], code))
    return
  # Fetch the credentials by extracting an OAuth 2.0 authorization code from
  # the request URL. If the code is not present, redirect to the OAuth 2.0
  # authorization URL.
  creds = self.GetCodeCredentials()
  if not creds:
    return self.RedirectAuth()

  # Extract the numerical portion of the client_id from the stored value in
  # the OAuth flow. You could also store this value as a separate variable
  # somewhere.
  client_id = self.CreateOAuthFlow().client_id.split('.')[0].split('-')[0]
  self.RenderTemplate()

def RenderTemplate(self):
  """Render a named template in a context."""
  self.response.headers['Content-Type'] = 'text/html'
  self.response.out.write(INDEX_HTML)

Requests to the Files service

The /svc endpoint allows three HTTP methods.

  • GET: Fetches file metadata and content from Google Drive.
  • POST: Creates new files in Google Drive.
  • PUT: Updates existing files in Google Drive.

Data is returned as JSON with the application/json content-type and when data is required (in POST and PUT requests) it is provided from the user interface as JSON with the application/json content-type.

Fetching file metadata and content from Google Drive

The file is retrieved in two steps. First, the metadata is fetched using the files.get method, passing the file_id that was sent in the initial redirect. Second, the file content is fetched by making a simple authorized GET request to the file's downloadUrl.

def get(self):
  """Called when HTTP GET requests are received by the web application.

  Use the query parameter file_id to fetch the required file's metadata then
  content and return it as a JSON object.

  Since DrEdit deals with text files, it is safe to dump the content directly
  into JSON, but this is not the case with binary files, where something like
  Base64 encoding is more appropriate.
  """
  # Create a Drive service
  service = self.CreateDrive()
  if service is None:
    return
  try:
    # Requests are expected to pass the file_id query parameter.
    file_id = self.request.get('file_id')
    if file_id:
      # Fetch the file metadata by making the service.files().get method of
      # the Drive API.
      f = service.files().get(fileId=file_id).execute()
      downloadUrl = f.get('downloadUrl')
      # If a download URL is provided in the file metadata, use it to make an
      # authorized request to fetch the file ontent. Set this content in the
      # data to return as the 'content' field. If there is no downloadUrl,
      # just set empty content.
      if downloadUrl:
        resp, f['content'] = service._http.request(downloadUrl)
      else:
        f['content'] = ''
    else:
      f = None
    # Generate a JSON response with the file data and return to the client.
    self.RespondJSON(f)
  except AccessTokenRefreshError:
    # Catch AccessTokenRefreshError which occurs when the API client library
    # fails to refresh a token. This occurs, for example, when a refresh token
    # is revoked. When this happens the user is redirected to the
    # Authorization URL.
    self.RedirectAuth()

Saving files

When the user clicks Save from the DrEdit user interface, an AJAX request is made from the user interface to the server. This request is either:

  • POST: indicating that a new file is to be created.
  • PUT: indicating that an existing file is to be updated, or

These methods match standard REST semantics, and are convenient because both can be represented as a single URL which handles different HTTP methods.

The data is sent as JSON containing the following fields:

  • title: Title of the file.
  • description: Description of the file.
  • content: Content of the file, i.e., the content from the text editor.
  • file_id: File ID of the file, or an empty string if this is a new file.

Both PUT and POST methods return a JSON response that contains the file ID of the saved or created file. The file ID is stored in the user interface, and is sent along with future requests.

Saving new files

The POST method is used to save newly created files. A files.insert call is made to create the new file and set the metadata.

def post(self):
  """Called when HTTP POST requests are received by the web application.

  The POST body is JSON which is deserialized and used as values to create a
  new file in Drive. The authorization access token for this action is
  retreived from the data store.
  """
  # Create a Drive service
  service = self.CreateDrive()
  if service is None:
    return

  # Load the data that has been posted as JSON
  data = self.RequestJSON()

  # Create a new file data structure.
  resource = {
    'title': data['title'],
    'description': data['description'],
    'mimeType': data['mimeType'],
  }
  try:
    # Make an insert request to create a new file. A MediaInMemoryUpload
    # instance is used to upload the file body.
    resource = service.files().insert(
        body=resource,
        media_body=MediaInMemoryUpload(
            data.get('content', ''),
            data['mimeType'],
            resumable=True)
    ).execute()
    # Respond with the new file id as JSON.
    self.RespondJSON(resource['id'])
  except AccessTokenRefreshError:
    # In cases where the access token has expired and cannot be refreshed
    # (e.g. manual token revoking) redirect the user to the authorization page
    # to authorize.
    self.RedirectAuth()

Updating files

The process for updating a file is similar to that of creating one. File updates are made using an HTTP PUT instead of an HTTP POST.

Unlike when creating files, the files.update method of the API is used, and this requires the file ID to be passed as the id parameter.

Additionally, a newRevision parameter is passed as a boolean, to indicate whether the update should be marked as a new revision in the Drive user interface.

def put(self):
  """Called when HTTP PUT requests are received by the web application.

  The PUT body is JSON which is deserialized and used as values to update
  a file in Drive. The authorization access token for this action is
  retreived from the data store.
  """
  # Create a Drive service
  service = self.CreateDrive()
  if service is None:
    return
  # Load the data that has been posted as JSON
  data = self.RequestJSON()
  try:
    # Create a new file data structure.
    content = data.get('content')
    if 'content' in data:
      data.pop('content')
    if content is not None:
      # Make an update request to update the file. A MediaInMemoryUpload
      # instance is used to upload the file body. Because of a limitation, this
      # request must be made in two parts, the first to update the metadata, and
      # the second to update the body.
      resource = service.files().update(
          fileId=data['resource_id'],
          newRevision=self.request.get('newRevision', False),
          body=data,
          media_body=MediaInMemoryUpload(
              content, data['mimeType'], resumable=True)
          ).execute()
    else:
      # Only update the metadata, a patch request is prefered but not yet
      # supported on Google App Engine; see
      # http://code.google.com/p/googleappengine/issues/detail?id=6316.
      resource = service.files().update(
          fileId=data['resource_id'],
          newRevision=self.request.get('newRevision', False),
          body=data).execute()
    # Respond with the new file id as JSON.
    self.RespondJSON(resource['id'])
  except AccessTokenRefreshError:
    # In cases where the access token has expired and cannot be refreshed
    # (e.g. manual token revoking) redirect the user to the authorization page
    # to authorize.
    self.RedirectAuth()

Uploading content

DrEdit uses a helper class to assist with media uploads. Since normal media uploads used by the Python client library are performed from files on disk, the MediaUpload interface has been implemented specifically to provide uploads of a file as a string. The interface is simple and the implementation is presented here.

class MediaInMemoryUpload(MediaUpload):
  """MediaUpload for a chunk of bytes.

  Construct a MediaFileUpload and pass as the media_body parameter of the
  method. For example, if we had a service that allowed plain text:
  """

  def __init__(self, body, mimetype='application/octet-stream',
               chunksize=256*1024, resumable=False):
    """Create a new MediaBytesUpload.

    Args:
      body: string, Bytes of body content.
      mimetype: string, Mime-type of the file or default of
        'application/octet-stream'.
      chunksize: int, File will be uploaded in chunks of this many bytes. Only
        used if resumable=True.
      resumable: bool, True if this is a resumable upload. False means upload
        in a single request.
    """
    self._body = body
    self._mimetype = mimetype
    self._resumable = resumable
    self._chunksize = chunksize

  def chunksize(self):
    """Chunk size for resumable uploads.

    Returns:
      Chunk size in bytes.
    """
    return self._chunksize

  def mimetype(self):
    """Mime type of the body.

    Returns:
      Mime type.
    """
    return self._mimetype

  def size(self):
    """Size of upload.

    Returns:
      Size of the body.
    """
    return len(self._body)

  def resumable(self):
    """Whether this upload is resumable.

    Returns:
      True if resumable upload or False.
    """
    return self._resumable

  def getbytes(self, begin, length):
    """Get bytes from the media.

    Args:
      begin: int, offset from beginning of file.
      length: int, number of bytes to read, starting at begin.

    Returns:
      A string of bytes read. May be shorter than length if EOF was reached
      first.
    """
    return self._body[begin:begin + length]

Responding with the file ID

Both creating new files and updating existing files returns a JSON response containing the file ID of the file that has been created or modified. The user interface can use this file ID to ensure that future saves are to the same file, and that a new file is not created each time.

def RespondJSON(self, data):
  """Generate a JSON response and return it to the client.

  Args:
    data: The data that will be converted to JSON to return.
  """
  # Set the content type to JSON's
  self.response.headers['Content-Type'] = 'application/json'
  # Write the response body as JSON
  self.response.out.write(json.dumps(data))

Personalizing the user interface

Additional services are provided to customize the user interface for the authorized user. This has the benefit of providing a familiar and personalized experience for the user.

About service

The /about endpoint allows one HTTP method.

  • GET: Fetches information about the authorized user and their account.

This service returns information about the currently authorized user and settings for their account. This service fetches and returns the JSON resource representation described in about.get.

def get(self):
  """Called when HTTP GET requests are received by the web application."""
  # Create a Drive service
  service = self.CreateDrive()
  if service is None:
    return
  try:
    result = service.about().get().execute()
    # Generate a JSON response with the file data and return to the client.
    self.RespondJSON(result)
  except AccessTokenRefreshError:
    # Catch AccessTokenRefreshError which occurs when the API client library
    # fails to refresh a token. This occurs, for example, when a refresh token
    # is revoked. When this happens the user is redirected to the
    # Authorization URL.
    self.RedirectAuth()

User service

The /user endpoint allows one HTTP method.

  • GET: Fetches information about the user

This service returns information about the user that was cached during authorization.

def get(self):
  """Called when HTTP GET requests are received by the web application."""
  # Create a Drive service
  service = self.CreateUserInfo()
  if service is None:
    return
  try:
    result = service.userinfo().get().execute()
    # Generate a JSON response with the file data and return to the client.
    self.RespondJSON(result)
  except AccessTokenRefreshError:
    # Catch AccessTokenRefreshError which occurs when the API client library
    # fails to refresh a token. This occurs, for example, when a refresh token
    # is revoked. When this happens the user is redirected to the
    # Authorization URL.
    self.RedirectAuth()

Installation notes

Create a secret key

python -c "import os; print os.urandom(64)" > session.secret

Additional resources

Authentication required

You need to be signed in with Google+ to do that.

Signing you in...

Google Developers needs your permission to do that.