This article was written and submitted by an external developer. The Google App Engine team thanks Andi Albrecht for his time and expertise.
NOTE (Nov 2010): The App Engine team recommends readers also consider an alternative tool called Django-nonrel, a (currently) maintained fork of the latest version of Django which allows developers to run native Django applications (via Django's ORM) on traditional SQL databases as well as non-relational datastores (including App Engine's). (This differs from the approach in this article which attempts to port various App Engine APIs to Django.) For more information on using Django-nonrel with App Engine, please see our Django-nonrel article which shows you how to convert a native App Engine webapp app to a pure Django app.
In this article we explain how to port applications that were originally designed for App Engine with Django to any environment that supports Django. The helper application provides a Django based implementation of the App Engine APIs and its aim is to minimize the effort of porting your App Engine application to run anywhere using Django.
After a short instruction on how to obtain the
gae2django helper application, we demonstrate using the helper so that Rietveld, the code review tool written by Guido van Rossum, runs on any environment that supports Django.
Finally, we talk through how to use the helper with your own App Engine Django application, and give some notes about required changes and potential pitfalls.
To follow the examples in this article, we assume you are using the command line and have a Mercurial client installed on your system.
Obtaining the Helper
The project is hosted on Google project hosting, which uses Mercurial. To obtain the
gae2django helper run
hg clone http://code.google.com/p/django-gae2django django-gae2django from the command line to fetch the latest version from the project's repository.
django-gae2django directory contains a Django project structure along with the helper. You should use this same directory structure when including the helper with your existing Google App Engine applications.
The Rietveld Example
In addition to the helper, there's an
examples directory with everything prepared to convert Rietveld to pure Django. To learn more about Rietveld visit its project page at http://code.google.com/p/rietveld/ or see it in action at http://codereview.appspot.com.
The core of Rietveld is a Django application called
codereview. Rietveld was built originally for App Engine and uses Django's URL resolving and template rendering functionality.
Since Rietveld makes heavy use of the Google App Engine APIs, it will not run by default on pure Django. And maintaining two separate versions of the application would also require the added problem of keeping the features of both versions in sync. Instead, we'll use gae2django to allow the code originally written for App Engine to run in a pure Django environment.
To get started with the Rietveld example, change into the
examples/rietveld directory and
make all (refer to the
README file in that directory for manual setup instructions).
make all does three things:
- Fetches the entire Rietveld application from its repository
- Applies the patches required to get it running on Django
./manage.py syncdbto setup the database used in this example
During the final step, Django will ask if you would like to create a superuser account. Answer "yes" here and give the required information. You can create additional users later by following the steps outlined in User authentication in Django.
After you have installed Rietveld, run
./manage.py runserver localhost:8000 to start the server. Point your browser to
http://localhost:8000 to use Rietveld running on pure Django.
In the next sections, we step through all of the changes the helper made to the original
codereview application, but if you want to have a look at the changes first, why not use the code review tool?
To upload the changes made by the helper, run the following command in the
../static/upload.py -s 127.0.0.1:8000 -m "codereview on Django" Upload server: 127.0.0.1:8000 (change with -s/--server) Login URL: 'http://127.0.0.1:8000/accounts/login/' Username: USERNAME_AS_PROVIDED_ABOVE Password: Issue created. URL: http://127.0.0.1:8000/1 Uploading views.py Uploading library.py
Use the username and password of the superuser created when setting up Rietveld. The above command uploads the original and modified files to your local instance of Rietveld. Loading the "Recent Issues" page will show the change as the first issue.
Next, we explain how the helper works and how to use it for your own application.
Installing the Helper in Your Own Project
To install the
gae2django helper in your own project, open your
manage.py script and add the following at the top before any Django imports:
import gae2django gae2django.install()
gae2django under the
INSTALLED_APPS setting. Add
MIDDLEWARE_CLASSES directly after Django's authentication middleware.
The App Engine API replacement is exposed as the
./manage.py shell >>> from google.appengine.api import users >>> users.create_login_url("foo") '/accounts/login/?next=foo'
gae2django is installed, calls to App Engine's Users API are mapped to Django's authentication system, App Engine models and properties are mapped to Django models and fields, GQL queries are translated to model filters, and calls to the Mail API are handled by Django's mail module.
Changes to Your Application
If you are following along, you've probably noticed the
patches directory and the
rietveld_helper application. These contain modifications to the original application, and are required to run Rietveld using pure Django. Not all differences between App Engine and pure Django's are automatically covered by using the helper.
Depending on your use the App Engine APIs, a few additional changes to your original App Engine application might be required to run on pure Django. The following sections give you some hints about the differences between both frameworks and how these issues are resolved for Rietveld. This should allow you to implement similar changes for your own application.
The User Class
Semantically, the most significant difference between pure Django and App Engine involve the User's API. With Django's
User model, email is a field, but in the App Engine Users API the email is retrieved using a method. Also, Django doesn't have the
nickname method which is available in App Engine. When gae2django translates Rietveld, every call to
user.email() in Rietveld is replaced by
user.email and the
user.nickname() is set from the profile class.
Another difference between the App Engine Users API and Django's behavior is how anonymous users are handled. If the current user isn't logged in on App Engine
None. Django has an
AnonymousUser instance. With Rietveld we solve this by adding another middleware class in
rietveld_helper. This class removes the user that is added to the
request instance when the user is anonymous. Since the Rietveld only checks
request.user and constructs a
User object from scratch, this is all we need to do.
You may need to implement additional solutions for this issue with your own application, such as replacing conditional statements like
user is None with
Models and Properties
The Datastore API implementation of the helper is fairly robust. For the Rietveld example no changes to the original models or GQL queries were made.
gae2django implementation of
google.appengine.ext.db.Model creates additional model fields (prefixed with
gae_*) to store a key for an instance and, if necessary, to store an ancestry path to resolve parent relationships.
GQL queries are analyzed and translated into Django field lookups. The returned
QuerySets work almost exactly the same as result sets in App Engine.
Application Specific Changes
The changes to
upload.py are specific to the code review application.
upload.py is a command line script to upload changes in a version controlled working tree to the review server and it requires authentication on the server from the command line. The modified version of this script in the
static directory uses Django's login form to retrieve an authentication cookie from the Django server. Have a look at
patches/upload.diff if you are interested in this more advanced topic.
In addition to the middleware class for the Users API,
rietveld_helper provides the
ROOT_URLCONF for Django. The URLs for Rietveld are imported in
rietveld_helper/urls.py and are extended by additional, Django specific, URLs for sign in, sign out, and for serving static files.
gae2django helper application is still in an early stage of development and not all parts of Google's App Engine APIs are currently covered by this project. At the moment the project focuses on Rietveld as an reference implementation. Upcoming development will include greater coverage of the App Engine APIs, unit tests, and other example applications. If you know an App Engine based application suitable for demonstrating and testing the helper's functionality, read the section about contributing to this project below.
Please file bug reports and feature requests on the project site at http://code.google.com/p/django-gae2django/issues/entry or if you have code to contribute upload it to http://codereview.appspot.com and add
email@example.com as reviewer.
gae2django helper application can be used to simplify the porting of applications designed for Google App Engine to pure Django projects. It simplifies the reuse of the application code as it provides Django adapters for most of App Engine's methods and classes. It reduces the effort porting an application to Django by freeing the developer to deal only with cases when the frameworks differ and application specific issues.