Google App Engine

Google App Engine for Java Questions

Runtime Environment

Performance

Tools

Python and Java

Common Issues


Runtime Environment

Can I use my favorite framework on App Engine?

The App Engine runtime environment imposes some constraints to ensure that your app can be scaled to multiple instances on App Engine's distributed infrastructure. Many frameworks will work seamlessly within the App Engine server runtime environment but some of them will not. Others may require some modification.

There are quite a few frameworks available, so we're using a community oriented approach with a collaborative page here: Will it Play in App Engine Please search this page to check on the status of your favorite framework, and write a comment if your framework is not listed or if you have updates or fixes.

Which JVM languages can I use in my App Engine app?

Many of the languages designed to run in the JVM (like JRuby, Jython, Scala, and Groovy just to name a few) can be used on App Engine. At this point not all of these languages have been fully tested, so we're using a community oriented approach with a collaborative page here: Will it Play in App Engine. Please add information as you try out the runtime language of your choice.

Why can't I read from this file?

It is possible to read from a file which is uploaded as part of your application provided that it is in the following locations:

  • war/WEB-INF
  • in a location matching the <resource-files> pattern in appengine-web.xml (which by default includes everything)

If the file location is not the issue, the problem may be that the method you are using to read from the file is not whitelisted. Your application can use any IO classes that are useful for reading from the file system, such as File, FileInputStream, FileReader, or RandomAccessFile. For a full list of whitelisted classes, please see the JRE Class White List.

If you need to get file access to your own resources (such as properties files), you could put these files inside of jars and use Class or ClassLoader to load them.

Why can't I write to this file?

Writing to local files is not supported in App Engine due to the distributed nature of your application. Instead, data which must be persisted should be stored in the distributed datastore. For more information see the documentation on the runtime sandbox.

Can I create new threads in my app?

No, an App Engine app may not spawn new threads. For more information see the documentation on the runtime sandbox.

Does App Engine support Java Enterprise Edition?

While we do not support the entirety of the Java EE specification, we support many of its individual components. For a specific list of which components are and are not supported, see our collaboratively edit page for tracking framework, language, and library compatibility: Will it Play in App Engine.

Performance

What is a loading request?

Some requests run slower because App Engine needs to create a new Java Virtual Machine (JVM) to service the request. That kind of request is called a Loading Request. During a loading request, your application undergoes initialization (such as class loading, JIT compiling, etc.) which causes the request to take longer.

Requests have an approximately 60-second deadline, which includes initialization and any additional work your application needs to perform on startup. If your app exceeds this limit, a DeadlineExceededException is returned.

You can find these types of requests in your request logs by filtering on loading_request=1

What causes loading requests?

App Engine spins up JVMs on demand, so there are several reasons why you may receive a loading request:

  1. You just uploaded a new version of your application.
  2. If your application has no traffic, the first request always results in a loading request.
  3. Your traffic has increased, which has caused a new instance to initialize.

You can expect that during the course of developing your application, you will often experience the first two scenarios. In comparison, for a production app receiving even a very small but steady amount of traffic, loading requests are relatively infrequent.

How do I distinguish normal requests from loading requests in my application logs?

You can register an HttpSessionListener in your web.xml which logs from its sessionCreated method. For example:

// web.xml snippet
<listener>
  <listener-class>
  com.example.LogLoadingRequest
  </listener-class>
</listener>

// LogLoadingRequest.java
public class LogLoadingRequest implements ServletContextListener {
  private static final Logger logger = Logger.getLogger(LogLoadingRequest.class.getName());
  public void contextInitialized(ServletContextEvent sce) {
    logger.log(Level.INFO, "Loading request occuring.");
  }

  public void contextDestroyed(ServletContextEvent sce) {
  }
}

In the future, the Admin Console logs viewer will mark loading requests specifically so that they can be easily identified.

How do I avoid loading requests?

You can avoid some loading requests by using warmup requests. Warmup requests load application code on a new instance before any live requests are made. To enable warm up requests, please see Enabling Warmup Requests (Java | Python).

Note: Even with warmup requests enabled, they may not be used for all requests. In some instances, the App Engine scheduler will determine that it is simply faster for the user's request to be sent to the new instance directly rather than holding the user's request while a warmup request is issued. This is most common when an application experiences an instantaneous spike in traffic.

Do I need to be concerned about high CPU warnings in the admin console for my loading requests?

App Engine provides high CPU warnings to help you determine which requests might need optimization. In the case of loading requests, though, the execution time is artificially longer due to the extra application initialization required. In addition, the number of loading requests is inversely proportional to the amount of traffic your application receives. So, while your CPU usage due to additional traffic will increase, your CPU usage due to loading requests will decrease.

Given that, your time is most often better spent focusing on optimizing other high CPU warnings in relation to your application's total CPU usage.

How can I speed up loading requests?

Here are a few suggestions:

  1. Perform application initialization lazily, rather than eagerly, so it doesn't all occur within a single request.
  2. Share expensive initialization between JVMs. For example, put data which is expensive to read or compute into memcache, where it can be quickly read by other JVMs during startup.
  3. Move initialization from application startup to build-time where reasonable. For example, convert a complex datafile into a simple, quick-to-read datafile in your build process.
  4. Use slimmer dependencies. For example, prefer a library that is optimized to your task, as opposed to a large library that performs very heavy initialization.

What is Google doing to speed up loading requests?

  1. In the release of 1.2.8 we introduced a new class-loading optimization called precompilation, which improved loading requests by 30% and greater.
  2. We're also making runtime optimizations guided by profiling applications with longer loading requests. In addition, we've provided profiling feedback to third-party language runtimes such as Groovy and JRuby and suggestions for optimization of their own libraries and runtimes.
  3. We're actively working on further startup optimizations.

How long does it take for an application to be unloaded when it has no traffic?

The mean idle time between eviction is dependent on several factors. Application loading and unloading is based upon an LRU (Least Recently Used) algorithm, so one application's mean time between an idle application being unloaded may be completely different from another application's, as it can depend on the usage characteristics of the other applications running in the same pool. We monitor the mean idle time of applications and target it to be on the order of a few minutes.

Can I pay to keep a JVM reserved for my application?

We've seen this request from some developers with low-traffic applications who'd like to reduce the percentage of loading requests they receive. Although we have many improvements in the pipeline to improve loading request performance, we'd like to gauge the general interest in this feature. If you'd like to be able to reserve a JVM at a price, please star this issue. If there's a particular pricing scheme you're interested in, let us know.

Should I run a cron job to keep my JVMs alive and reduce my loading requests?

We discourage developers from doing this because it increases the average number of loading requests for all low-traffic applications. Instead, we will continue to improve the performance of loading requests for everyone, and you can use the advice on this page to optimize your application's startup performance.

Tools

I'm already using port 8080, how do I change the dev appserver's port?

If you are seeing the following error message:

Could not open the requested socket: Address already in use

You may want to change the port on which the dev server will listen for incoming connections. You can set the port with the following argument:

--port=desired-port-number

If you are using an ant target with a dev_appserver element, you can specify the port argument by adding the port attribute as follows.

<dev_appserver war="war" port="desired-port-number"/>

Can I use the Google Data API library on App Engine?

Yes, the Google Data Java client library can be used in App Engine, but you need to set a configuration option to avoid a runtime permissions error. Add the following to your appengine-web.xml file:

<system-properties>
  <property name="com.google.gdata.DisableCookieHandler" value="true"/>
</system-properties>

If the preceding is not included, you may see the following exception:

java.security.AccessControlException: access denied (java.net.NetPermission getCookieHandler)

The Eclipse plugin is not working.

If you are running into unexpected issues with the Google Eclipse Plugin or even with Eclipse itself, a good place to start looking is the 'Error Log' view. From the Eclipse menu select 'Window -> Show View -> Other...' and then select 'General -> Error Log'. You can double-click on individual entries to get more information. Alternatively, you can find a text version of these issues in a file called your-eclipse-workspace-directory/.metadata/.log.

The email argument in appcfg doesn't seem to work.

The email command line argument must be specified before the appcfg operation. For example, the following is correct:

appcfg.sh -e youremail@example.com update app_directory/war

Python and Java

I have an exising Python app, is it possible to access the existing Python data store?

Yes, you can upload Java code to your app (I recommend specifying a new version number) and your Java code will have access to the data in the datastore. The kind names and property names are consistent across languages.

Be careful to ensure that the data types used in your model entities in Python can be loaded into your Java code.

Can I run Java and Python code in the same app?

Yes, each version of the app must specify a runtime language and it is possible to have version x of your app running Java, while version y is running Python. It would also be possible to use Jython.

Common Issues

How do I handle multipart form data? or How do I handle file uploads to my app?

You can obtain the uploaded file data from a multipart form post using classes from the Apache Commons FileUpload package. Specifically you may want to use FileItemStream, FileItermIterator and ServletFileUpload as illustrated below.

If you see a java.lang.NoClassDefFoundError after starting your application, make sure that the Apache Commons FileUpload JAR file has been copied to your war/WEB-INF/lib directory and added to your build path.

import org.apache.commons.fileupload.FileItemStream;
import org.apache.commons.fileupload.FileItemIterator;
import org.apache.commons.fileupload.servlet.ServletFileUpload;

import java.io.InputStream;
import java.io.IOException;
import java.util.logging.Logger;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

public class FileUpload extends HttpServlet {
  private static final Logger log =
      Logger.getLogger(FileUpload.class.getName());

  @Override
  public void doPost(HttpServletRequest req, HttpServletResponse res)
      throws ServletException, IOException {
    try {
      ServletFileUpload upload = new ServletFileUpload();
      res.setContentType("text/plain");

      FileItemIterator iterator = upload.getItemIterator(req);
      while (iterator.hasNext()) {
        FileItemStream item = iterator.next();
        InputStream stream = item.openStream();

        if (item.isFormField()) {
          log.warning("Got a form field: " + item.getFieldName());
        } else {
          log.warning("Got an uploaded file: " + item.getFieldName() +
                      ", name = " + item.getName());

          // You now have the filename (item.getName() and the
          // contents (which you can read from stream). Here we just
          // print them back out to the servlet output stream, but you
          // will probably want to do something more interesting (for
          // example, wrap them in a Blob and commit them to the
          // datastore).
          int len;
          byte[] buffer = new byte[8192];
          while ((len = stream.read(buffer, 0, buffer.length)) != -1) {
            res.getOutputStream().write(buffer, 0, len);
          }
        }
      }
    } catch (Exception ex) {
      throw new ServletException(ex);
    }
  }
}

Why do I get a UserServiceFailureException when I try to log in to my app?

If you associated your application with a Google Apps domain, you must access the application on a subdomain of your Google Apps account. For example, if example.com is your Google Apps domain, you must create a subdomain (we'll use www for this example) and point it at your app running on App Engine. You would then sign in to your app at www.example.com. If you try to access app through the http://your_app_id.appspot.com URL, you will not be able to sign in using your Google Apps account. This restriction applies only to apps which are restricted to a Google Apps domain.

I'm seeing an Initialization failed exception for all of my request handlers.

One of your filters, servlets, or JSPs likely has an error which prevents all of the handlers in the app from being properly initialized.

One possible cause for an initialization problem could arise if you are precompiling JSPs. The appcfg command precompiles the JSPs for you as part of the update process, so you probably do not need to precompile.

If you need to precompile them yourself for some reason, the fix is to bundle any JSP libraries along with the application. This includes:

  • Jasper runtime and the compiler, assuming that's what you used to do the precompilation.
  • Any tag libraries (including JSTL) that you need.
  • An EL implementation, if you're using it.

These libraries are added to each application as part of our offline JSP compilation which is done by appcfg, so if you are doing your own JSP compilation, you need to do the same thing.

If you have precompiled JSPs using Jasper, you will need to include the following jars in WEB-INF/lib:

  • commons-logging-api-1.1.1.jar (or later)
  • commons-el.jar
  • jasper-runtime.jar

Note that EL and logging are actually required even if you're not using them.

Authentication required

You need to be signed in with Google+ to do that.

Signing you in...

Google Developers needs your permission to do that.