Local List Mode

When clients choose to use Google Safe Browsing v5 in this mode, the client behavior is similar to the v4 Update API except using the improved API surface of v5. Clients will maintain in their local database a set of threat lists formatted as SHA256 hash prefixes of host-suffix/path-prefix URL expressions. Whenever the client wishes to check a particular URL, a check is performed using the local threat list. If and only if there is a match, the client connects to the server to continue the check.

As with the above, the client will also maintain a local cache that need not be in persistent storage.

The Local Threat List URL Check Procedure

This procedure is also performed when the client is using the Real-Time Mode procedure and it returns the value UNSURE.

This procedure takes a single URL u and returns SAFE or UNSAFE.

1. Let expressions be a list of suffix/prefix expressions generated by the URL u.
2. Let expressionHashes be a list, where the elements are SHA256 hashes of each expression in expressions.
3. Let expressionHashPrefixes be a list, where the elements are the first 4 bytes of each hash in expressionHashes.
4. For each expressionHashPrefix of expressionHashPrefixes:
   1. Look up expressionHashPrefix in the local cache.
   2. If the cached entry is found:
      1. Determine whether the current time is greater than its expiration time.
      2. If it is greater:
         1. Remove the found cached entry from the local cache.
         2. Continue with the loop.
      3. If it is not greater:
         1. Remove this particular expressionHashPrefix from expressionHashPrefixes.
         2. Check whether the corresponding full hash within expressionHashes is found in the cached entry.
         3. If found, return UNSAFE.
         4. If not found, continue with the loop.
   3. If the cached entry is not found, continue with the loop.
5. For each expressionHashPrefix of expressionHashPrefixes:
   1. Look up expressionHashPrefix in the local threat list database.
   2. If the expressionHashPrefix cannot be found in the local threat list database, remove it from expressionHashPrefixes.
6. Send expressionHashPrefixes to the Google Safe Browsing v5 server using RPC SearchHashes or the REST method hashes.search. If an error occurred (including network errors, HTTP errors, etc), return SAFE. Otherwise, let response be the response received from the SB server, which is a list of full hashes together with some auxiliary information identifying the nature of the threat (social engineering, malware, etc), as well as the cache expiration time expiration.
7. For each fullHash of response:
   1. Insert fullHash into the local cache, together with expiration.
8. For each fullHash of response:
   1. Let isFound be the result of finding fullHash in expressionHashes.
   2. If isFound is False, continue with the loop.
   3. If isFound is True, return UNSAFE.
9. Return SAFE.