Getting Started with Sandbox2

On this page, you will learn how to create your own sandboxed environment with Sandbox2. You will learn how to define a Sandbox Policy, and some advanced, but common, tweaks. Use the information here as a guide, alongside the examples and code documentation in the header files.

1. Choose a Sandbox Executor Method

Sandboxing starts with an executor (see Sandbox Executor), which is responsible for running the Sandboxee. The executor.h header file contains the API needed for this purpose. The API is very flexible and lets you choose what works best for your use case. The following sections describe the 3 different methods from which you can choose.

Method 1: Stand-alone – Execute a binary with sandboxing already enabled

This is the simplest way to use sandboxing and is the recommended method when you want to sandbox an entire binary which you have no source code for. It is also the safest way to use sandboxing, as there is no unsandboxed initialization that could have adverse effects.

In the following code snippet, we define the path of the binary to be sandboxed and the arguments we have to pass to an execve syscall. As you can see in the executor.h header file, we don't specify a value for envp and therefore copy the environment from the parent process. Remember, the first argument is always the name of the program to be executed, and our snippet doesn't define any other argument.

Examples of this executor method are: static and tool.

#include "sandboxed_api/sandbox2/executor.h"

std::string path = "path/to/binary";
std::vector<std::string> args = {path};  // args[0] will become the sandboxed
                                         // process' argv[0], typically the
                                         // path to the binary.
auto executor = absl::make_unique<sandbox2::Executor>(path, args);

Method 2: Sandbox2 Forkserver – Tell the executor when to be sandboxed

This method offers the flexibility of being unsandboxed during initialization, and then choosing when to enter sandboxing by calling ::sandbox2::Client::SandboxMeHere(). It requires you to be able to define in code when you want to start sandboxing, and it has to be single-threaded (read why in the FAQ).

In the following code snippet, we use the same code as outlined in Method 1 above. However, in order to allow the program to execute in an unsandboxed manner during initialization, we call set_enable_sandbox_before_exec(false).

#include "sandboxed_api/sandbox2/executor.h"

std::string path = "path/to/binary";
std::vector<std::string> args = {path};
auto executor = absl::make_unique<sandbox2::Executor>(path, args);
executor->set_enable_sandbox_before_exec(false);

As the executor now has a disabled sandbox until it is notified by the Sandboxee, we have to create a ::sandbox2::Client instance, set up communication between the executor and Sandboxee, and then notify the executor that our initialization is finished and we want to start sandboxing now by calling sandbox2_client.SandboxMeHere().

// main() of sandboxee
int main(int argc, char** argv) {
  gflags::ParseCommandLineFlags(&argc, &argv, false);

  // Set-up the sandbox2::Client object, using a file descriptor (1023).
  sandbox2::Comms comms(sandbox2::Comms::kSandbox2ClientCommsFD);
  sandbox2::Client sandbox2_client(&comms);
  // Enable sandboxing from here.
  sandbox2_client.SandboxMeHere();
  …

Caution: Be careful to ensure the code always calls SandboxMeHere(), otherwise it is unsafe to proceed.

An example of this executor method is crc4, where crc4bin.cc is the Sandboxee and notifies the executor (crc4sandbox.cc) when it should enter the sandbox.

Method 3: Custom Forkserver – Prepare a binary, wait for fork requests, and sandbox on your own

This mode allows you to start a binary, prepare it for sandboxing, and, at a specific moment of your binary's lifecycle, make it available to the executor.

The executor will send a fork request to your binary, which will fork() (via ::sandbox2::ForkingClient::WaitAndFork()). The newly created process will be ready to be sandboxed with ::sandbox2::Client::SandboxMeHere().

#include "sandboxed_api/sandbox2/executor.h"

// Start the custom ForkServer
std::string path = "path/to/binary";
std::vector<std::string> args = {path};
auto fork_executor = absl::make_unique<sandbox2::Executor>(path, args);
fork_executor->StartForkServer();

// Initialize Executor with Comms channel to the ForkServer
auto executor = absl::make_unique<sandbox2::Executor>(
    fork_executor->ipc()->GetComms());

Keep in mind that this mode is quite complicated and applicable only in a few specific cases; for example, when you have tight memory requirements. You will benefit from COW but have the downside that there is no real ASLR. Another typcial usage example would be when the Sandboxee has a long, CPU-intensive initialization that can be run before the untrusted data is processed.

For an example of this executor method, see custom_fork.

2. Create a Sandbox Policy

Once you have an executor, you will likely want to define a Sandbox Policy for the Sandboxee. Otherwise, the Sandboxee is only protected by the Default Syscall Policy.

With the Sandbox Policy, the objective is to restrict the syscalls and arguments that the Sandboxee can make, as well as the files it can access. You will need to have a detailed understanding of the syscalls required by the code you plan to sandbox. One way of observing syscalls is to run the code with Linux's command-line tool strace.

Once you have the list of syscalls, you can use the PolicyBuilder to define the policy. PolicyBuilder comes with many convenience and helper functions that allow many common operations. The following list is only a small excerpt of available functions:

• Allowlist any syscall for process startup:
  • AllowStaticStartup();
  • AllowDynamicStartup();
• Allowlist any open/read/write* syscalls:
  • AllowOpen();
  • AllowRead();
  • AllowWrite();
• Allowlist any exit/access/state related syscalls:
  • AllowExit();
  • AllowStat();
  • AllowAccess();
• Allowlist any sleep/time related syscalls:
  • AllowTime();
  • AllowSleep();

These convenience functions allowlist any relevant syscall. This has the advantage that the same policy can be used over different architectures where certain syscalls are not available (e.g. ARM64 has no OPEN syscall), but with the minor security risk of enabling more sycsalls than might be necessary. For example, AllowOpen() enables the Sandboxee to call any open related syscall. If you only want to allowlist one specific syscall, you can use AllowSyscall(); to allow multiple syscalls at once you can use AllowSyscalls().

So far the policy only checks the syscall identifier. If you have the need to further strengthen the policy and want to define a policy in which you only allow a syscall with particular arguments, you need to use AddPolicyOnSyscall() or AddPolicyOnSyscalls(). These functions not only take the syscall ID as an argument, but also a raw seccomp-bpf filter using the bpf helper macros from the Linux kernel. See the kernel documentation for more information about BPF. If you find yourself writing repetitive BPF code that you think should have a usability-wrapper, feel free to file a feature request.

Apart from syscall-related functions, the PolicyBuilder also provides a number of filesystem-related functions like AddFile() or AddDirectory() to bind-mount a file/directory into the sandbox. The AddTmpfs() helper can be used to add a temporary file storage within the sandbox.

A particularly useful function is AddLibrariesForBinary() which adds the libraries and linker required by a binary.

Coming up with the syscalls to allowlist is still a bit of manual work unfortunately. Create a policy with the syscalls you know your binary needs and run it with a common workload. If a violation is triggered, allowlist the syscall and repeat the process. If you run into a violation that you think might be risky to allowlist and the program handles errors gracefully, you can try to make it return an error instead with BlockSyscallWithErrno().

#include "sandboxed_api/sandbox2/policy.h"
#include "sandboxed_api/sandbox2/policybuilder.h"
#include "sandboxed_api/sandbox2/util/bpf_helper.h"

std::unique_ptr<sandbox2::Policy> CreatePolicy() {
  return sandbox2::PolicyBuilder()
    .AllowSyscall(__NR_read)  // See also AllowRead()
    .AllowTime()              // Allow time, gettimeofday and clock_gettime
    .AddPolicyOnSyscall(__NR_write, {
        ARG(0),        // fd is the first argument of write (argument #0)
        JEQ(1, ALLOW), // allow write only on fd 1
        KILL,          // kill if not fd 1
    })
    .AddPolicyOnSyscall(__NR_mprotect, {
        ARG_32(2), // prot is a 32-bit wide argument, so it's OK to use *_32
                   // macro here
        JNE32(PROT_READ | PROT_WRITE, KILL), // prot must be the RW, otherwise
                                             // kill the process
        ARG(1), // len is a 64-bit argument
        JNE(0x1000, KILL),  // Allow single page syscalls only, otherwise kill
                            // the process
        ALLOW,              // Allow for the syscall to proceed, if prot and
                            // size match
    })
    // Allow the openat() syscall but always return "not found".
    .BlockSyscallWithErrno(__NR_openat, ENOENT)
    .BuildOrDie();
}

Note: Test the most frequently used syscalls at the beginning so you can allow them early on without consulting the rest of the policy.

3. Adjust Limits

The Sandbox Policy prevents the Sandboxee from calling specific syscalls and thus reduces the attack surface. However, an attacker might still be able to cause undesired effects by running a process indefinitely or exhausting RAM and other resources.

To address this threat, the Sandboxee runs under tight execution limits by default. If these default limits cause issues for the legitimate execution of your program, you can adjust them using the sandbox2::Limits class by calling limits() on the executor object.

The code snippet below shows some example limit adjustments. All available options are documented in the limits.h header file.

// Restrict the address space size of the sandboxee to 4 GiB.
executor->limits()->set_rlimit_as(4ULL << 30);
// Kill sandboxee with SIGXFSZ if it writes more than 1 GiB to the filesystem.
executor->limits()->set_rlimit_fsize(1ULL << 30);
// Number of file descriptors which can be used by the sandboxee.
executor->limits()->set_rlimit_nofile(1ULL << 10);
// The sandboxee is not allowed to create core files.
executor->limits()->set_rlimit_core(0);
// Maximum 300s of real CPU time.
executor->limits()->set_rlimit_cpu(300);
// Maximum 120s of wall time.
executor->limits()->set_walltime_limit(absl::Seconds(120));

For an example of the use of the sandbox2::Limits class, see the example tool.

4. Run the Sandbox

In the previous sections, you prepared the sandboxed environment, policy, and executor and Sandboxee. The next step is to create the Sandbox2 object and run it.

Run synchronously

The sandbox can run synchronously, thus blocking until there is a result. The code snippet below demonstrates the instantiation of the Sandbox2 object and its synchronous execution. For a more detailed example, see static.

#include "sandboxed_api/sandbox2/sandbox2.h"

sandbox2::Sandbox2 s2(std::move(executor), std::move(policy));
sandbox2::Result result = s2.Run();  // Synchronous
LOG(INFO) << "Result of sandbox execution: " << result.ToString();

Run asynchronously

You can also run the sandbox asynchronously, thus not blocking until there is a result. This is useful, for instance, when communicating with the Sandboxee. The code snippet below demonstrates this use case, for more detailed examples see crc4 and tool.

#include "sandboxed_api/sandbox2/sandbox2.h"

sandbox2::Sandbox2 s2(std::move(executor), std::move(policy));
if (s2.RunAsync()) {
  // Communicate with sandboxee, use s2.Kill() to kill it if needed
  // ...
}
Sandbox2::Result result = s2.AwaitResult();
LOG(INFO) << "Final execution status: " << result.ToString();

5. Communicating with the Sandboxee

By default, the executor can communicate with the Sandboxee through file descriptors. This might be all you need, for example if you just want to share a file with the Sandboxee, or read the Sandboxee's standard output.

However, you most likely have the need for more complex communication logic between the executor and Sandboxee. The comms API (see the comms.h header file) can be used to send integers, strings, byte buffers, protobufs, or file descriptors.

Sharing File Descriptors

Using the Inter-Process Communication API (see ipc.h), you can use MapFd() or ReceiveFd():

• Use MapFd() to map file descriptors from the executor to the Sandboxee. This can be used to share a file opened from the executor for use in the Sandboxee. An example use can be seen in static.

  // The executor opened /proc/version and passes it to the sandboxee as stdin
  executor->ipc()->MapFd(proc_version_fd, STDIN_FILENO);
  

• UseReceiveFd() to create a socketpair endpoint. This can be used to read the Sandboxee's standard output or standard errors. An example use can be seen in the tool.

  // The executor receives a file descriptor of the sandboxee stdout
  int recv_fd1 = executor->ipc())->ReceiveFd(STDOUT_FILENO);
  

Using the comms API

Sandbox2 provides a convenient comms API. This is a simple and easy way to share integers, strings, or byte buffers between the executor and Sandboxee. Below are some code snippets that you can find in the crc4 example.

To get started with the comms API, you first have to get the comms object from the Sandbox2 object:

sandbox2::Comms* comms = s2.comms();

Once the comms object is available, data can be sent to the Sandboxee using one of the Send* family of functions. You can find an example use of the comms API in the crc4 example. The code snippet below shows an excerpt from that example. The executor sends an unsigned char buf[size] with SendBytes(buf, size):

if (!(comms->SendBytes(static_cast<const uint8_t*>(buf), sz))) {
  /* handle error */
}

To receive data from the Sandboxee, use one of the Recv* functions. The code snippet below is an excerpt from the crc4 example. The executor receives the checksum in a 32-bit unsigned integer: uint32_t crc4;

if (!(comms->RecvUint32(&crc4))) {
  /* handle error */
}

Sharing Data with Buffers

Another data sharing functionality is to use the buffer API to share large amounts of data and to avoid expensive copies that are sent back and forth between the executor and Sandboxee.

The executor creates a Buffer, either by size and data to be passed, or directly from a file descriptor, and passes it to the Sandboxee using comms->SendFD() in the executor and comms->RecvFD() in the Sandboxee.

In the code snippet below, you can see the executor's side. The sandbox runs asynchronously and shares data via a buffer with the Sandboxee:

// start the sandbox asynchronously
s2.RunAsync();

// instantiate the comms object
sandbox2::Comms* comms = s2.comms();

// random buffer data we want to send
constexpr unsigned char buffer_data[] = /* random data */;
constexpr unsigned int buffer_dataLen = 34;

// create sandbox2 buffer
absl::StatusOr<std::unique_ptr<sandbox2::Buffer>> buffer =
     sandbox2::Buffer::CreateWithSize(1ULL << 20 /* 1Mib */);
std::unique_ptr<sandbox2::Buffer> buffer_ptr = std::move(buffer).value();

// point to the sandbox2 buffer and fill with data
uint8_t* buf = buffer_ptr‑>data();
memcpy(buf, buffer_data, buffer_data_len);

// send the data to the sandboxee
comms‑>SendFd(buffer_ptr‑>fd());

On the Sandboxee's side, you also have to create a buffer object and read the data from the file descriptor sent by the executor:

// establish the communication with the executor
int fd;
comms.RecvFD(&fd);

// create the buffer
absl::StatusOr<std::unique_ptr<sandbox2::Buffer>> buffer =
     sandbox2::Buffer::createFromFd(fd);

// get the data
auto buffer_ptr = std::move(buffer).value();
uint8_t* buf = buffer_ptr‑>data();

/* work with the buf object */

6. Exiting the sandbox

Depending on how you run the sandbox (see this step), you have to adjust the way you terminate the sandbox, and thus also the Sandboxee.

Exiting a sandbox running synchronously

If the sandbox has been running synchronously, then Run will only return when the Sandboxee is finished. Therefore no additional step for termination is required. The code snippet below shows this scenario:

Sandbox2::Result result = s2.Run();
LOG(INFO) << "Final execution status: " << result.ToString();

Exiting a sandbox running asynchronously

If the sandbox has been running asynchronously, then two options are available for termination. First, you can just wait for the completion of the Sandboxee and receive the final execution status:

sandbox2::Result result = s2.AwaitResult();
LOG(INFO) << "Final execution status: " << result.ToString();

Alternatively, you can kill the Sandboxee at any time, but it's still recommended to call AwaitResult() because the Sandboxee might terminate because of another reason in the meantime:

s2.Kill();
sandbox2::Result result = s2.AwaitResult();
LOG(INFO) << "Final execution status: " << result.ToString();

7. Test

Like any other code, your sandbox implementation should have tests. Sandbox tests are not meant to test the program's correctness, but instead to check whether the sandboxed program can run without issues like sandbox violations. This also makes sure that the sandbox policy is correct.

A sandboxed program is tested in the same way you would run it in production, with the arguments and input files it would normally process.

These tests can be as simple as a shell test or C++ tests using sub processes. Check out the examples for inspiration.

Conclusion

Thanks for reading this far, we hope you liked our guide and now feel empowered to create your own sandboxes to help keep your users safe.

Creating sandboxes and policies is a difficult task and prone to subtle errors. To remain on the safe side, we recommend you have a security expert review your policy and code.