This guide shows you how to use the Cardboard SDK for iOS to create your own Virtual Reality (VR) experiences.
You can use the Cardboard SDK to turn a smartphone into a VR platform. A smartphone can display 3D scenes with stereoscopic rendering, track and react to head movements, and interact with apps by detecting when the user presses the viewer button.
To get started, you'll use HelloCardboard, a demo game that demonstrates the core features of the Cardboard SDK. In the game, users look around a virtual world to find and collect objects. It shows you how to:
- Set up your development environment
- Download and build the demo app
- Scan the QR code of a Cardboard viewer to save its parameters
- Track the user’s head movements
- Render stereoscopic images by setting the correct distortion for each eye
Set up your development environment
Hardware requirements:
- iPhone running iOS 11 or higher
- Cardboard viewer
Software requirements:
- Xcode 10.3 or higher
Download and build the demo app
HelloCardboard is built using a pre-built Protocol Buffers binary. Steps to build the binary from scratch can be found here.
Clone the Cardboard SDK and the Hello Cardboard demo app from GitHub by running this command:
git clone https://github.com/googlevr/cardboard.git
Open the Cardboard workspace (
Cardboard.xcworkspace) in Xcode.Change the app's bundle ID so that you can sign the app with your team.
Click Run.
Scan the QR code
To save the device parameters, scan the QR code on the Cardboard viewer:
Try the demo
In HelloCardboard, you'll look for and collect geodesic spheres in 3D space.
To find and collect a sphere:
Move your head in any direction until you see a floating sphere.
Look directly at the sphere. This causes it to change colors.
Press the Cardboard viewer button to "collect" the sphere.
Configure the device
When the user taps the gear icon to switch Cardboard viewers, the didTapSwitchButton
method is called in HelloCardboardOverlayView.
- (void)didTapSwitchButton:(id)sender {
if ([self.delegate respondsToSelector:@selector(didTapBackButton)]) {
[self.delegate didChangeViewerProfile];
}
self.settingsBackgroundView.hidden = YES;
}
This calls CardboardQrCode_scanQrCodeAndSaveDeviceParams, which opens the
window for scanning the viewer’s QR code. When the user scans the QR code, the device's
distortion parameters are updated.
- (void)switchViewer {
CardboardQrCode_scanQrCodeAndSaveDeviceParams();
}
- (void)didChangeViewerProfile {
[self pauseCardboard];
[self switchViewer];
[self resumeCardboard];
}
Head tracking
Create head tracker
The head tracker is created once in the viewDidLoad method of HelloCardboardViewController:
_cardboardHeadTracker = CardboardHeadTracker_create();
Pause and resume head tracker
The pauseCardboard and resumeCardboard methods in the
HelloCardboardViewController class pause and resume the head tracker,
respectively. resumeCardboard also sets the _updateParams flag, which causes
the device parameters to be updated in the next draw call.
- (void)pauseCardboard {
self.paused = true;
CardboardHeadTracker_pause(_cardboardHeadTracker);
}
- (void)resumeCardboard {
// Parameters may have changed.
_updateParams = YES;
// Check for device parameters existence in app storage. If they're missing,
// we must scan a Cardboard QR code and save the obtained parameters.
uint8_t *buffer;
int size;
CardboardQrCode_getSavedDeviceParams(&buffer, &size);
if (size == 0) {
[self switchViewer];
}
CardboardQrCode_destroy(buffer);
CardboardHeadTracker_resume(_cardboardHeadTracker);
self.paused = false;
}
Lens distortion
Every time Cardboard scans a new QR code, the following code reads the saved parameters and uses them to create the lens distortion object, which applies the proper lens distortion to the rendered content:
CardboardQrCode_getSavedDeviceParams(&encodedDeviceParams, &size);
// Create CardboardLensDistortion.
CardboardLensDistortion_destroy(_cardboardLensDistortion);
_cardboardLensDistortion =
CardboardLensDistortion_create(encodedDeviceParams, size, width, height);
// Initialize HelloCardboardRenderer.
_renderer.reset(new cardboard::hello_cardboard::HelloCardboardRenderer(
_cardboardLensDistortion, _cardboardHeadTracker, width, height));
Rendering
Rendering content in Cardboard involves the following:
- Creating textures
- Getting view and projection matrices for the left and right eyes
- Creating the renderer and setting the distortion mesh
- Rendering each frame
Create textures
The content is drawn onto a texture, which is split into sections for the left and right eye.
These sections are initialized in _leftEyeTexture and _rightEyeTexture, respectively.
The sample app uses a single texture for both eyes, but it is also possible to create a separate
texture for each eye.
// Generate texture to render left and right eyes.
glGenTextures(1, &_eyeTexture);
glBindTexture(GL_TEXTURE_2D, _eyeTexture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, _width, _height, 0, GL_RGB, GL_UNSIGNED_BYTE, 0);
_leftEyeTexture.texture = _eyeTexture;
_leftEyeTexture.left_u = 0;
_leftEyeTexture.right_u = 0.5;
_leftEyeTexture.top_v = 1;
_leftEyeTexture.bottom_v = 0;
_rightEyeTexture.texture = _eyeTexture;
_rightEyeTexture.left_u = 0.5;
_rightEyeTexture.right_u = 1;
_rightEyeTexture.top_v = 1;
_rightEyeTexture.bottom_v = 0;
CheckGLError("Create Eye textures");
These textures are passed in as parameters to CardboardDistortionRenderer_renderEyeToDisplay.
Get view and projection matrices for left and right eye
First, retrieve eye matrices for the left and right eyes:
CardboardLensDistortion_getEyeFromHeadMatrix(_lensDistortion, kLeft, _eyeMatrices[kLeft]);
CardboardLensDistortion_getEyeFromHeadMatrix(_lensDistortion, kRight, _eyeMatrices[kRight]);
CardboardLensDistortion_getProjectionMatrix(_lensDistortion, kLeft, kZNear, kZFar,
_projMatrices[kLeft]);
CardboardLensDistortion_getProjectionMatrix(_lensDistortion, kRight, kZNear, kZFar,
_projMatrices[kRight]);
Next, get the distortion meshes for each of the eyes and pass it to the distortion renderer:
CardboardLensDistortion_getDistortionMesh(_lensDistortion, kLeft, &leftMesh);
CardboardLensDistortion_getDistortionMesh(_lensDistortion, kRight, &rightMesh);
Create the renderer and set the correct distortion mesh
The renderer needs to be initialized only once. Once the renderer is created, set the new
distortion mesh for the left and right eyes according to the mesh values returned from the
CardboardLensDistortion_getDistortionMesh function.
_distortionRenderer = CardboardOpenGlEs2DistortionRenderer_create();
CardboardDistortionRenderer_setMesh(_distortionRenderer, &leftMesh, kLeft);
CardboardDistortionRenderer_setMesh(_distortionRenderer, &rightMesh, kRight);
Rendering the content
Retrieve the current head orientation from CardboardHeadTracker_getPose:
CardboardHeadTracker_getPose(_headTracker, targetTime, position, orientation);
_headView =
GLKMatrix4Multiply(GLKMatrix4MakeTranslation(position[0], position[1], position[2]),
GLKMatrix4MakeWithQuaternion(GLKQuaternionMakeWithArray(orientation)));
Use the current head orientation with the view and projection matrices to compose a view projection matrix, and use them to render the world content for each of the eyes:
// Draw left eye.
glViewport(0, 0, _width / 2.0, _height);
glScissor(0, 0, _width / 2.0, _height);
DrawWorld(_leftEyeViewPose, GLKMatrix4MakeWithArray(_projMatrices[kLeft]));
// Draw right eye.
glViewport(_width / 2.0, 0, _width / 2.0, _height);
glScissor(_width / 2.0, 0, _width / 2.0, _height);
DrawWorld(_rightEyeViewPose, GLKMatrix4MakeWithArray(_projMatrices[kRight]));
Use CardboardDistortionRenderer_renderEyeToDisplay to apply the distortion
correction to the content, and render the content to the screen.
CardboardDistortionRenderer_renderEyeToDisplay(_distortionRenderer, renderTarget, /*x=*/0,
/*y=*/0, _width, _height, &_leftEyeTexture,
&_rightEyeTexture);