When building apps for kids, safety should always be top of mind. Developers can help kids stay safe by carefully considering each and every feature that goes into their app.

The majority of your users will be kids, parents, teachers, and caregivers who use your app as you intended. But there may also be bad actors, and you need to be prepared.

Potentially risky features

Think of unintended consequences. Are there ways that someone could use your app to harm kids? What features of your app could expose kids to worst-case uses by bad actors? Here are a few common features that bad actors are known to use.

Does your app let users chat with each other? Bad actors often exploit chat functionality, even in apps for kids. Bad actors can use chat to trick kids into making purchases or downloading other apps with malware.

More worryingly, they may use chat for predatory purposes. They can use the app to initiate inappropriate conversations with kids, or even use a child’s username to reach them outside of the app. Once they have established a relationship and a means of communicating with a child, these bad actors may extort them into sending inappropriate images of themselves, groom them for sexual exploitation, or lure them into meeting in person.

Ways to reduce in-app chat risks for kids

  • Disable image sharing for your chat function.
  • Allow only pre-approved chats that kids can select from, like pre-approved phrases and sticker/emoji communications.
  • Moderate the content of group chats.
  • Identify and flag potentially problematic chat for manual review by your moderators.
  • Allow users to report inappropriate contact and/or block users.
  • Make strict parental controls available so parents can monitor their kids’ chats. Explore tips for talking to kids about online safety from Family Online Safety Institute.
  • Let parents approve and manage connections their child makes, block and remove users, and review messages sent to and from their child.

If your app allows comments, reviews, or other content to be uploaded by users, then your app allows UGC. Allowing UGC increases the risk profile of your app. Bad actors may use comments to bully, post offensive or inappropriate photos, or inappropriately share contact information.

Ways to reduce UGC risks for kids

  • Moderate all content for appropriateness before it’s posted.
  • Post clear community guidelines explaining what is and isn’t allowed, and then strictly enforce these guidelines.
  • Allow users to report inappropriate posts.
  • Make parental controls available so parents can monitor what their kids post and what their kids can see posted.
By providing tools that empower parents and kids to control the experience, you can encourage safe and responsible use of your app. Give users easy-to-use, clearly explained settings that allow them to control when, how, and who they interact with on your app. You can also empower teachers and parents to help kids be safer, more confident explorers of the online world with Google's Be Internet Awesome program.