The Submit function of the robots.txt Tester tool
allows you to easily put in place and ask Google to more quickly crawl and index a
robots.txt file for your site. Update and notify Google of changes
robots.txt file by following the steps below.
- Click Submit in the bottom-right corner of the
robots.txteditor. This action opens up a Submit dialog.
- Download your edited
robots.txtcode from the robots.txt Tester page by clicking Download in the Submit dialog.
- Upload your new
robots.txtfile to the root of your domain as a text file named
robots.txt(the URL for your
robots.txtfile should be
If you do not have permission to upload files to the root of your domain, you should contact your domain manager to make changes.
For example, if your site home page resides under
subdomain.example.com/site/example/, you likely cannot update the robots file
subdomain.example.com/robots.txt. In this case, you should contact the owner of
example.com/to make any necessary changes to the
- Click Verify live version to see that your live
robots.txtis the version that you want Google to crawl.
- Click Submit live version to notify Google that changes have been made to your
robots.txtfile and request that Google crawl it.
- Check that your newest version was successfully crawled by Google by refreshing the page in your
browser to update the tool’s editor and see your live
robots.txtcode. After you refresh the page, you can also click the dropdown above the text editor to view the timestamp of when Google first saw the latest version of your