Hide

Appendix: References

RFCs and existing standards

Previous robots.txt documents

Shared robots.txt extensions

Tools

Other implementations

Other references