This plugin needs more documentation!
You can edit your robots.txt and add remote content to it.
E.g. you have several sites and want to use a centralized robots.txt.
Features
- Include or exclude WordPress' own robots.txt (core function)
- Include or exclude plugins - e.g. sitemap plugins - output to robots.txt (filter output)
- Include or exclude a remote text file (the common part)
- Include or exclude custom records from the settings page (the site specific part)
Where is robot.txt?
WordPress handles robots.txt as a virtual URL - just the same way as posts and pages.
So when you browse to
https://example.com/robots.txt
WordPress generates robots.txt on the fly.
TODO
- add more description here
- add a video too
- add an admin notice for subdir installs (robots.txt is useless in a subdir)
- 'At least one "Disallow" field must be present in the robots.txt file.' - check for that
Links
Development of this plugin goes on on
GitHub.
This section describes how to install the plugin and get it working.
- Upload the content of the ZIP feil to the
/wp-content/plugins/
directory
- Activate the plugin through the 'Plugins' menu in WordPress