Descripción
AI Scrape Protect is a WordPress plugin designed to protect your website from scraping for AI training purposes. It achieves this by adding opt-out instructions to the robots.txt file for the most common AI scraping bots and including meta tags to control how your content is used.
Note: These instructions are not always respected by all bots.
Features
- Adds specific
User-agentandDisallowrules to yourrobots.txtfile to block a comprehensive list of AI scraping bots. - Introduces meta tags in the HTML
<head>to provide additional instructions to AI bots, including new tags for Bingbot and general AI compliance. - Prepares for future standards with support for
DisallowAITrainingandnoimageaimeta tags. - Dedicated handling of specific bots like CCBot and Bingbot for better protection and compatibility.
- Admin bar icon to indicate plugin activity.
Capturas
robots.txt File Example: Shows how the plugin updates the robots.txtfile.
meta tags Example: Shows an example of the Meta Tags added to the head section.
Instalación
- Upload the
ai-scrape-protectfolder to the/wp-content/plugins/directory. - Activate the plugin through the ‘Plugins’ menu in WordPress.
Preguntas frecuentes
-
How does this plugin protect my site from AI scraping?
-
The plugin adds specific
User-agententries to yourrobots.txtfile to instruct common AI scraping bots not to crawl or scrape your site. It also introduces meta tags in the HTML<head>to provide additional instructions to AI bots. -
Will this completely stop AI scraping of my site?
-
While this plugin adds recommendations to the
robots.txtfile and includes meta tags, not all bots follow these rules. This is a measure to discourage scraping rather than a foolproof solution. -
Can I add or remove bots from the list?
-
Currently, the plugin includes a predefined list of bots. If you need to add or remove specific bots, you would need to modify the plugin code or contact the plugin author for customization.
-
What happens if I deactivate the plugin?
-
The
robots.txtfile will revert to its previous state before the plugin was activated, and the meta tags added to the HTML<head>will be removed.
Reseñas
Colaboradores y desarrolladores
“AI Scrape Protect” es un software de código abierto. Las siguientes personas han colaborado con este plugin.
Colaboradores
Traduce “AI Scrape Protect” a tu idioma.
¿Interesado en el desarrollo?
Revisa el código , echa un vistazo al repositorio SVN , o suscríbete al log de desarrollo por RSS .
Registro de cambios
3.1
- Added new AI bots to the block list: DuckDuckBot, OpenAIContentCrawler, YandexBot, NeevaBot, AIMatrixCrawler.
- Improved admin bar icon clarity with higher resolution and visual indicators (green dot for active state).
3.0
- Added admin bar icon functionality to indicate plugin activity.
- Updated meta tags for compliance with official documentation and improved AI scraping protection:
- Adjusted Bingbot tag to use
nocachefor better compatibility with Bing AI Chat. - Removed “noindex” from the
ai-botmeta tag to allow search engine indexing. - Added
DisallowAITrainingandnoimageaito therobotsmeta tag. - Introduced dedicated meta tags for CCBot.
- Adjusted Bingbot tag to use
- Removed OpenAI SearchBot from the
robots.txtblocklist based on OpenAI’s recommendations.
2.4
- Updated meta tags for improved AI scraping protection.
- Prevent Bingbot and general AI bots from using the content for AI purposes.
- Added clear comments in the code to describe the functionality of each meta tag.