Description

Traditional SEO is evolving rapidly. Today, autonomous AI agents, LLMs, and semantic search engines drive web traffic. If your Joomla content is buried under heavy layouts, scripts, tracking pixels, navigation menus, and other syntactic noise, AI crawlers will misinterpret your data or skip your website entirely.

Crusco Agentic Content bridges the gap between your website and the Agentic Web.

It automatically strips the noise and transforms your standard HTML into flawless, AI-ready Markdown. At the same time, it secures your intellectual property using the absolute latest, official AI Policy protocols. Don't let AI ignore your site; make it easy to read, index, and cite.

Why Choose Crusco Agentic Content?

• Future-Proof Your Traffic: Traditional search is shrinking; ensure your site is perfectly indexed and cited by AI agents, ChatGPT, Claude, etc.
• Cut AI Token Costs: Clean, optimized Markdown eliminates background noise, reducing token payload and server resource consumption.
• Enforce Legal Control: Do not let AI companies scrape your data for free; block unauthorized training using official Content Signals and EU AI Act protocols.

How Crusco Agentic Content Outperforms Generic Solutions

Unlike standard HTML-to-Markdown conversion tools or simple llms.txt generators, which often output noisy code and generic links, Crusco Agentic Content is an enterprise-grade optimization suite that turns raw web content into clean, structured, AI-ready assets.

Key Differences

Parsing Quality
* Generic Solution: Messy, basic HTML conversion.
* Crusco Agentic Content: Advanced hybrid engine with structural fallbacks.

Token Cost Control
* Generic Solution: High payload (Base64 data, hidden CSS, banners, and unwanted modules left intact).
* Crusco Agentic Content: Active token stripping removes noise at the source (up to 80% compression).

AI Copyright Protection
* Generic Solution: None or basic robots.txt.
* Crusco Agentic Content: Official W3C TDMRep, Content Signals, and EU AI Act compliance.

RAG System Readiness
* Generic Solution: Plain text only.
* Crusco Agentic Content: Native YAML Front Matter with 28+ schema types, engineered for immediate integration into any retrieval-augmented generation system.

Agent Readability
* Generic Solution: Only sets a basic Vary: Accept header, completely lacking .md sibling support or failing to implement it on the homepage. Relies on silent, messy HTML fallbacks when formatting demands mismatch.
* Crusco Agentic Content: Parses full Accept headers and quality values, enforcing a strict RFC 9110 compliant 406 Not Acceptable fallback if the AI crawler's request cannot be satisfied. Instead of a silent error, it serves a transparent response body listing available formats (HTML/Markdown) alongside alternative Link headers, guiding the crawler to the correct resource. Includes full .md sibling support with a dedicated index.md endpoint for the homepage.

Main Features

🛠️ Enterprise-Grade Markdown Engine

Built on top of the industry-standard League HTML-to-Markdown library, Crusco Agentic Content extends its capabilities with a specialized, custom-coded processing layer. Because every website is unique due to its specific structure, CSS, frameworks, and installed extensions, Crusco Agentic Content gives you absolute control over the exact data served to Large Language Models, completely eliminating AI hallucinations.

• Smart H1 Fallback: Automatically recovers missing H1 tags from template positions or falls back to the title tag.
• AI-Ready Page Anchors: Converts page layout anchors into a cross-platform Markdown format that preserves table-of-contents (TOC) navigation for LLMs.
• Process Media Tags: Because Markdown lacks native syntax for video and audio tags, Crusco Agentic Content converts multimedia elements into Thumbnail Links, a cross-platform Markdown representation ideal for AI.
• Process Map Tags: Although map and area tags have no direct equivalent in Markdown, Crusco Agentic Content transforms image maps into a semantic, structured Markdown text block that language models will ingest to understand the image map and boost their overall comprehension.
• Base64 Stripping: Optionally removes embedded images to save precious token space and prevent inaccurate AI answers.
• Hidden Element Purge: Instantly deletes content concealed via inline CSS, ensuring a leaner, cost-effective AI input.
• CSS Selector Targeting: Clean up your content by stripping cookie banners, captchas, comment boxes, and any unwanted or low-value elements, including specific Joomla modules.
• JSON-LD Controls: Easily include or exclude Schema markup to eliminate syntactic noise based on your needs.
• Component Blacklisting: Exclude specific core or third-party Joomla components from Markdown generation with one click.

🤖 Future-Proof Bot Discovery

Ensure universal compatibility with all modern and legacy AI crawlers. No bot gets left behind.

• HTTP Content Negotiation: Fully compliant with RFC 9110 directives. Achieves a perfect score on the acceptmarkdown.com “How AI-Ready is your URL?” test.
• Dedicated .md Endpoint: Leverages the RFC 8288 standard to mirror your content. Inserts alternative Markdown links into both the HTML head and HTTP headers for instant bot discovery.

⚡ SERP & CDN Safeguards

Engineered defensively. Prevents SEO cannibalization and eliminates caching mix-ups between human visitors and AI bots on CDNs or edge-caching layers.

🛡️ Security Header & Robots Preservation

Zero security compromises. Crusco Agentic Content serves Markdown while fully respecting your Joomla "HTTP Headers" system plugin configuration, including Strict-Transport-Security (HSTS) and Content-Security-Policy (CSP).

Robots restrictions are managed with smart, automated routing:

• Negotiated Markdown: Requests served from the original URL automatically mirror the exact noindex restrictions of the corresponding HTML page. This approach avoids sending conflicting signals (such as combining noindex with a canonical tag), ensuring search engines process the exclusion directive cleanly without ambiguity.
• Dedicated .md Endpoint: The standalone Markdown Mirror strictly enforces a noindex rule. This isolates alternative content views and prevents any risk of keyword cannibalization, ensuring your primary HTML pages retain full SEO priority.

📊 Structured YAML Metadata (Front Matter)

LLMs process information with massive accuracy when the context is declared in the very first lines. Crusco Agentic Content injects a rich YAML Front Matter block at the top of every Markdown response to guide models with zero ambiguity.

• Token Optimization: YAML cuts complexity and token consumption compared to bulky JSON-LD. It packages your author identity (E-E-A-T), schema type, chronological freshness, and taxonomies into a format that models assimilate instantly.
• RAG System Routing: Empowers AI agents to filter, catalog, and validate your source's authority before text chunking even begins.
• 4 Preset Formatting Modes: Choose the style that fits your strategy: Accurate, Balanced, Token Saver, or Custom.

Supported Content Types (With Automated Fallback)

The plugin dynamically maps and falls back across 28+ schema types (from Article, Product, and Guide down to a universal WebPage wrapper). This guarantees the crucial type attribute is always present for LLM classification. Supported types:
- AdvertiserContentArticle, Article, Blog, BlogPosting, Book, ComicStory, Course, CreativeWork, Diet, DiscussionForumPosting, Event, Guide, HowTo, JobPosting, MobileApplication, NewsArticle, Product, Recipe, Report, SatiricalArticle, ScholarlyArticle, SocialMediaPosting, SoftwareApplication, TechArticle, Thesis, VideoGame, WebApplication, WebPage.

🇪🇺 AI Policy & EU AI Act Compliance

Regain full legal and technical control over your data directly within your site's HTTP headers, fully compliant with Article 53 of the EU AI Act, which legally binds AI providers to respect machine-readable copyright reservations.

• Global Guardrails (TDMRep & Content Signals): The plugin simultaneously activates two major global standards to configure bot access. It implements the official W3C-ratified TDMRep protocol for European copyright protection, alongside Content Signals syntax, the HTTP-based framework backed by Cloudflare and Thoughtworks, to dictate whether bots can use your texts for real-time answers (RAG) or block commercial training datasets.
• Machine-Readable TDM Policy URL: In strict compliance with the EU AI Act and the CDSM Directive, you can attach a formal policy URL to your network responses. This allows organizations to provide automated systems with precise terms, structured in a machine-readable ODRL profile following official W3C specifications.
• Link Directives: Control whether AI agents can follow links within the Markdown output to explore topics further and cite sources, or restrict them to protect your link network.

Explore all features and in-depth guides in our official documentation.

Translations

The Extension is translated into the following languages:

• Italian
• English
• German
• Spanish
• French
• Russian
• Chinese