Introducing the RSL Standard: A New Licensing Layer for an AI-First Web

Brett Tabkeon

For years, SEOs and site owners have had to set by and watch their content get scraped, trained on, and repurposed by SE’s and AI systems without much more than a nod in return.  Google, Yahoo, Microsoft, and even rolling back to Inktomi and Altavista, all refused to strike general deals with site owners.

Enough websites have now come together to say “enough is enough”. A coalition of big publishers and tech players: Reddit, Yahoo, Quora, Ziff Davis, Medium, O’Reilly, Fastly, Internet Brands, People Inc. have stepped forward with an honest real solution: the Really Simple Licensing (RSL) Standard.

We’ve been stuck with robots.txt standard that was never endorsed by anyone anywhere, as the only gatekeeper. It could say “yes” or “no” to bots, but that’s it. Effectively useless. RSL takes the next step: machine-readable licenses baked right into your own website. Instead of blocking or allowing, you can now publish actual terms of use and monetization rules for crawlers and AI agents!

That means:

  • Licensing options ranging from attribution-only to full-on subscription models!
  • Per-crawl or per-inference royalties
  • Compatibility with the same workflow you already use – robots.txt, HTTP headers, RSS

The Collective Muscle

The launch also includes the RSL Collective, a nonprofit built to give publishers strength in numbers. Modeled after how ASCAP and BMI represent musicians, this group helps site owners band together to negotiate with the big bad AI companies. Membership is free (as in free beer!)  but the bargaining power is enormous with these powerhouses. Yes friends, this is a big deal.

This  is similar to when RSS starts. It is pretty much the same idea, with this time adding aa licensing and payment layer.

What It Looks Like in Practice

Implementation is straightforward. You can:

  • Add an RSL pointer in your robots.txt (easy)
  • Declare terms via HTTP headers (easy)
  • Embed licensing info directly into RSS feeds (easy)

From there, AI systems and crawlers have no more excuses. The terms are published, the pricing is machine-readable, and the licensing is fullon enforceable.

For SEOs and site owners, the benefits are clear:

  • Control over your content and how it is used in AI products
  • Compensation when your pages fuel someone else’s paying model
  • Transparency about what bots are allowed on and to do on your site

The Takeaway

Gosh, we are entering a stage where every site owner will need an AI licensing strategy as much as they need an SEO strategy. The RSL Standard and Collective give us a unified way to protect the value of our work and finally force AI companies to treat publishers as partners, not free training data.

Our final thought is this: how long will it be before Google says no access to your content, or you don’t get in the engine?