I used to be a food blogger many years ago and spent a fair bit of time learning the craft of SEO. In those days, discoverability meant optimizing for search engines - getting your content in front of human readers through clever keywords, backlinks, and structured content.

But as I set up a new blog today, the rules are changing. In these agentic times, the question isn’t just “How do humans discover my content?” but also “How do AI agents find and understand it?”

That’s why Dharmesh Shah’s recent post on Agent Index Optimization (AIO) landed exactly the right moment, especially with tools like Comet by Perplexity reshaping how people search. As AI agents increasingly become our intermediaries for search, learning, and decision-making, being agent-readable is becoming just as important as being human-readable.

---

Value Still Comes First

Dharmesh shares some practical guidance, but two points stood out for me:

1. It’s still about value.
Whether your content is discovered via search engines or AI agents, what ultimately matters is whether it delivers genuine value to people. Without substance, no optimization can save it.

2. Technical steps can help.
From improving your robots.txt to implementing llms.txt, and even setting up an MCP (Model Clarity Protocol) server, there are ways to make your content easier for agents to find, parse, and understand. These steps don’t replace value - they amplify it.

A big thank you to Dharmesh Shah for sharing timely, actionable insights. His work is invaluable for anyone navigating this new space.

---

The Future of Content Discovery

This is a space I’m increasingly curious about, particularly as it relates to the evolving AI economy and the democratization of AI.

In the search engine and browser era, monetization wasn’t built in from the start. It was added later through mechanisms like ads and platform-based engagement - what Ben Thompson calls the internet’s “Original Sin”.

In the agentic era, the challenge may be even greater. As AI agents become the primary interface between users and the web, traditional models of value creation and capture may no longer hold. We’ll need to be more intentional about how content is discovered, and how creators are credited, compensated, and trusted in this new paradigm.

Content that combines authentic value + technical clarity will stand the best chance of thriving.

---

Your Thoughts

How can creators ensure they remain visible, valuable, and fairly rewarded in an agent-driven web? I’d love to hear your ideas.