How to Get Your Business Recommended by Claude
A homeowner in Huntington has water coming through the kitchen ceiling at nine at night. Instead of scrolling a page of ads, she types the problem into Claude and asks who she should call. A few seconds later she has an answer with a couple of company names in it and a set of links under them. If your business does that work in that town and you aren’t in the answer, she’ll never know you exist.
Owners now ask us how to be the name in that answer. As an SEO company on Long Island and in Ventura County, we start with what Anthropic, the company that makes Claude, has actually published about how Claude reads the web. It’s shorter than you’d think, and most of it comes down to settings on your own site.
When Claude goes to the live web
Claude doesn’t always answer from memory. Anthropic’s help article on web search says: When you ask about topics that benefit from current information, Claude invokes a search tool to inform and ground its generated responses with content from the live web. Every response includes citations, so you can easily verify sources yourself.
The same article says that when Claude searches, it gives a response that includes Direct citations to sources
, Source links for further reading
and Relevant quotes when appropriate
.
Anthropic’s developer documentation for the web search tool lays out the loop in three steps:
- Decides to search
Claude determines when to search based on the prompt.
- Reads the results
The API runs the searches and provides Claude with the results. This process can repeat multiple times throughout a single request.
- Answers with sources
At the end of its turn, Claude provides a final response with cited sources.
Source: Claude API documentation, "Web search tool." That page describes the developer tool. For the Claude app, Anthropic's help center says: If you have the new Claude experience, there's no web search toggle. Claude searches the web when it helps.
So the useful question is whether Claude, when it searches for your customer’s question, can reach a page of yours and has a reason to cite it. We won’t guess at which search engine sits behind those results. What Anthropic does spell out is how its own crawlers work, and that part is in your hands.
Three Anthropic bots, three different jobs
Anthropic’s crawler help article says it uses different robots to enable website owner transparency and choice
and lists three of them. Each one can be blocked on its own in robots.txt, and each has a different effect when you do.
-
TrainingClaudeBotWhat it does
ClaudeBot helps enhance the utility and safety of our generative AI models by collecting web content that could potentially contribute to their training.
If you block itWhen a site restricts ClaudeBot access, it signals that the site's future materials should be excluded from our AI model training datasets.
-
Your customer's questionClaude-UserWhat it does
When individuals ask questions to Claude, it may access websites using a Claude-User agent.
If you block itDisabling Claude-User on your site prevents our system from retrieving your content in response to a user query, which may reduce your site's visibility for user-directed web search.
-
Search qualityClaude-SearchBotWhat it does
Claude-SearchBot navigates the web to improve search result quality for users.
If you block itDisabling Claude-SearchBot on your site prevents our system from indexing your content for search optimization, which may reduce your site's visibility and accuracy in user search results.
Source: Claude Help Center, "Does Anthropic crawl data from the web, and how can site owners block the crawler?"
Look at the right-hand column of each badge. Anthropic describes blocking ClaudeBot only in terms of training data. Blocking Claude-User or Claude-SearchBot is the one it says may reduce your visibility in search. A business that wants to be recommended should make sure those two aren’t shut out.
Anthropic also says its Bots respect “do not crawl” signals by honoring industry standard directives in robots.txt
, and that a block goes in your top-level directory
and should be repeated for every subdomain that you wish to opt out from.
In practice that means one file, at your domain followed by /robots.txt, and you can open it in any browser.
User-agent: * Allow: /User-agent: Claude-User Disallow: /
User-agent: Claude-SearchBot Disallow: /
User-agent: * Allow: /User-agent: ClaudeBot Disallow: /
Sitemap: https://yoursite.com/sitemap.xml
Whether to block ClaudeBot is a fair business decision, and plenty of site owners make it. The trouble starts when a line nobody chose ends up in the file: a copied “block all AI bots” list, a plugin default, a leftover from an old developer. That’s worth five minutes of your time today.
The noindex tag and Claude's search partners
Robots.txt isn’t the only setting that matters. Anthropic’s article on blocking and removing content explains how a site owner keeps pages out of Claude’s web search, which tells you exactly what not to do by accident. It says: The noindex robots meta tag is a rule that tells our partners not to index your content so that they don’t send it to us in response to your web search query.
And it is plain about the result: the content will not appear in Claude outputs that use web search.
A noindex tag is one line in a page’s code that nobody sees. It’s normal on a test site, and it sometimes survives the move to the live one. If it’s sitting on your services page, Claude’s web search won’t show that page, whatever it says.
The same article covers photos and video: Our search partners only index images and videos that their bots are allowed to crawl.
If your project photos sit in a folder your robots.txt blocks, they can’t show up either.
Write pages Claude can quote back
Once Claude can reach your site, it still needs something to cite. In the developer documentation, Anthropic says Citations are always enabled for web search
, and each citation carries the source’s URL, its title and Up to 150 characters of the cited content
. That’s about the length of one sentence.
We read that as practical writing advice, not a rule Claude announces: put the answer to a customer’s question in one plain sentence, near the top of the section that covers it, with your service and your town in it. “We repair flat roofs in Suffolk County year round, including winter patch jobs” can be lifted and cited. “Quality you can trust since day one” can’t, because it doesn’t answer anything. Building pages and short videos around the questions customers actually ask is what our content and video work is for.
Location matters too. Anthropic’s web search article lists this under limitations: Claude may use your location (inferred from your IP address) when responding to a request for localized results.
When a customer asks Claude who to call, the town they’re sitting in can shape the answer, so your pages should say plainly which towns you serve. Keeping your name, address, phone and service area the same everywhere they appear is everyday local SEO, and it gives any search a consistent set of facts to work from.
Ask Claude who it would tell you to hire
You can see where you stand in about ten minutes. Anthropic’s help article says: If you want to ensure that Claude uses web search, include “Search the web” or “Use web search” in your prompt to Claude.
- Ask like a customer. Start with "Search the web," then ask who to hire for your service in your town. Try three or four wordings, the way people actually talk.
- Write down every link. Claude's answers with web search include citations. Note each site it cites and the exact page, not just the business name.
- Read the cited pages. Find the sentence Claude leaned on. Ask whether any page on your site says the same thing as clearly.
- Check your own settings. Open your robots.txt and look for Claude-User and Claude-SearchBot. Ask whoever built your site whether any key page carries a noindex tag.
Run it again every month or two and keep the list. When your pages start showing up as cited sources, you’ll see it there first. Tracking that across Claude, ChatGPT, Gemini and the rest is the heart of our AI search visibility service.
If someone sells you a “Claude ranking” package and can’t tell you which of those four things it changes, that’s BSEO: paying for something and not knowing what you bought.
Sources: Claude Help Center, "Enable and use web search," "Does Anthropic crawl data from the web, and how can site owners block the crawler?" and "Report, block, and remove content from Claude"; Claude API documentation, "Web search tool."
Send us your website. We'll check your robots.txt for Anthropic's bots, look for stray noindex tags on the pages that matter, and show you the searches you're found for and the ones you're missing. No charge, and no obligation. Or call or text 805.243.8321.
Get My Free Market AssessmentClear pages, open settings and consistent business details are the same groundwork that good SEO built on what search engines actually publish has always needed.