Shopify’s robots.txt is the file that tells search engines and AI crawlers which pages they can visit and which they should skip. Store owners used to see the generated version and nothing more, with no way to adjust it. Since June 2021 that has changed, and the file is now editable through the theme code. The real question is not where it lives, but which rules a Shopify store should actually set, and whether that extends to the newer bots that read llms.txt and agents.md.

Key Takeaways
1
Shopify generates robots.txt automatically, and since June 2021 you can override it by editing robots.txt.liquid in the theme’s Assets folder.
2
Robots.txt controls crawling. Noindex rules and the XML sitemap control indexing. Running all three in sync is what prevents duplicate pages and filtered variants from eating crawl budget.
3
AI crawlers like GPTBot, PerplexityBot and ClaudeBot read llms.txt and agents.md, two newer files that sit in the same site root and extend what robots.txt does.

Why Shopify Hides the Robots.txt File

Shopify locks store owners out of FTP, and robots.txt sits inside the same restricted file system. The reason is security and product integrity: if the files were open, anyone could probe them for weaknesses or alter the proprietary software behind the scenes. For most of Shopify’s history, that meant robots.txt was fixed, visible at /robots.txt but not editable.

Shopify changed that in June 2021 by adding robots.txt.liquid, a template file in the theme’s Assets folder. The URL still serves a generated robots.txt, but the contents are now whatever rules you add to that template.

How to Edit robots.txt.liquid Step by Step

The process takes about a minute. The file lives in the live theme, so changes take effect the moment you save.

  1. Open your Shopify admin and go to Online Store → Themes.
  2. On your live theme, click Actions → Edit code.
  3. In the left panel, scroll to the Templates section and click Add a new template. From the dropdown, choose robots.txt, and Shopify will create robots.txt.liquid for you with the default rules already inside.
  4. Edit the Liquid template to add or remove rules, then click Save. The live file at yourstore.com/robots.txt updates right away.

If you break something, delete the template file and Shopify will go back to serving its default rules. There is no risk of permanently locking yourself out.

What to Control: Sitemap, Noindex and Crawl Budget

Robots.txt is a crawling instruction, not an indexing instruction, and the two get confused often. Blocking a URL in robots.txt stops Google from fetching the page, but it does not reliably remove the URL from search results if the page has links pointing to it. The right pattern is to combine three signals: robots.txt for crawling, a noindex meta tag or header for indexing, and an XML sitemap for the pages you want found.

On larger Shopify catalogs, that balance matters. Filtered collection URLs, tag pages and the many variant URLs a product can generate all share the same site crawl budget as your real pages, and sloppy rules leave Google reading junk instead of your products.

The apps that handle this well do it as one connected job rather than three unrelated features. Sitemap NoIndex Pro SEO is one of them, pairing XML and HTML sitemap control with per-page noindex rules so a merchant can see which pages the signals cover in one place. Its own adoption data shows the kind of stores that reach for it:

0.4%
Share of stores in our dataset
12.8
Avg apps on their stores

Based on Shopify stores in our dataset · Last updated October 12, 2026.

AI Crawlers: agents.md and llms.txt

Google’s bot is no longer the only crawler that reads the root of your site. GPTBot (OpenAI), PerplexityBot, ClaudeBot (Anthropic), Google-Extended and a handful of smaller indexers now visit stores to decide what to include in AI answer results. Two newer files have appeared alongside robots.txt to speak to them: llms.txt, a short text summary of what the site is and what pages are worth citing, and agents.md, a markdown version with the same idea.

Shopify does not generate either file by default. You can serve them yourself by adding a page or an asset at the root, or by running an app that writes them for you. Sitemap NoIndex Pro SEO holds a 5.0/5 rating across 193 merchant reviews precisely because it ships the AI-SEO layer most of the older SEO apps have not written code for yet.

Robots.txt is one piece of a larger picture. Our guide to on-page optimization for Shopify stores covers the surrounding settings that actually shift rankings.