{"id":3938,"date":"2021-01-24T22:15:59","date_gmt":"2021-01-24T22:15:59","guid":{"rendered":"https:\/\/shopthemedetector.com\/blog\/?p=3938"},"modified":"2026-10-11T10:35:38","modified_gmt":"2026-10-11T10:35:38","slug":"how-to-access-robot-txt-on-shopify","status":"publish","type":"post","link":"https:\/\/shopthemedetector.com\/blog\/how-to-access-robot-txt-on-shopify\/","title":{"rendered":"How to Access Robots TXT on Shopify"},"content":{"rendered":"

Shopify\u2019s robots.txt is the file that tells search engines and AI crawlers which pages they can visit and which they should skip. Store owners used to see the generated version and nothing more, with no way to adjust it. Since June 2021 that has changed, and the file is now editable through the theme code. The real question is not where it lives, but which rules a Shopify store should actually set, and whether that extends to the newer bots that read llms.txt and agents.md.<\/p>\n

Key Takeaways<\/div>
1<\/div>
Shopify generates robots.txt automatically, and since June 2021 you can override it by editing robots.txt.liquid in the theme\u2019s Assets folder.<\/div><\/div>
2<\/div>
Robots.txt controls crawling. Noindex rules and the XML sitemap control indexing. Running all three in sync is what prevents duplicate pages and filtered variants from eating crawl budget.<\/div><\/div>
3<\/div>
AI crawlers like GPTBot, PerplexityBot and ClaudeBot read llms.txt and agents.md, two newer files that sit in the same site root and extend what robots.txt does.<\/div><\/div><\/div>\n

Why Shopify Hides the Robots.txt File<\/h2>\n

Shopify locks store owners out of FTP, and robots.txt sits inside the same restricted file system. The reason is security and product integrity: if the files were open, anyone could probe them for weaknesses or alter the proprietary software behind the scenes. For most of Shopify\u2019s history, that meant robots.txt was fixed, visible at \/robots.txt but not editable.<\/p>\n

Shopify changed that in June 2021 by adding robots.txt.liquid, a template file in the theme\u2019s Assets folder. The URL still serves a generated robots.txt, but the contents are now whatever rules you add to that template.<\/p>\n

How to Edit robots.txt.liquid Step by Step<\/h2>\n

The process takes about a minute. The file lives in the live theme, so changes take effect the moment you save.<\/p>\n

    \n
  1. Open your Shopify admin and go to Online Store \u2192 Themes<\/strong>.<\/li>\n
  2. On your live theme, click Actions \u2192 Edit code<\/strong>.<\/li>\n
  3. In the left panel, scroll to the Templates<\/strong> section and click Add a new template<\/strong>. From the dropdown, choose robots.txt<\/strong>, and Shopify will create robots.txt.liquid<\/code> for you with the default rules already inside.<\/li>\n
  4. Edit the Liquid template to add or remove rules, then click Save<\/strong>. The live file at yourstore.com\/robots.txt<\/code> updates right away.<\/li>\n<\/ol>\n

    If you break something, delete the template file and Shopify will go back to serving its default rules. There is no risk of permanently locking yourself out.<\/p>\n

    What to Control: Sitemap, Noindex and Crawl Budget<\/h2>\n

    Robots.txt is a crawling instruction, not an indexing instruction, and the two get confused often. Blocking a URL in robots.txt stops Google from fetching the page, but it does not reliably remove the URL from search results if the page has links pointing to it. The right pattern is to combine three signals: robots.txt for crawling, a noindex meta tag or header for indexing, and an XML sitemap for the pages you want found.<\/p>\n

    On larger Shopify catalogs, that balance matters. Filtered collection URLs, tag pages and the many variant URLs a product can generate all share the same site crawl budget as your real pages, and sloppy rules leave Google reading junk instead of your products.<\/p>\n

    The apps that handle this well do it as one connected job rather than three unrelated features. Sitemap NoIndex Pro SEO<\/a> is one of them, pairing XML and HTML sitemap control with per-page noindex rules so a merchant can see which pages the signals cover in one place. Its own adoption data shows the kind of stores that reach for it:<\/p>\n

    0.4%<\/div>
    Share of stores in our dataset<\/div><\/div>
    12.8<\/div>
    Avg apps on their stores<\/div><\/div><\/div>

    Based on Shopify stores in our dataset \u00b7 Last updated October 12, 2026.<\/p>\n