Shopify Robots.txt File

For the past 3 months I’ve been developing, building, populating, and promoting a Shopify ecommerce store. I transitioned from a self hosted WordPress website, utilizing Premium Press to sell product to Shopify. The transaction has been as smooth as can be expected, but it’s caused me to raise a lot of questions. Specifically, it’s forced me to ask the question, why is Shopify disallowing my Blogs in the Robots.txt file?

Shopify Logo

For the past 3 months I’ve been developing, building, populating, and promoting a Shopify ecommerce store.  I transitioned from a self hosted WordPress website, utilizing Premium Press to sell product to Shopify.  The transaction has been as smooth as can be expected, but it’s caused me to raise a lot of questions.  Specifically, it’s forced me to ask the question, why is Shopify disallowing my Blogs in the Robots.txt file?

Thankfully, the answer is that I just don’t know what I’m talking about.  I simply do not know as much about web marketing and SEO as I thought I did.  The Robots.txt file on all Shopify stores looks incorrect, but it’s done properly to save your ecommerce store from yourself.  Allow me to explain why it’s important to block Shopify blogs and collections in the Robots.txt file.

Many shops use blogs and collections, along with hundreds of Tags.  These tags were creating duplicate content for stores and unneeded server strain on Shopify.  The answer is to simply disallow bots from crawling all the tags repeatedly.

Shopify uses a qualifier within the Robots.txt to allow Google to index and search your collections and blog, it just doesn’t allow it on the tags.  The Robots.txt file change on Shopify stores happened right around January 2014.  The following lines were added:

Disallow: /collections/*+*
Disallow: /blogs/*+*

We have an official Shopify response to the change quoted below:

Why

Most shops use tags to tag products and blog articles. Shops can have hundreds or even thousands of tags. When Google found these tags, it would begin querying Shopify repeatedly using every possible combination of tags. It’s similar in nature to a denial of service attack.

What’s the Impact?

Google still sees all of your shop’s collections and blog articles. This patterns “*+*” simply prevent Google from indexing every possible permutation of tags in the URLs listed above.

TL;DR

Great SEO, no Google spam, less load on your shop.

Unfortunately, even if you wanted to allow Google all over your Tags and various collections, you cannot edit your Shopify robots.txt file.  Part of the package is having limited control over your ecommerce store.  I personally feel that what could be perceived as a set back is ultimately worth having a protected, constantly updated, and solid ecommerce solution.  I am happy with my move to Shopify, but time will tell if it was the best decision to make in regards to SEO.

If you are considering signing up to Shopify, please use my referral link: Free Trial of Shopify

You get a free trial and I get a free month of service.  It’s a win-win-win.

Leave a Reply

Your email address will not be published. Required fields are marked *