Let us do a multiplication together, and then we can talk.

You have a clothing shop online. You add a colour filter, ten options. A size filter, six. A brand filter, fifteen. A price filter, four ranges. And sorting, three ways.

Ten times six times fifteen times four times three. Ten thousand eight hundred combinations. And each one of them, if your shop works like most, has its own URL. Ten thousand URLs, from a store that perhaps sells three hundred products.

Now add that nobody ever told you, because the platform does it by itself. Am I wrong?

What Google itself says about this

You do not have to take my word for it, there is a dedicated guide. And its wording is harsher than mine: faceted navigation “can generate infinite URL spaces”, and the problem is that “crawlers can’t determine whether the URLs are going to be useful without crawling first”.

Hold on to that last part, because it is the whole matter. The crawler does not know in advance that the page for “red, size 42, price 20 to 40, sorted by popularity” is worth nothing. It has to go, download it, look at it, and only then decide. And by then it has already spent the time.

Google calls this overcrawling and spells out the consequence: crawlers “will typically access a very large number of faceted navigation URLs” before working out they are useless, which produces slower discovery crawls, because “crawling is spent on useless URLs, the crawlers have less time to spend on new, useful URLs”.

Translated into your terms: you upload thirty new products on Monday morning, and Google discovers them on Thursday, because Monday and Tuesday it was downloading filter combinations no human will ever see.

The myth that has to die first

Before the fixes, let us clear away the worry I always hear: “Google will penalise me for duplicate content.”

No. Google does not penalise duplicate content, it consolidates it. Its canonicalisation documentation puts it calmly: if you do not specify a canonical URL, “Google will identify which version of the URL is objectively the best version to show to users in Search”.

So there is no penalty. There is something less dramatic and equally annoying: it decides instead of you. And it may pick “red shoes, size 41” as the representative page, where you wanted the “shoes” category to show.

The problem, then, is not punishment. It is waste and loss of control. Which is worse, if you ask me, because it never appears anywhere as a warning.

What it actually recommends

The guide gives concrete things, and there are fewer of them than you fear.

Cut the crawling off in robots.txt. The suggestion is to disallow crawling of the URLs your filters generate, and to allow only the individual product pages along with a dedicated listing page. It is the most drastic and the most effective.

The canonical helps, but do not expect it tomorrow. Notice how carefully Google words it: using rel="canonical" “may, over time, decrease the crawl volume of non-canonical versions”. Not “stops”, but “may, over time”. It is a signal, not a switch. The crawler will keep coming, just less and less often.

If you do want some filters indexed, write them properly. Use & as the separator, the industry standard. Google notes that characters like comma, semicolon and brackets are hard to detect for crawlers. And keep the logical order of filters constant, with no duplicates. The same user choice should always produce exactly the same URL, not colour-then-size one time and the reverse the next.

And the most neglected one: 404 on the empties. When a filter combination returns no products, the guide asks for a 404 status code, not a redirect to an error page. Most platforms serve a cheerful “no products found” page with a 200 status, which tells Google “all good here, I have content”. And Google believes them, and comes back.

If you read an old guide, it will tell you to configure your parameters in the URL Parameters tool in Search Console.

Do not go looking. It does not exist. Google announced its retirement in March 2022 and the tool went offline on 26 April 2022. Any text recommending it is at least four years old, and it is probably not the only old thing in there.

I mention it because it is a useful test. When you read SEO advice, check whether it references a tool that has been retired. It is the fastest way to tell whether the text was written this year or copied from last year, which had been copied from the year before.

What you do tomorrow morning, in order

You do not need to do all of it. You need to know how big your problem is before you spend anything.

Open Search Console and look at how many of your pages Google has indexed. If your shop has three hundred products and it tells you five thousand pages, you have found the problem without looking further. Then look at the shape of the URLs it has collected: if they are full of ?color=, &size=, &orderby=, you know where they came from.

Then decide the one thing that is genuinely a decision: which filters deserve to exist in search. Because some do. “Women’s running shoes” is a real search that real people make, and if your shop has a page for it, good. “Red, size 41, sorted by price descending” has never been searched by anybody.

The job is to keep the first kind and close the second. It is not technically hard. It is a commercial decision made by someone who understands both sides.

Which brings us back to the multiplication at the start. Ten thousand eight hundred pages from three hundred products. It is not your shop’s mistake, it is the platform’s default. Somebody just has to change it. If you want to see how many pages yours is generating today and which of them are worth keeping, get in touch.