Old topics published via WordPress embed are missing x-robots: noindex and canonical tags

Hi everyone,

I’ve noticed a strange behavior with old topics that were automatically published from WordPress to Discourse (to be used as the comment section).

Normally, when a post is published this way, Discourse correctly adds the X-Robots-Tag: noindex to the HTTP header and sets the canonical URL pointing back to the WordPress blog post.

However, I discovered that older topics are losing these tags. The noindex header disappears, and the canonical tag is no longer present. Here are some examples of topics where this is happening:

• https://tecnoblog.net/comunidade/t/como-colocar-musica-no-status-do-whatsapp/73608
• https://tecnoblog.net/comunidade/t/como-compartilhar-a-internet-com-outro-celular-ou-pc/79015
• https://tecnoblog.net/comunidade/t/como-usar-a-netflix-em-outro-pais-viagem-ou-mudanca/74248
• https://tecnoblog.net/comunidade/t/como-usar-o-google-docs-7-super-dicas-e-truques-rapidos/76598
• https://tecnoblog.net/comunidade/t/como-atender-chamadas-do-seu-celular-android-no-windows-10/76615

Does anyone know a way to fix this issue?

Keep in mind that I have no way of knowing exactly how many topics have been affected so far, but it looks like it’s quite a few.

It would be great if there were a checkbox in the category (or tag?) settings that, when enabled, would automatically add noindex to all topics published under that category. Something like:

[ ] Hide Topics from this category in search results.

Hey Thiago,

Just so I understand, are you saying the following:

1. You have the site setting Embed set canonical URL enabled, and have always had it enabled.
2. You have multiple topics published from your Wordpress to your Discourse via the Wordpress Discourse Plugin over a period of some time.
3. Up until recently, all of the Discourse topics published as described in 2 all had a a link rel="canonical" with the href set to the wordpress url in their head.
4. At some point recently, you think some subset of those topics which previously conformed to 3 now have a link rel="canonical" with the href set to the Discourse url.

Is that correct?

Hi Angus!

Yes, that’s it.

The `embed set canonical URL` is enabled, too:

Screenshot 2026-05-18 at 13.55.191914×2372 415 KB

You can see here that new topics are published with the noindex + canonical tags. But i’m seeing old topics without these tags.

Thiago, if you have access to the server, could you please get the id of a topic where the canonical url is not working, run the following in the rails console and share the result.

./launcher enter app
rails c
TopicEmbed.with_deleted.find_by(topic_id: add the topic id here)

discourse(prod)> TopicEmbed.with_deleted.find_by(topic_id:73608)
=> nil
discourse(prod)> TopicEmbed.with_deleted.find_by(topic_id:79015)
=> nil
discourse(prod)> TopicEmbed.with_deleted.find_by(topic_id:74248)
=> nil
discourse(prod)> TopicEmbed.with_deleted.find_by(topic_id:76598)
=> nil

That’s the issue. In order for the canonical url for embeds feature to work, the topic needs a topic_embed record. Can you think of any reason why those topics may not have embed records?

I honestly don’t know what could have caused those topics to miss their topic_embed records.

But looking at the bigger picture, wouldn’t it make more sense to go with the configuration I suggested earlier? If we add a checkbox directly in the category settings to apply noindex to all topics within it, we wouldn’t have to rely on the embed feature or worry about whether those records exist in the first place.