Page MenuHomePhabricator

Add sitemap on Wikinews to improve Google crawling
Open, Stalled, LowestPublic

Description

https://fa.wikinews.org/sitemap.xml, or other URL defined at https://fa.wikinews.org/robots.txt, should contain a sitemap udpated regularly with generateSitemap.php. Any reason not to?


Context:

I've been writing articles for fawikinews, I realized my articles are not being indexed by google, latest article was indexed was for about a month ago, also we use GoogleNewsSitemap extension to generate news xml sitemaps using [[Special:NewsFeed]] so I added sitemap line to MediaWiki:Robots.txt but nothing happened, I also realized www.wikinews.org is redirecting to en.wikinews.org so I checked MediaWiki:Robots.txt on English wikinews, I saw people were adding sitemap links for frwikinews and eswikinews, so I asked an sysop to add a sitemap line for fawikinews, and right after that night I saw my articles are being shown in the first page of Google SERP.

I also read someone in mznwiki is telling some pages with the age of at least half year are not being indexed by google.

My question is do we use generateSitemap.php? where are the generated files located at? can we use wgSitemapNamespaces and wgSitemapNamespacesPriorities for them?

Event Timeline

Mjbmr created this task.Jan 18 2015, 1:21 PM
Mjbmr raised the priority of this task from to Needs Triage.
Mjbmr updated the task description. (Show Details)
Mjbmr added a subscriber: Mjbmr.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 18 2015, 1:21 PM

so I asked an sysop to add a sitemap line for fawikinews, and right after that night I saw my articles are being shown in the first page of Google SERP.

Got any link to that change by a sysop handy, please?
What exactly is the request in this task? That the same change is performed for mzn.wikinews? Or is there still some issue with fa.wikinews?
Who is "we"? mzn? fa?

Mjbmr added a comment.EditedJan 19 2015, 4:30 AM

As I told the change was made at en.wikinews.org on [[MediaWiki:Robots.txt]], and I didn't say mzn.wikinews, where is no mzn.wikinews, mzn.wikipedia.org, and no not the same edit was made for any wikipedia project, no wikipedia is using GoogleNewsSitemap extension, it's only for news articles.

Yes, still there are problems, still only pages in main category which are using for GoogleNewsSitemap, are being indexed well.

What do you mean who is we? we, me, you, people with shell access on wikimedia projects, any other user who is contributing to these projects.

My request is, can we run generateSitemap.php for these projects?

Mjbmr added a comment.EditedFeb 8 2015, 5:13 AM

If start a new sitemap extension will wikimedia allow us enable it on our wikis?

Thanks for pinging here! I unfortunately still don't fully understand what this ticket is about. It is a bit hard to read. It would be helpful if you could provide three sections:

  • Steps to reproduce (as a numbered list of steps, one per line)
  • Expected outcome
  • Actual outcome

with clear information and avoiding ambiguity. And generally, for any statements, links are very welcome. Thank you!

If start a new sitemap extension will wikimedia allow us enable it on out wikis?

See https://www.mediawiki.org/wiki/Writing_an_extension_for_deployment - questions could probably be answered on its talk page.

jayvdb added a subscriber: jayvdb.Feb 15 2015, 4:42 AM
Aklapper changed the task status from Open to Stalled.Feb 16 2015, 10:09 AM
Aklapper triaged this task as Lowest priority.Feb 23 2015, 4:29 PM
Nemo_bis renamed this task from Google crawling to Add sitemap on Wikinews to improve Google crawling.Feb 19 2017, 12:11 PM
Nemo_bis added a project: SEO.
Nemo_bis updated the task description. (Show Details)

Someone with access to google search console can add the Special:NewPages atom feed. This will help to index new articles created

See https://webmasters.googleblog.com/2014/10/best-practices-for-xml-sitemaps-rssatom.html

herron added a subscriber: herron.Jul 30 2018, 5:47 PM
Ejs-80 added a subscriber: Ejs-80.Dec 19 2018, 7:09 AM