Page MenuHomePhabricator
Create Task
Maniphest T87140

Regularly run GenerateSitemap.php on Farsi Wikinews to improve Google crawling
Closed, DeclinedPublic
Actions

Description

https://fa.wikinews.org/sitemap.xml, or other URL defined at https://fa.wikinews.org/robots.txt, should contain a sitemap udpated regularly with generateSitemap.php. Any reason not to?

Context:

I've been writing articles for fawikinews, I realized my articles are not being indexed by google, latest article was indexed was for about a month ago, also we use GoogleNewsSitemap extension to generate news xml sitemaps using [[Special:NewsFeed]] so I added sitemap line to MediaWiki:Robots.txt but nothing happened, I also realized www.wikinews.org is redirecting to en.wikinews.org so I checked MediaWiki:Robots.txt on English wikinews, I saw people were adding sitemap links for frwikinews and eswikinews, so I asked an sysop to add a sitemap line for fawikinews, and right after that night I saw my articles are being shown in the first page of Google SERP.

I also read someone in mznwiki is telling some pages with the age of at least half year are not being indexed by google.

My question is do we use generateSitemap.php? where are the generated files located at? can we use wgSitemapNamespaces and wgSitemapNamespacesPriorities for them?

Related Objects
Search...

StatusSubtypeAssignedTask
 InvalidFeatureNoneT16720 robots.txt (tracking)
 DeclinedNoneT87140 Regularly run GenerateSitemap.php on Farsi Wikinews to improve Google crawling

Event Timeline

Mjbmr created this task.Jan 18 2015, 1:21 PM
Mjbmr raised the priority of this task from to Needs Triage.
Mjbmr updated the task description. (Show Details)
Mjbmr added a project: WMF-General-or-Unknown.
Mjbmr subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 18 2015, 1:21 PM
Aklapper added a comment.Jan 18 2015, 6:41 PM

so I asked an sysop to add a sitemap line for fawikinews, and right after that night I saw my articles are being shown in the first page of Google SERP.

Got any link to that change by a sysop handy, please?
What exactly is the request in this task? That the same change is performed for mzn.wikinews? Or is there still some issue with fa.wikinews?
Who is "we"? mzn? fa?

Mjbmr added a comment.EditedJan 19 2015, 4:30 AM

As I told the change was made at en.wikinews.org on [[MediaWiki:Robots.txt]], and I didn't say mzn.wikinews, where is no mzn.wikinews, mzn.wikipedia.org, and no not the same edit was made for any wikipedia project, no wikipedia is using GoogleNewsSitemap extension, it's only for news articles.

Yes, still there are problems, still only pages in main category which are using for GoogleNewsSitemap, are being indexed well.

What do you mean who is we? we, me, you, people with shell access on wikimedia projects, any other user who is contributing to these projects.

My request is, can we run generateSitemap.php for these projects?

Mjbmr added a comment.EditedFeb 8 2015, 5:13 AM

If start a new sitemap extension will wikimedia allow us enable it on our wikis?

Aklapper added a comment.Feb 9 2015, 12:41 AM

Thanks for pinging here! I unfortunately still don't fully understand what this ticket is about. It is a bit hard to read. It would be helpful if you could provide three sections:

  • Steps to reproduce (as a numbered list of steps, one per line)
  • Expected outcome
  • Actual outcome

with clear information and avoiding ambiguity. And generally, for any statements, links are very welcome. Thank you!

In T87140#1023255, @Mjbmr wrote:

If start a new sitemap extension will wikimedia allow us enable it on out wikis?

See https://www.mediawiki.org/wiki/Writing_an_extension_for_deployment - questions could probably be answered on its talk page.

jayvdb subscribed.Feb 15 2015, 4:42 AM
Aklapper changed the task status from Open to Stalled.Feb 16 2015, 10:09 AM
Aklapper triaged this task as Lowest priority.Feb 23 2015, 4:29 PM
Danny_B added a parent task: T16720: robots.txt (tracking).Feb 15 2016, 9:59 AM
Nemo_bis renamed this task from Google crawling to Add sitemap on Wikinews to improve Google crawling.Feb 19 2017, 12:11 PM
Nemo_bis added a project: SEO.
Nemo_bis updated the task description. (Show Details)
Nemo_bis mentioned this in T158506: Add a sitemap with all indexable placeholders on cywiki (Welsh Wikipedia).Feb 19 2017, 12:42 PM
Ciencia_Al_Poder subscribed.Jul 17 2018, 7:53 PM

Someone with access to google search console can add the Special:NewPages atom feed. This will help to index new articles created

See https://webmasters.googleblog.com/2014/10/best-practices-for-xml-sitemaps-rssatom.html

• Imarlier mentioned this in T199252: Search engines continue to link to JS-redirect destination after Wikipedia copyright protest.Jul 27 2018, 9:08 PM
• herron subscribed.Jul 30 2018, 5:47 PM
Ejs-80 subscribed.Dec 19 2018, 7:09 AM
Aklapper renamed this task from Add sitemap on Wikinews to improve Google crawling to Regularly run GenerateSitemap.php on Farsi Wikinews to improve Google crawling.May 25 2020, 3:41 PM
Aklapper changed the task status from Stalled to Open.
Aklapper added a project: Wikimedia-maintenance-script-run.
VulpesVulpes825 mentioned this in T54429: Canonical URL should include language variant.Jul 5 2020, 8:03 AM
VulpesVulpes825 mentioned this in T108443: Google doesn't honor canonical URLs of zh.wiki.
Pppery moved this task from Backlog to WMF Prod on the Wikimedia-maintenance-script-run board.Aug 19 2024, 8:00 PM
Restricted Application added a subscriber: Huji. · View Herald TranscriptAug 19 2024, 8:00 PM
Pppery subscribed.Aug 20 2024, 12:15 AM

Is this still wanted? Search engine stuff has changed a lot in the years since this task was filed.

Aklapper closed this task as Declined.Jan 13 2025, 1:46 PM

No reply so far; boldly declining.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits