Page MenuHomePhabricator

Ajax request to external URL from wikimedia website
Closed, DeclinedPublic

Description

Hello.

In middle east countries they use what called Hijri date (Islamic date). This date is supported in Mediawiki core. And can be retrieved using time function like this:

{{#time:xmj xmF xmY}}

There is a problem always with the hijri date in wikimedia websites. I use time function to retrieve the hijri date in any sisters wiki, but sometimes it retrieve the wrong date. And we have to delay, or advance the date each month. The problem with hijri date is that it depends on Moon move. And calculations doesn't work every time.

In arwiki, we've discussed this problem several times. And the best option we reach is to write a script (JavaScript) to retrieve the date from trusted website (https://hijri.today/). This website is very trusted, and it get the exact date, without any percentage of mistake. The solution is to make ajax request to the previous website, and then get the date from that ajax request, then inner it inside wikipedia template.

The problem is that when I make the ajax request to the previous website it always said:

Report Only] Refused to connect to 'https://hijri.today/' because it violates the following Content Security Policy directive: "default-src 'self' data: blob: upload.wikimedia.org https://commons.wikimedia.org meta.wikimedia.org *.wikimedia.org *.wikipedia.org *.wikinews.org *.wiktionary.org *.wikibooks.org *.wikiversity.org *.wikisource.org wikisource.org *.wikiquote.org *.wikidata.org *.wikivoyage.org *.mediawiki.org wikimedia.org". Note that 'connect-src' was not explicitly set, so 'default-src' is used as a fallback.
Access to XMLHttpRequest at 'https://hijri.today/' from origin 'https://ar.wikipedia.org' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.

Is there is any way to make ajax request to the previous website without facing these errors?

Event Timeline

ASammour created this task.Apr 3 2019, 6:19 PM
Restricted Application added subscribers: Liuxinyu970226, alanajjar, Aklapper. · View Herald TranscriptApr 3 2019, 6:19 PM
Aklapper closed this task as Declined.Apr 3 2019, 9:20 PM

Is there is any way to make ajax request to the previous website without facing these errors?

No, because passing every user's personal data without any warning to a random 3rd party websites violates all users' privacy.

If you have specific steps to reproduce a Hijri related problem in MediaWiki itself, please create a task with steps to reproduce, expected results, and actual results. Thanks!

Restricted Application removed a subscriber: Liuxinyu970226. · View Herald TranscriptApr 3 2019, 9:20 PM

(And for completeness only: Visiting that "trusted" website in a web browser tries to share my personal data with random websites like addthis.com, googlesyndication.com, googletagmanager.com and jquery.com.)

Reedy added a subscriber: Reedy.Apr 3 2019, 9:25 PM

While I completely agree with Andre's decline, and for the reasons... As he says, this is not going to happen, ever.

Can we get the javascript that that site uses (is it open source/freely licensed?) to integrate into MediaWiki? If we can, that is certainly doable (though, it would need porting to PHP) to help improve all Arabic language wikis

Bawolff added a subscriber: Bawolff.Apr 3 2019, 9:30 PM

The error is only a warning so it doesnt actually block things yet. But will soon. It is a violation of the privacy policy unless the user has specificly opted in.

So i can see two reasons why date might be wrong

  1. caching - the date gets saved for a little while. Putting {{CURRENTHOUR}} somewhere else on the page might help
  2. timezone being something other than what users expect
Krenair added a subscriber: Krenair.Apr 3 2019, 9:40 PM

@Aklapper You misunderstand the solution. My idea was to write a gadget, and any user can enable it. And if he enable it from the preferences, then he must agree that it may share some data with that website. And anyway, if that happened, it will be enabled for only very few users in arwiki (interface sysop).

@Reedy Unfortunately the website seems that it depends on PHP code. There is no evidence that it depends on JavaScript code to calculate the date. I've made a lot of research about this problem, and The only way in my opinion to solve this problem is by depending on 3rd party website. This can help also generate the calendars in Middle eastern wikis by defining the first day of hijri month, and count of days in the month; since these values changes every month depends on moon move.

@Bawolff It block the ajax request. Not only warning. Try to put this code in your browser, then press F12 to see the error in details.

$.ajax({
    type: "GET",
    url: "https://hijri.today/"
}).done(function (data) {
    alert ("Done");
});
Reedy added a comment.EditedApr 3 2019, 10:03 PM

@Reedy Unfortunately the website seems that it depends on PHP code. There is no evidence that it depends on JavaScript code to calculate the date. I've made a lot of research about this problem, and The only way in my opinion to solve this problem is by depending on 3rd party website. This can help also generate the calendars in Middle eastern wikis by defining the first day of hijri month, and count of days in the month; since these values changes every month depends on moon move.

So the same question still applies... Is their PHP code open source/freely licensed? Can you contact them to find out? Because sending the request to another site is not going to be an acceptable solution for Wikimedia wikis; the CSP blocks are only going to become stricter

This website doesn't have a contact email, or a form to reach them. Which is very strange!. But As I see from the footer, they said "All rights reserved ©"Personally I contact other websites that offer the same service (hijri date) Like this, but they refuse each time, or didn't reply at all.

@Reedy Unfortunately the website seems that it depends on PHP code. There is no evidence that it depends on JavaScript code to calculate the date. I've made a lot of research about this problem, and The only way in my opinion to solve this problem is by depending on 3rd party website.

Can you clarify under what circumstances the MW version is wrong? If MW actually is wrong (and not just sometimes outdated due to caching) then we should figure out what's going on, and fix it.

In terms of

The only way in my opinion to solve this problem is by depending on 3rd party website.

If you want to do in it a gadget, have you considered something like https://github.com/xsoh/moment-hijri ?

@Bawolff It block the ajax request. Not only warning. Try to put this code in your browser, then press F12 to see the error in details.

$.ajax({
    type: "GET",
    url: "https://hijri.today/"
}).done(function (data) {
    alert ("Done");
});

That's not related to CSP. That's just how the web works in general (The so-called same origin policy). You're not allowed to do GET requests to other websites unless the other website says its ok (via CORS). hijri.today has not set the CORS header to mark cross-site GET requests ok.

Eventually this type of request will be blocked by both the same origin policy and CSP. However, for the moment, CSP is just giving a warning, and same-origin policy is doing the actual blocking.

See https://en.wikipedia.org/wiki/Same-origin_policy for info on that

@Bawolff I will explain how hijri date is calculated.

In hijri date, there are 12 months. Each month has days between 29-30 days. And we can't know whether the number of days are 29, or 30 except if we look to the moon move. So in middle east countries they look to the moon in 28th night to see what is the status of the moon. If it's new moon, then this means that a new month will start.

For example, today is 28 Regeb 1440 (hijri date), and today night each middle eastern country will look at the moon at night (Most countries followed Saudi Arabia decision), and determine if tomorrow will be start of new month, or not. If it's the start of new month then this means that the current hijri month number of days will be 29, else it will be 30 days.

The difference between 29, or 30 cause all the mistakes and the differences. All the tools in the internet depends on mathematical calculations to determine the length of each month (To my knowledge). Except the two websites I've listed before. The advantage of these websites is that it's updated each day, very trusted, and it's operated by special institutions that focus on hijri date calendars, pray times...etc.

The problem with MW is in Language.php file. Specially the tsToHijri function. This is where hijri date is calculated.

@ASammour: The solution would be enhancing https://phabricator.wikimedia.org/source/mediawiki/browse/master/languages/Language.php$1676 by writing additional code to somehow calculate if it's 29 or if it's 30, it seems. (How to do that might be a tough question though?)

Can you please file a new task under MediaWiki-Internationalization , provide clear steps that allow someone else reproduce the problem, add that link to tsToHijri, explain what is the current result/behavior, and explain which result/behavior you would expect? See https://www.mediawiki.org/wiki/How_to_report_a_bug - Thanks!

@Bawolff I will explain how hijri date is calculated.
In hijri date, there are 12 months. Each month has days between 29-30 days. And we can't know whether the number of days are 29, or 30 except if we look to the moon move. So in middle east countries they look to the moon in 28th night to see what is the status of the moon. If it's new moon, then this means that a new month will start.
For example, today is 28 Regeb 1440 (hijri date), and today night each middle eastern country will look at the moon at night (Most countries followed Saudi Arabia decision), and determine if tomorrow will be start of new month, or not. If it's the start of new month then this means that the current hijri month number of days will be 29, else it will be 30 days.
The difference between 29, or 30 cause all the mistakes and the differences. All the tools in the internet depends on mathematical calculations to determine the length of each month (To my knowledge). Except the two websites I've listed before. The advantage of these websites is that it's updated each day, very trusted, and it's operated by special institutions that focus on hijri date calendars, pray times...etc.
The problem with MW is in Language.php file. Specially the tsToHijri function. This is where hijri date is calculated.

Thank you. I wasn't aware of that. I understand why you are saying it can't be determined algorithmicly.

I think the best solution (in the short term) would be instead of using ajax in a gadget, use a bot running on toolforge, that once a day will update a template on arwiki for the correct date.

ASammour added a comment.EditedApr 4 2019, 7:09 PM

@Bawolff This is exactly what I do right now. But my idea is to make the process less reliable than one tool. That's why I think gadget is very good solution to this.

sbassett triaged this task as Medium priority.Oct 16 2019, 4:36 PM
sbassett moved this task from Backlog to Done on the Privacy board.