Page MenuHomePhabricator

Move ZhConversion conversion tables from static class to array file
Open, Needs TriagePublic

Description

languages\data\ZhConversion.php contains a single class that holds 5 static arrays. These arrays can be converted to 5 separate PHP array files.

Event Timeline

siebrand created this task.Mar 10 2016, 4:39 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 10 2016, 4:39 PM

Does it really matter? Currently the php file is built by maintenance tool anyway.

Does it really matter? Currently the php file is built by maintenance tool anyway.

[Citation needed]

[Citation needed]

In languages\data\ZhConversion.php

* Automatically generated using code and data in maintenance/language/zhtable/
* Do not modify directly!
Liuxinyu970226 added a comment.EditedApr 27 2016, 9:30 AM

[Citation needed]

In languages\data\ZhConversion.php

* Automatically generated using code and data in maintenance/language/zhtable/
* Do not modify directly!

Sorry for my 1 month delaying. But I'm doubt if such 2 lines could be a reason to hold up this task. Could you please tell me more background informations? Such as "what is you 'maintenance tool' mentioned?" and "Why those very codes can't be converted to JSON?"

Sorry for my 1 month delaying. But I'm doubt if such 2 lines could be a reason to hold up this task. Could you please tell me more background informations? Such as "what is you 'maintenance tool' mentioned?" and "Why those very codes can't be converted to JSON?"

https://github.com/wikimedia/mediawiki/tree/master/maintenance/language/zhtable

I would agree that a json dump is a better alternative than this hack. Is a CDB cache necessary for large JSON decodings?

@Liuxinyu970226 It is not that these codes cannot be converted to JSON. Instead, there's no reason to convert them to JSON. They are statically generated and no one is supposed to change it manually. By keeping them in PHP files, we can actually gain performance from OPcache. If we made them JSON, an extra step of JSON parsing will be required, or extra caching will be needed to compensate the performance degradation.

This comment was removed by Liuxinyu970226.

@Liuxinyu970226 It is not that these codes cannot be converted to JSON. Instead, there's no reason to convert them to JSON. They are statically generated and no one is supposed to change it manually. By keeping them in PHP files, we can actually gain performance from OPcache. If we made them JSON, an extra step of JSON parsing will be required, or extra caching will be needed to compensate the performance degradation.

If you are asking why to do it, See Our Documentation on Registration, thanks for regarding

Keeping the giant arrays in PHP makes sense for performance reasons (opcode caching). But I believe the initial motivation behind this task was to make the data more reusable so non-PHP applications can use it. Since the data is generated by a script, modifying it to also output JSON shouldn't be too difficult. (But that's more of a Librarization thing I think)

Shizhao added a subscriber: Shizhao.EditedJul 6 2017, 12:39 PM

Whether it can be converted to JSON for more usability, and at the same time keep arrays?

Such as adding a maintenance script converted to JSON?

Given these arrays are auto-generated and not maintained by hand, it is unclear to me why JSON would be preferable. In terms of readability a PHP array and JSON array are about equally easy to read.

In terms of performance and architecture consistency, we are actually standardising on using arrays, not JSON. See T129499 for details.

I'll leave this open because while it does use a PHP array and is fine for performance, for API consistency it might still make sense to convert to a static array file that returns the array (with the public API separate from the data file, e.g. the class method that returns the array should be separate).

@Krinkle So the focus of this task should actually be to dump the static properties of the class into one/multiple php array files instead of json?

@RazeSoldier Yes, I believe that would be in line with the larger direction of how we store data on disk for MediaWiki.

Krinkle renamed this task from Change ZhConversion conversion tables to JSON to Move ZhConversion conversion tables from static class to array file.Dec 22 2018, 1:20 AM

It seems that this task is a bit complicated to implement for me, because we need to rebuild the maintenance script written in python. Maybe we can coding a php wrapper that wraps the python script first.

[Citation needed]

In languages\data\ZhConversion.php

* Automatically generated using code and data in maintenance/language/zhtable/
* Do not modify directly!

Sorry for my 1 month delaying. But I'm doubt if such 2 lines could be a reason to hold up this task. Could you please tell me more background informations? Such as "what is you 'maintenance tool' mentioned?" and "Why those very codes can't be converted to JSON?"

The php file is automatically generated by maintenance/anguage/zhtable/Makefile.py . If you decide to converted it into 5 separate JSON files, please rewrite this python script, otherwise we will be unable to maintain it.

RazeSoldier updated the task description. (Show Details)Dec 22 2018, 6:39 AM