Page MenuHomePhabricator
Create Task
Maniphest T184712

Page.title(as_url=True) should encode also slashes
Open, LowPublic
Actions

Description

Page.title(as_url=True) currently does not encode the slash (/) character, because that is a default exception in the underlying quote_from_bytes.

Unencoded slashes that are part of page title (e.g. "A/B_testing") are not a problem in Wikipedia article URLs, but it breaks the REST API that makes extensive use of the slash to separate its arguments. Accessing https://en.wikipedia.org/api/rest_v1/page/html/A/B_testing results in an error, while https://en.wikipedia.org/api/rest_v1/page/html/A%2FB_testing works as expected.

On the other hand, I have not identified any case in which escaping a slash in a page title embeded in a URL would break existing functionality.

Therefore I suggest removing the slash character from the exception list, basically by adding safe='' as the second argument of quote_from_bytes.

Details

SubjectRepoBranchLines +/-
Fix encoding of titles with a '/' in the namepywikibot/coremaster+1 -1
BasePage.title: Encode slashes when asUrl is Truepywikibot/coremaster+4 -4
Customize query in gerrit

Related Objects

Event Timeline

Blahma created this task.Jan 11 2018, 12:28 PM
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptJan 11 2018, 12:28 PM
Blahma updated the task description. (Show Details)Jan 11 2018, 12:29 PM
Blahma renamed this task from Page.title() should encode a slash / to Page.title(asUrl=True) should encode also slashes.Jan 11 2018, 12:48 PM
Blahma updated the task description. (Show Details)
Blahma updated the task description. (Show Details)Jan 11 2018, 12:52 PM
Blahma updated the task description. (Show Details)
Blahma updated the task description. (Show Details)Jan 11 2018, 2:11 PM
Framawiki subscribed.Jan 11 2018, 8:08 PM
Dalba claimed this task.Jan 18 2018, 2:50 AM
Dalba triaged this task as Low priority.
gerritbot subscribed.Jan 18 2018, 2:51 AM

Change 404907 had a related patch set uploaded (by Dalba; owner: Dalba):
[pywikibot/core@master] BasePage.title: Encode slashes when asUrl is True

https://gerrit.wikimedia.org/r/404907

gerritbot added a project: Patch-For-Review.Jan 18 2018, 2:52 AM
gerritbot added a comment.Jan 20 2018, 2:10 PM

Change 404907 merged by jenkins-bot:
[pywikibot/core@master] BasePage.title: Encode slashes when asUrl is True

https://gerrit.wikimedia.org/r/404907

Xqt closed this task as Resolved.Jan 20 2018, 4:16 PM
• 238482n375 removed Dalba as the assignee of this task.Jun 15 2018, 8:03 AM
• 238482n375 lowered the priority of this task from Low to Lowest.
• 238482n375 moved this task from Next Up to In Code Review on the Analytics-Kanban board.
• 238482n375 added projects: Analytics-Kanban, acl*security, Wikimedia-VE-Campaigns (S2-2018), Scap (Scap3-Adoption-Phase2), AbuseFilter, Data-release, Hashtags, LabsDB-Auditor, Ladies-That-FOSS-MediaWiki, Language-2018-Apr-June, Language-2018-Jan-Mar, HHVM, HAWelcome.
• 238482n375 edited subscribers, added: Dalba, • 238482n375; removed: Aklapper.

SG9tZVBoYWJyaWNhdG9yCk5vIG1lc3NhZ2VzLiBObyBub3RpZmljYXRpb25zLgoKICAgIFNlYXJjaAoKQ3JlYXRlIFRhc2sKTWFuaXBoZXN0ClQxOTcyODEKRml4IGZhaWxpbmcgd2VicmVxdWVzdCBob3VycyAodXBsb2FkIGFuZCB0ZXh0IDIwMTgtMDYtMTQtMTEpCk9wZW4sIE5lZWRzIFRyaWFnZVB1YmxpYwoKICAgIEVkaXQgVGFzawogICAgRWRpdCBSZWxhdGVkIFRhc2tzLi4uCiAgICBFZGl0IFJlbGF0ZWQgT2JqZWN0cy4uLgogICAgUHJvdGVjdCBhcyBzZWN1cml0eSBpc3N1ZQoKICAgIE11dGUgTm90aWZpY2F0aW9ucwogICAgQXdhcmQgVG9rZW4KICAgIEZsYWcgRm9yIExhdGVyCgpUYWdzCgogICAgQW5hbHl0aWNzLUthbmJhbiAoSW4gUHJvZ3Jlc3MpCgpTdWJzY3JpYmVycwpBa2xhcHBlciwgSkFsbGVtYW5kb3UKQXNzaWduZWQgVG8KSkFsbGVtYW5kb3UKQXV0aG9yZWQgQnkKSkFsbGVtYW5kb3UsIEZyaSwgSnVuIDE1CkRlc2NyaXB0aW9uCgpPb3ppZSBqb2JzIGhhdmUgYmVlbiBmYWlsaW5nIGF0IGxlYXN0IGEgZmV3IHRpbWVzIGVhY2guIE1vcmUgaW52ZXN0aWdhdGlvbiBuZWVkZWQuCkpBbGxlbWFuZG91IGNyZWF0ZWQgdGhpcyB0YXNrLkZyaSwgSnVuIDE1LCA3OjIxIEFNCkhlcmFsZCBhZGRlZCBhIHN1YnNjcmliZXI6IEFrbGFwcGVyLiC3IFZpZXcgSGVyYWxkIFRyYW5zY3JpcHRGcmksIEp1biAxNSwgNzoyMSBBTQpKQWxsZW1hbmRvdSBjbGFpbWVkIHRoaXMgdGFzay5GcmksIEp1biAxNSwgNzoyMiBBTQpKQWxsZW1hbmRvdSB1cGRhdGVkIHRoZSB0YXNrIGRlc2NyaXB0aW9uLiAoU2hvdyBEZXRhaWxzKQpKQWxsZW1hbmRvdSBhZGRlZCBhIHByb2plY3Q6IEFuYWx5dGljcy1LYW5iYW4uCkpBbGxlbWFuZG91IG1vdmVkIHRoaXMgdGFzayBmcm9tIE5leHQgVXAgdG8gSW4gUHJvZ3Jlc3Mgb24gdGhlIEFuYWx5dGljcy1LYW5iYW4gYm9hcmQuCkNoYW5nZSBTdWJzY3JpYmVycwpDaGFuZ2UgUHJpb3JpdHkKQXNzaWduIC8gQ2xhaW0KTW92ZSBvbiBXb3JrYm9hcmQKQ2hhbmdlIFByb2plY3QgVGFncwpBbmFseXRpY3MtS2FuYmFuCtcKU2VjdXJpdHkK1wpXaWtpbWVkaWEtVkUtQ2FtcGFpZ25zIChTMi0yMDE4KQrXClNjYXAK1wpTY2FwIChTY2FwMy1BZG9wdGlvbi1QaGFzZTIpCtcKQWJ1c2VGaWx0ZXIK1wpEYXRhLXJlbGVhc2UK1wpIYXNodGFncwrXCkxhYnNEQi1BdWRpdG9yCtcKTGFkaWVzLVRoYXQtRk9TUy1NZWRpYVdpa2kK1wpMYW5ndWFnZS0yMDE4LUFwci1KdW5lCtcKTGFuZ3VhZ2UtMjAxOC1KYW4tTWFyCtcKSEhWTQrXCkhBV2VsY29tZQrXCkJvbGQKSXRhbGljcwpNb25vc3BhY2VkCkxpbmsKQnVsbGV0ZWQgTGlzdApOdW1iZXJlZCBMaXN0CkNvZGUgQmxvY2sKUXVvdGUKVGFibGUKVXBsb2FkIEZpbGUKTWVtZQpQcmV2aWV3CkhlbHAKRnVsbHNjcmVlbiBNb2RlClBpbiBGb3JtIE9uIFNjcmVlbgoyMzg0ODJuMzc1IGFkZGVkIHByb2plY3RzOiBTZWN1cml0eSwgV2lraW1lZGlhLVZFLUNhbXBhaWducyAoUzItMjAxOCksIFNjYXAgKFNjYXAzLUFkb3B0aW9uLVBoYXNlMiksIEFidXNlRmlsdGVyLCBEYXRhLXJlbGVhc2UsIEhhc2h0YWdzLCBMYWJzREItQXVkaXRvciwgTGFkaWVzLVRoYXQtRk9TUy1NZWRpYVdpa2ksIExhbmd1YWdlLTIwMTgtQXByLUp1bmUsIExhbmd1YWdlLTIwMTgtSmFuLU1hciwgSEhWTSwgSEFXZWxjb21lLlBSRVZJRVcKMjM4NDgybjM3NSBtb3ZlZCB0aGlzIHRhc2sgZnJvbSBJbiBQcm9ncmVzcyB0byBJbiBDb2RlIFJldmlldyBvbiB0aGUgQW5hbHl0aWNzLUthbmJhbiBib2FyZC4KMjM4NDgybjM3NSByZW1vdmVkIEpBbGxlbWFuZG91IGFzIHRoZSBhc3NpZ25lZSBvZiB0aGlzIHRhc2suCjIzODQ4Mm4zNzUgdHJpYWdlZCB0aGlzIHRhc2sgYXMgTG93ZXN0IHByaW9yaXR5LgoyMzg0ODJuMzc1IHJlbW92ZWQgc3Vic2NyaWJlcnM6IEFrbGFwcGVyLCBKQWxsZW1hbmRvdS4KQ29udGVudCBsaWNlbnNlZCB1bmRlciBDcmVhdGl2ZSBDb21tb25zIEF0dHJpYnV0aW9uLVNoYXJlQWxpa2UgMy4wIChDQy1CWS1TQSkgdW5sZXNzIG90aGVyd2lzZSBub3RlZDsgY29kZSBsaWNlbnNlZCB1bmRlciBHTlUgR2VuZXJhbCBQdWJsaWMgTGljZW5zZSAoR1BMKSBvciBvdGhlciBvcGVuIHNvdXJjZSBsaWNlbnNlcy4gQnkgdXNpbmcgdGhpcyBzaXRlLCB5b3UgYWdyZWUgdG8gdGhlIFRlcm1zIG9mIFVzZSwgUHJpdmFjeSBQb2xpY3ksIGFuZCBDb2RlIG9mIENvbmR1Y3QuILcgV2lraW1lZGlhIEZvdW5kYXRpb24gtyBQcml2YWN5IFBvbGljeSC3IENvZGUgb2YgQ29uZHVjdCC3IFRlcm1zIG9mIFVzZSC3IERpc2NsYWltZXIgtyBDQy1CWS1TQSC3IEdQTApZb3VyIGJyb3dzZXIgdGltZXpvbmUgc2V0dGluZyBkaWZmZXJzIGZyb20gdGhlIHRpbWV6b25lIHNldHRpbmcgaW4geW91ciBwcm9maWxlLCBjbGljayB0byByZWNvbmNpbGUu

Aklapper raised the priority of this task from Lowest to Low.Jun 15 2018, 12:54 PM
Aklapper removed projects: HAWelcome, HHVM, Language-2018-Jan-Mar, Language-2018-Apr-June, Ladies-That-FOSS-MediaWiki, LabsDB-Auditor, Hashtags, Data-release, AbuseFilter, Scap (Scap3-Adoption-Phase2), Wikimedia-VE-Campaigns (S2-2018), acl*security, Analytics-Kanban.
Aklapper edited subscribers, added: Aklapper; removed: • 238482n375.
Aklapper assigned this task to Dalba.Jun 15 2018, 12:55 PM
Framawiki mentioned this in T208086: XTools doesn't handle URL-escaped slash correctly.Oct 26 2018, 7:04 PM
Yurik subscribed.EditedOct 26 2018, 9:10 PM

Turns out this does break the 3rd party sites, e.g. wiki.openstreetmap.org. I think we should revert and introduce a separate path for the rest API.

See https://gerrit.wikimedia.org/r/#/c/pywikibot/core/+/468903/

Yurik reopened this task as Open.Oct 26 2018, 9:11 PM
gerritbot added a comment.Oct 26 2018, 9:12 PM

Change 468903 had a related patch set uploaded (by Yurik; owner: Yuri Astrakhan):
[pywikibot/core@master] Fix encoding of titles with a '/' in the name

https://gerrit.wikimedia.org/r/468903

Dalba removed Dalba as the assignee of this task.Oct 28 2018, 9:53 AM
Xqt subscribed.EditedNov 1 2018, 4:06 AM

Maybe we should use Page.urlname() method with additional parameter(s) like safe parameter for encoding here to allow different results?

Or the title parameter as_url must hold more than True or False e.g. the safe parameter. Think that’s the easiest way. But what is the default behaviour? Normally the current implementation is the default one.

Xqt moved this task from Backlog to Needs Review on the Pywikibot board.Feb 3 2019, 11:22 AM
Xqt renamed this task from Page.title(asUrl=True) should encode also slashes to Page.title(as_url=True) should encode also slashes.May 6 2019, 2:52 PM
Xqt updated the task description. (Show Details)
Dvorapa subscribed.May 6 2019, 3:29 PM

For me it works?

$ python2 pwb.py shell
Welcome to the Pywikibot interactive shell!
>>> import pwb, pywikibot as py
>>> s = py.Site()
>>> p = py.Page(s, u'AC/DC')
>>> p.title(as_url=True)
'AC%2FDC'
Dvorapa added a comment.EditedMay 16 2019, 6:51 PM

Perhaps this is the Windows-only issue

zhuyifei1999 subscribed.May 16 2019, 8:00 PM

No, the title was 'fixed', but is reopened for a (partial) revert afaict.

Pppery edited projects, added Patch-Needs-Improvement; removed Patch-For-Review.Mar 23 2023, 12:58 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL