Page MenuHomePhabricator

Support Type:URL values with more than 255 characters
Closed, DeclinedPublic

Description

Type:URL is currently limited to 255 characters, since this is the maximum length of the string field in the standard database schema of SMW. Arbitrary length types Text and Code cannot be queried in the same way, and do therefore not provide an alternative.

As a solution, all long Text data should be stored together with a hash to allow efficient querying, and a way should be provided for URL data to use this facility (either with a new type LongURL or by changing the default behaviour of URL). It might be possible to construct the hash by using, say, 247 initial characters of the string, followed by an 8 character alphanumeric hash code. This could have some advantages:

(1) String comparison functions (that are based on lexicographic order) would produce meaningful results unless the first 247 chars agree. Even in the latter case, meaningful comparisons would be possible.

(2) It might be possible to leave the long text field set to NULL if the string is short enough for 255 letters. This should safe reduce the overhead of supporting long strings significantly.

With this method, all "long string" types should be reconcilable with the normal short versions, and String as well as URL should be storable in the same table.


Version: unspecified
Severity: major
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=32955

Details

Reference
bz34511

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 12:12 AM
bzimport set Reference to bz34511.
bzimport added a subscriber: Unknown Object (MLST).

bugzilla wrote:

I just came across this limitation too:
When querying an long URL it is limited to 255 characters and thus gets corrupted (in most cases).

It would be nice if URLs could be longer than 255 characters.

Bug 32955 is now fixed, and I added it to the "See Also" area for this bug. I also changed the priority of this from "Enhancement" to "Major", because the feature already exists (storing URL's as property data), but it is essentially non-working. Beyond that, Stefahn reported that attempting to use the feature will cause a loss of the data, which would normally prompt me to assign "Critical" priority.

I decided against "Critical" priority because the feature was essentially non-working until today, when Bug 32955 was fixed (I haven't tested the fix yet), so users would immediately notice the feature does not work, before significant data could be lost. Note that none of these bugs will cause the loss of existing data, because it's impossible to store it in the first place. Thus, data losses do not occur unexpectedly, so the threat is as minimal as it gets, as far as data loss is concerned (an immediately obvious failure to store the data).

Now that Bug 32955 is fixed, I expect the next release of SMW will make it much more likely the feature will be used, and data will be lost (at save time) before users notice the problem (at query time). That could arguably create the conditions to justify upgrading this bug to "Critical" importance. Ideally, this bug would be fixed before the next SMW version is released.

For now, the only workaround available is to use URL shorteners. There are MediaWiki extensions for producing them automatically, but they are beta quality, and users report high dissatisfaction with them. SMW query URL's can be gargantuan, and the usage of square brackets in the URL's makes it impossible to share them as-is on forums that support BBCode (all modern forums). SMW is desperately in need of an URL shortener feature, which has been requested for other non-SMW uses of MediaWiki too: Bug 42085. I will request the feature at GitHub:

https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues?milestone=&q=is%3Aopen

Aklapper subscribed.

The Semantic MediaWiki developers requested in https://phabricator.wikimedia.org/T64114 to move their task tracking to https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues and to close remaining tasks in Wikimedia Phabricator. If you still face the problem reported in this task in a supported version of SMW, please feel free to transfer your report to https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues . We are sorry for the inconvenience.