getUnusedPropertiesSpecial has serious performance problems
Closed, ResolvedPublic
Actions

Assigned To

None

Authored By

	• bzimport
	Feb 6 2008, 10:01 PM

Description

Author: hsolbrig

Description:
Patch for revision 30649 of SMW_SQLStore.php

getUnusedPropertiesSpecial in SMW_SQLStore.php exhibits horrible performance when the number of attributes and/or relations is large (>200,000). The outer joins result in several million tuples needing to be examined to select the non-matching fields and this can take many (many!) minutes, even on large processors.

The attached patch has a proposed alternative - instead of outer joins, we create a temporary table which contains all of the properties and then delete rows from the table that occur in the relations, attributes, ... tables instead.

Version: unspecified
Severity: normal

Attached:

SMW_SQLStore.php.patch3 KBDownload

Details

Reference: bz12946

Event Timeline

• bzimport raised the priority of this task from to Medium.Nov 21 2014, 10:01 PM

• bzimport added a project: MediaWiki-extensions-Semantic-MediaWiki.

• bzimport set Reference to bz12946.

• bzimport added a subscriber: Unknown Object (MLST).

• bzimport created this task.Feb 6 2008, 10:01 PM

For large wikis, this special page should probably be disabled or cached (as all slow Special pages are on large wikis). I have applied the above patch now, and SMW's new storage implementation (to be released soon) will also use the above method with temporary queries.

I still note that, in principle, this can also fail if there are really a lot of properties. One certainly would not wish a temporary table with 10000s of entries (with VARCHAR(255) one can expect about 64000 entries to be the absolute maximum for a temporary table in standard MySQL settings).

getUnusedPropertiesSpecial has serious performance problemsClosed, ResolvedPublicActions

Description

Details

Event Timeline

getUnusedPropertiesSpecial has serious performance problems
Closed, ResolvedPublic
Actions