(1) would support an easier filling of forms in one go
(2) would support probably a more feature rich search but interrupt the form filling process

I would vote for (1) at the moment.

I've been thinking about this for a while, too. The problem is not that mapping templates are not working when remote autocomplete is enabled (they do!), but the value is queried only against the page title (and not the displayed label). Only the returning results are then processed by the mapping template.

In the case that the page title does not contain the search string (as it might be a random/enumerated ID), the query returns no results and therefore the mapping template isn't even applied, as the selection could not find a match.

I can think of an solution for this problem that involves @cicalese Semantic Title / Display Title approach. This will presume that you don't use mapping templates if it can be avoided (they have very high performance costs when the number of retrieved values becomes high). Use Display Title (Semantic Title) instead

Then SF would need to add native support for Display Title in the following areas:

• Introduce something like mapping display title or just display title. Instead of having to call the wikitext parser (which is very expensive) this would just SQL query for the display title directly. This can also be done through mapping property=Display title of currently, but adding native support would allow for better performance.
• Rewrite functions like SFUtils::getAllPagesForCategory so that they do the query selection on the display title (if given) of a page instead/in additionto the page title itself. This could also directly return both page title and display title so mapping wouldn't be neccessary at all.

This would at least solve the problem's we currently have and enable some very interesting possibilities like using generated ID's for page titles. Currently we cannot do this when we use autocomplete fields and the number of pages becomes to big to pre-fetch them (remote autocomplete OFF)

Best,
Simon

I've tried to implement the first change as it is very simple. Here's a possible patch for it.

This does not solve the original problem yet. The autocomplete queries still would need to be rewritten. This patch has only the benefit of better performance when using display titles.

Index: includes/SF_Utils.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- includes/SF_Utils.php	(revision 8a4dad116c93e0b47b1452d903520fb3cbd7d41d)
+++ includes/SF_Utils.php	(revision )
@@ -795,6 +795,19 @@
 		return SFUtils::disambiguateLabels( $labels );
 	}
 
+	/**
+	 * Helper function to get the display title from an array of values (page titles)
+	 */
+	public static function getLabelsFromDisplayTitle($values) {
+		$labels = array();
+		foreach ( $values as $value ) {
+			$titleObject = Title::newFromText($value);
+			$labels[$value] = SemanticTitle::getText($titleObject);
+		}
+		return SFUtils::disambiguateLabels( $labels );
+		return $labels;
+	}
+
 
 	/**
 	 * Private function to disambiguate labels.
Index: includes/SF_FormField.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- includes/SF_FormField.php	(revision 8a4dad116c93e0b47b1452d903520fb3cbd7d41d)
+++ includes/SF_FormField.php	(revision )
@@ -332,7 +332,7 @@
 		}
 
 		// If we're using Cargo, there's no equivalent for "values from
-		// property" - instead, we just always get the values if a 
+		// property" - instead, we just always get the values if a
 		// field and table have been specified.
 		if ( is_null( $f->mPossibleValues ) && defined( 'CARGO_VERSION' ) && $cargo_table != null && $cargo_field != null ) {
 			// We only want the non-null values. Ideally this could
@@ -353,6 +353,8 @@
 			} elseif ( array_key_exists( 'mapping cargo table', $f->mFieldArgs ) &&
 				array_key_exists( 'mapping cargo field', $f->mFieldArgs ) ) {
 				$f->mPossibleValues = SFUtils::getLabelsFromCargoField( $f->mPossibleValues, $f->mFieldArgs['mapping cargo table'], $f->mFieldArgs['mapping cargo field'] );
+			} elseif ( array_key_exists( 'mapping display title', $f->mFieldArgs ) ) {
+				$f->mPossibleValues = SFUtils::getLabelsFromDisplayTitle( $f->mPossibleValues);
 			}
 		}
 		// Backwards compatibility.

Here is a patch that adds experimental support for remote autocomplete on display title for "values from category"

Work in progress! The patch is also not on the current master state, sorry!

https://gist.github.com/Fannon/031d4c51e91dccd74aa34648c9c44aea

I have been thinking about the same issue recently. My thoughts are that, if a display title exists for a page, it should be used without requiring a mapping parameter. Rather than using the SemanticTitle:getText() function in your example above, which introduces a dependency on SemanticTitle into SemanticForms, I suggest using the PageProps class that will be available in MediaWiki core in the soon-to-be-released MediaWiki 1.27. Of course, that will introduce a dependency as well. The fallback would be the current functionality. I had planned to work on this functionality soon but have not yet found the time. If you get to it first, that would be great.

I've looked at remote autocompletion with mapping several times, and I do think it is implementable, but it is not straight forward to do in an efficient manner. I do think it will be easier and more efficient to implement using display title than when I previously looked at it using the mapping parameters.

Yes, exactly!

Do you have the changes for MW 1.27 somewhere documented? The page_props table is already existing in MW 1.26 - but using the Class would have the advantage of memoization/caching I guess?

If some of those changes will make it into MW 1.27 itself, will the Semantic Title Extension be neccesary at all?

I'm not sure using the display title as label per default is something users always wants, though it's probably a reasonable default.

I'll take a look into matters the next days. If the internal data structure of SF can be refactored accordingly, the display title could be retrieved as label by default and there's no need at all to have an additional step of mapping it. This would greatly improve performane and convenience. All forminputs need to be adjusted for this new data structure, so it will probably be a buch of work ;)

Great! If you follow the approach taken by the mapping parameters, you should not need to touch the forminputs. Everything automagically works when you update the "possible values" variable accordingly in SF_FormField.php. You will just need to check for whether the current input is of type Page and if it has a displaytitle page property assigned - rather than checking for the presence of a mapping parameter.

Yes, the advantage of using the PageProps class is caching. The class is documented at https://doc.wikimedia.org/mediawiki-core/master/php/classPageProps.html.

Once MW 1.27 is released, I plan to update SemanticTitle to use PageProps. The ability to use a property or cargo table/field to set the display title will then be unnecessary. Since the current implementation of those two capabilities set the display title in a way that has a potential race condition, my preference would be to remove those capabilities in the new Semantic Title release. Especially now that the "Display title of" property gets set automatically from the display title, setting the display title from a property is problematic and likely redundant.

SemanticTitle is still necessary for setting the link text of a link to a page with a display title set to be that display title - and for setting the text of a self link on a page to be the display title. I had tried to get that functionality merged into core with the PageProps class, but there were concerns about potential performance impacts. I decided to address that at a later date so as not to further delay the merge of PageProps. Unless and until this is merged into core, SemanticTitle will still be necessary.

SemanticTitle also optionally adds a subtitle containing the real page title to pages that have the display title set. I'm not sure how much that feature is used. DISPLAYTITLE does not do this. Do you need this feature?

Finally, SemanticTitle adds a parser function that allows you to get the display title for a page. This could potentially be added to core as a magic word,

@cicalese

Ok, I understand why Semantic Title is still necessary then. May I suggest something? You could also keep Semantic Title as it is and for MW 1.27 create a Display Title extension which handles the renaming of links and internal APIs. The current name indicates a dependency on SMW which isn't the case anymore, right?

We haven't been using the subtitle - it would add visual clutter and the real page title is part of the URL anyway. But it's probably nice to have as an option anyway? Maybe the most flexible way would be a small parser funtion which allows to add text to the subtitle in a generic way, then you could just add {{FULLPAGENAME}} to it and have the same functionality but with full control.

Are you sure about not needing to change the forminputs?

I've taken a deeper look into it and I fear it probably won't be as easy as that. The data-structure itself has to be changed / refactored throughout Semantic Forms if we want to support labels natively, and this will affect all forminputs (which have to be rewritten to reflect this, too). This would make your value_label variable unneccessary as the label is then part of possible_values by default.

There are at least four sources where possible_values comes from: Default Values, Existing Values, values from * and asynchronously via AJAX through Remote Autocomplete API. The first three have to be changed to use an associative array, the API itself already has the correct structure to simply add the label.

It looks like I haven't got enough time to do this bigger change right now, but it'll be on my todo list ;)

Best,
Simon

May I suggest something? You could also keep Semantic Title as it is and for MW 1.27 create a Display Title extension which handles the renaming of links and internal APIs.

It's funny - I was considering doing exactly that. People could continue using the existing SemanticTitle unchanged. I would add a note on the SemanticTitle extension page on mediawiki.org indicating that they should consider transitioning to the new Display Title extension, which would be supported going forward. I could definitely see the Display Title extension functionality being useful regardless of whether any semantic extensions are used.

Are you sure about not needing to change the forminputs?

I'm not sure about anything ;-) I do know that when I added the mapping functionality initially, I was surprised that the bulk of the changes were in SF_FormPrinter, some of the functionality of which is now in SF_FormField. But, I remember minor tweaks in text with autocomplete and tokens. I did not have to make any other changes, for example, to checkboxes, radio buttons, etc, though. The new functionality may indeed require that, but it would be good to avoid having to touch that much code, if possible.

I did see your mention that radio buttons are now not working. They were definitely working before. There may be an issue introduced by the recent code refactoring. I already found and had fixed a bug dealing with delimiters. This might be something similar.