Task description update
Adding feedback from mw.org, sv.wiki and cs.wiki:

• At sv.wiki (translated using Google Translate) | https://w.wiki/eCm
  • "I wish that the editing comment was open by default, or that it would be possible to "pin" it so that it was always open to me if I wanted it, and that it was under its own heading, not "advanced" if it could be closed." via LittleGun here: https://w.wiki/eCs
• At cs.wiki
  • "Feedback from Czech Wikipedia: Advanced user (sysop) didn't know that he can add custom summary because of "hided" field. And I agree with him, it isn't easy to find it. via @Patriccck here: https://w.wiki/eCr

DECIDED
During today's team stand up, we decided on the following initial approach for making it so people who always want the custom edit summary to appear within the Reply Tool have a way of making it so the tool behaves in this way.

Initial approach
Anytime someone opens or closes the Advanced shelf a preference is set.

(The above has been added to the task description)

Change 634275 had a related patch set uploaded (by Bartosz Dziewoński; owner: Bartosz Dziewoński):
[mediawiki/extensions/DiscussionTools@master] Add preference to expand the "Advanced" menu when replying

https://gerrit.wikimedia.org/r/634275

• ✅ Sticky preference: if you customize the edit summary of the last comment you posted with the Reply Tool, the next time you open the tool the "Advanced" shelf should be shown.
• Special:Preference: create a new preference in Special:Preferences that enables people to explicit decide whether the "Advanced" shelf be always shown or not

I think we should do both. Most users will probably only use the toggle in the reply tool interface, but displaying it on Special:Preferences as well allows it to be customized in global preferences.

I don’t like if such a simple action as clicking on a toggle changes something shown in my preferences. I think it would be better to have a three-way preference (show/hide/remember); the remember option could write another, hidden preference (or local storage or whatever) upon toggle.

Change 634275 merged by jenkins-bot:
[mediawiki/extensions/DiscussionTools@master] Add preference to expand the "Advanced" menu when replying

https://gerrit.wikimedia.org/r/634275

In T261539#6548621, @Tacsipacsi wrote:

I don’t like if such a simple action as clicking on a toggle changes something shown in my preferences.

For me, that is preferable to the other alternative being considered here: clicking on a toggle changes something in my preferences and doesn't give me any way to view what it changed ;)

(although my main goal with adding the visible preference was to allow customizing it in global preferences, as I wrote earlier)

Test wiki created on Patch Demo by PPelberg (WMF) using patch(es) linked to this task:

http://patchdemo.wmflabs.org/wikis/2898c78621c1901160d57cb54f9d8986/w/

In T261539#6582829, @matmarex wrote:

In T261539#6548621, @Tacsipacsi wrote:

I don’t like if such a simple action as clicking on a toggle changes something shown in my preferences.

For me, that is preferable to the other alternative being considered here: clicking on a toggle changes something in my preferences and doesn't give me any way to view what it changed ;)

There are quite a number of toggles that are remembered (or, in some cases, only ought to be remembered due to some bugs) yet they don’t have any visibility in the preferences: VisualEditor mode (unless set to something other than the default remember), CodeMirror (for wikitext) and CodeEditor (for CSS/JS/JSON/Lua) syntax highlight status and open/closed table of contents, just to mention a few. I don’t think these would have terrible UX. In contrast, I don’t know of any other tool (extension, core code etc.) that implicitly changes preferences.

(although my main goal with adding the visible preference was to allow customizing it in global preferences, as I wrote earlier)

I should have overlooked this. However, this leads to another question: what happens when you try to change a global preference using api.saveOption? If it gets ignored (regardless of whether it’s written to the database and ignored at read or not written at all), it kind of ruins the goal of this preference change method, while changing how Wikipedia works for me with a click on Meta is even less nice than changing only within wiki.

In T261539#6583297, @Tacsipacsi wrote:

There are quite a number of toggles that are remembered (or, in some cases, only ought to be remembered due to some bugs) yet they don’t have any visibility in the preferences: VisualEditor mode (unless set to something other than the default remember), CodeMirror (for wikitext) and CodeEditor (for CSS/JS/JSON/Lua) syntax highlight status and open/closed table of contents, just to mention a few. I don’t think these would have terrible UX. In contrast, I don’t know of any other tool (extension, core code etc.) that implicitly changes preferences.

I'd prefer most of those to be visible on the preferences page… maybe I'm just weird.

The watchlist/RC filters interface actually changes preferences like that. For example, using the "Results to show" / "Time period to search" dropdown will result in the changed values being saved in the "Days to show in watchlist" and "Maximum number of changes to show in watchlist" preferences.

I am pretty sure the Popups extension (Page Previews) also used to do that, but it looks like they deleted that feature and the button that used to display an interface to configure it is now just a link to your preferences. (It still displays a pop-up interface if you use it while logged out.)

(although my main goal with adding the visible preference was to allow customizing it in global preferences, as I wrote earlier)

I should have overlooked this. However, this leads to another question: what happens when you try to change a global preference using api.saveOption? If it gets ignored (regardless of whether it’s written to the database and ignored at read or not written at all), it kind of ruins the goal of this preference change method, while changing how Wikipedia works for me with a click on Meta is even less nice than changing only within wiki.

It gets ignored… 🤷‍♂️

@matmarex will submit another patch for the time being to remove the explicit advanced drawer setting in user preferences

Change 636996 had a related patch set uploaded (by Bartosz Dziewoński; owner: Bartosz Dziewoński):
[mediawiki/extensions/DiscussionTools@master] Make the "Advanced" menu preference hidden

https://gerrit.wikimedia.org/r/636996

Change 636996 merged by jenkins-bot:
[mediawiki/extensions/DiscussionTools@master] Make the "Advanced" menu preference hidden

https://gerrit.wikimedia.org/r/636996