In T281202#7099492, @atagar wrote:

Hi Xqt. I'm going to reopen this because T106121 did not answer this. This ticket is about documenting when deprecated code can be removed.

Can we remove deprecated code after a year? Two years? Or is it based on our version number?

There is not an expicit deprecation policy but an established procedure after T106112. There are 3 different warnings.
DeprecationWarning: method/function/property or variable is deprecated but users are not informed due to the usual debug filter. They will be ecalated to a FutureWarning usually after 5 years but this period may be shortened step by step.
FutureWarning: with this state method/function/property or variable is announced to be removed with the overnext release or later. This indicates the bot operators to update their own scripts as soon as possible. This may be used for long deprecated parts or for important code cleanups and improvements or supporting version changes (MediaWiki or Python releases)
ArgumentDeprecationWarning if a bot option is changed. This is like FutureWarning and usually there is no hidden deprection period.

All code must be deprecated unless it is private or part of scripts because it is assumed that scripts aren't subclassed. That's not really true but these script parts should be placed in specialbots module.

We don't hurt with deprecating. There are two different aspects: Bot owners shouldn't have to spent to much time to update their code. Developers on the other Hand shouldn't have to spent to much time to keep old code running (including documentation, supporting dead code etc.) Don't know what is the best way finally. In short:

Gain: Some code clean up
Pain: A lot of our long time users will have broken scrips they have to update

On the other hand we now have now code versioning and all version step has it own version tag in the repository and there is a stable branch different from development master branch. Each tag may be used als long as it works. But support can only be done for the newest release.

See also https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility

Thanks Xqt! Yesterday I finished evaluating all our deprecations prior to the end of 2019, removing FutureWarnings and changing DeprecationWarnings to FutureWarnings. In retrospect I should've discussed it here before digging in. Sorry about that.

They will be ecalated to a FutureWarning usually after 5 years

Could we please make this two years instead? Five is an exceedingly long time to keep our deprecations silent (DeprecationWarnings don't provide any practical value from what I can tell).

If two years is acceptable I'll implement a test that tells us when deprecated code can be transitioned to a FutureWarning and FutureWarnings can be removed.

Don't know what is the best way finally.

Agreed. Backward compatibility is a balancing act. Java and Ruby exemplify two extremes...

• Java offers infinite backward compatibility, but that has led to a painfully cluttered language.
• Ruby drops backward compatibility every other release, which makes code support an onerous treadmill.

Either extreme is problematic. Personally I used symantic versioning within tor, but in retrospect that leaned too much toward backward compatibility. We're on the same page about the tradeoffs here. :)

If we'd like to do the two stage deprecation you describe (DeprecationWarning => FutureWarning => Removal) we need to track both dates as discussed on CR 693589. Would you mind if I add a 'future_warning_since' attribute to our deprecation helpers?

Change 697156 had a related patch set uploaded (by Damian; author: Damian):

[pywikibot/core@master] [IMPR] Document deprecation policy

https://gerrit.wikimedia.org/r/697156

Change 697156 abandoned by Damian:

[pywikibot/core@master] [IMPR] Document deprecation policy

Reason:

Moved to https://www.mediawiki.org/wiki/Manual:Pywikibot/Development/Guidelines#Deprecation_Policy

https://gerrit.wikimedia.org/r/697156

Hi Xqt, I added a deprecation policy to our guidelines. When I get external link permission I'll link the tools functions to their API docs.

Would you mind reviewing this addition? Once we're happy with it this ticket can be closed.

+1

We should already be using semantic versioning since 4.0, but releases are apparently not following it with respect to breaking changes bumping the major version.

The DeprecationWarnings are effectively meaningless since they aren't seen.

Change 697874 had a related patch set uploaded (by Damian; author: Damian):

[pywikibot/core@master] [IMPR] Only use FutureWarnings

https://gerrit.wikimedia.org/r/697874

I amended our deprecation policy in the following ways...

1. Copy the description of semantic versioning I wrote for Tor. We now explain what version numbers convey in terms of backward compatibility.
2. Drop the description of DeprecationWarnings vs FutureWarnings.
3. Drop the "Deprecations might be expedited when necessary" clause. If we break backward compatibility we should bump our major version number.
4. State that prior to version 6.3.1 we did not use any particular versioning scheme since we haven't been following semantic versioning's policy for code removal.

Xqt, would you mind taking a look? To be very clear if we adopt this policy we should do the following...

1. All deprecated code will be removed upon Pywikibot 7.0.0.
2. No deprecated code will be removed until then. This includes code deprecated all the way back in 3.1.

Change 697874 merged by jenkins-bot:

[pywikibot/core@master] [IMPR] Only use FutureWarnings

https://gerrit.wikimedia.org/r/697874

Xqt merged our FutureWarning patch but hasn't said if he approves of the above (ie. to begin enforcing a symantic versioning policy). Since he merges most patches this will be a lost cause without his agreement so leaving this open to get his thoughts.

Change 698632 had a related patch set uploaded (by Damian; author: Damian):

[pywikibot/core@master] [IMPR] Default issue_deprecation_warning() to FutureWarning

https://gerrit.wikimedia.org/r/698632

Change 698632 merged by jenkins-bot:

[pywikibot/core@master] [IMPR] Default issue_deprecation_warning() to FutureWarning

https://gerrit.wikimedia.org/r/698632

In T281202#7130863, @atagar wrote:

I amended our deprecation policy in the following ways...

1. Copy the description of semantic versioning I wrote for Tor. We now explain what version numbers convey in terms of backward compatibility.
2. Drop the description of DeprecationWarnings vs FutureWarnings.
3. Drop the "Deprecations might be expedited when necessary" clause. If we break backward compatibility we should bump our major version number.
4. State that prior to version 6.3.1 we did not use any particular versioning scheme since we haven't been following semantic versioning's policy for code removal.

Xqt, would you mind taking a look? To be very clear if we adopt this policy we should do the following...

1. All deprecated code will be removed upon Pywikibot 7.0.0.
2. No deprecated code will be removed until then. This includes code deprecated all the way back in 3.1.

The curent rule was to increase the major version number if the Python code base wasn't supported (e.g. dropping 2.7) or a supported MediaWiki release has been dropped (see https://www.mediawiki.org/wiki/Manual:Pywikibot/Compatibility). Incompatible code changes has to be announced first with FutureWarning. This is the same as Python deprecates some functions. But I am also fine with your proposal and it might be better to have one big change with 7.0 than douzands every month. I explicitly aggree with your points above.

But what about deprendencies? Does it need to increase the major release too?

With 6.4.0 setuptools 49.4.0 is required due to version parsing and added in the setup.py/requrements.txt which can be updated easily.
With 6.4,0 also extract_templates_and_params_mwpfh and extract_templates_and_params_regex are removed. These both are functions which are dispatched and should never be used directly. I don't want to revert those changes in the current code.

With 6.0 all deprecations are now FutureWarnings including argument deprecations of functions and methods.

The earliest date for 7.0 is after 6.5. Probably 7.0 comes in September. I do not want to increase the major release every month. What would be a good frequent? Quarterly, yearly? In meantime we still need the FutureWarnings and if we have major release changes less than quarterly, a DeprecationWarning is appropriate again.

Can we increase the update_script.py script to show the problems?

But I am also fine with your proposal and it might be better to have one big change with 7.0 than douzands every month. I explicitly aggree with your points above.

Great! Thanks Xqt. :)

But what about deprendencies? Does it need to increase the major release too?

My understanding is that yes, it should. For example Stem (my Tor project) supported Python 2.6+ in its 1.x series and raised its requirement to Python 3.6+ upon version 2.0 because that was the oldest Python release that wasn't EOL.

When we release Pywikibot 7.0.0 we should consider bumping any dependency versions that are particularly old or lack features we want (like the setuptools features you mentioned).

The earliest date for 7.0 is after 6.5. Probably 7.0 comes in September. I do not want to increase the major release every month. What would be a good frequent? Quarterly, yearly?

Very much agreed. Major version bumps should be infrequent since they can necessitate code changes from our users. Yearly or every couple years seems like a good cadence to me but this is wholly up to you.

if we have major release changes less than quarterly, a DeprecationWarning is appropriate again

As discussed above I don't think DeprecationWarnings are useful but if you'd like to resume using them for something then happy to discuss.

Can we increase the update_script.py script to show the problems?

Oh interesting. I didn't know about update_script.py. Maybe we should file another ticket to expand update_script.py to include conversions for all our codebase's present deprecations?

I'm not sure what you had in mind but this could certainly be a useful upgrade tool if we keep it up to date.

The ticket is T283565. But I propose to keep the old script and have renamed it. Is was created for the compat to core migration, is very simple, doesn’t use AST but works good enough.