Page MenuHomePhabricator

Allow Support for Multiple PHP Versions - Patch Demo for Testing Failures
Open, Needs TriagePublicFeature

Description

Feature summary

What you would like to be able to do and where:

Enable MediaWiki to support and test against multiple PHP versions simultaneously, with a dedicated patch demo environment that can reproduce and isolate PHP version-specific failures. This would allow developers to:

  • Test patches against multiple PHP versions (e.g., PHP 8.1, 8.2, 8.3) in parallel
  • Identify version-specific compatibility issues before deployment
  • Create isolated demo environments for each PHP version to test specific failure scenarios
  • Validate that code changes work consistently across supported PHP versions

Use case(s)

List the steps that you performed to discover that problem, and describe the actual underlying problem:

  1. Current workflow limitations:
    • Developer creates a patch for MediaWiki core or extension
    • After merge, issues are discovered in production environments running different PHP versions
    • Debugging requires manual setup of different PHP environments
  1. Specific scenario encountered:
    • Patch works perfectly on PHP 8.1 (development environment)
    • Same patch causes fatal errors on PHP 8.2 (staging environment)
    • Issue discovered only after deployment, requiring emergency rollback
    • Root cause: deprecated function usage that behaves differently across PHP versions
  1. Testing challenges:
    • No easy way to spin up multiple PHP version environments for patch testing
    • Lack of automated cross-version compatibility checks
    • Manual testing across PHP versions is time-consuming and error-prone
    • Difficulty reproducing version-specific issues for debugging
  1. Underlying problem: The current development and testing infrastructure doesn't provide adequate coverage for PHP version compatibility, leading to production failures that could be caught earlier in the development cycle.

Benefits

Why should this be implemented:

    1. Reliability & Stability
  • Prevent production failures: Catch PHP version-specific issues before they reach production environments
  • Reduce emergency rollbacks: Identify compatibility problems during development phase *Unnecessary point for WMF production version*
  • Improve code quality: Ensure consistent behavior across all supported PHP versions
    1. Development Efficiency
  • Faster debugging: Dedicated demo environments for reproducing version-specific failures
  • Reduced manual work: Eliminate need for developers to manually set up multiple PHP environments
    1. Risk Mitigation
  • Compatibility assurance: Validate that new features work across entire PHP support matrix
  • Deprecation management: Early detection of deprecated function usage across PHP versions
    1. Community Benefits
  • Better contributor experience: Easier for community developers to test across PHP versions
  • Documentation improvements: Clear guidelines for PHP version compatibility requirements
  • Reduced support burden: Fewer version-specific bug reports and support requests
    1. Technical Advantages
  • Parallel testing: Multiple PHP versions tested simultaneously, reducing overall test time
  • Isolated environments: Each PHP version runs in its own container/environment
  • Comprehensive coverage: Full test suite execution across all supported PHP versions
  • Early warning system: Proactive identification of upcoming PHP version compatibility issues

Event Timeline

Example testing the T400401 error that I reported, I had to create several environments to give details with the error even though it does not affect production but it cannot be tested with stable versions of PHP

This is a good idea, but I don’t think it should be a means of, nor a substitute for, CI/CD running on different PHP versions. It can facilitate manual testing across PHP versions, which is also important, and justifies this feature, but automated tests don’t need a publicly available environment (and the infrastructure for running automated tests across PHP versions already exists, it just doesn’t include PHP 8.4 yet).

  • Security improvements: Test security patches against all supported PHP versions

I don’t think security patches should be tested on Patch Demo: https://patchdemo.wmcloud.org/ publicly lists all demo wikis, along with their patches, which should likewise be public, and this could lead to premature disclosure of security bugs.

This is a good idea, but I don’t think it should be a means of, nor a substitute for, CI/CD running on different PHP versions. It can facilitate manual testing across PHP versions, which is also important, and justifies this feature, but automated tests don’t need a publicly available environment (and the infrastructure for running automated tests across PHP versions already exists, it just doesn’t include PHP 8.4 yet).

  • Security improvements: Test security patches against all supported PHP versions

I don’t think security patches should be tested on Patch Demo: https://patchdemo.wmcloud.org/ publicly lists all demo wikis, along with their patches, which should likewise be public, and this could lead to premature disclosure of security bugs.

I understand what you have said, my idea was written a bit out of ignorance :D I must debug things I wrote that were unnecessary, I opened the task mostly for issues of testing PHP compatibility, more than saying that it is for critical risk tests among other tests

thcipriani subscribed.

In catalyst is seems possible with some caveats to implement this feature.

Caveats:

  • Users would have to know what PHP version is supported
  • Cartisian product of bug reports (i.e., reporting errors in PatchDemo for problems with particular extensions and particular versions) — we'd want to flag non-standard versions as "use at your own risk" or something.

Backlogging this for more discussion for now.

Caveats:

  • Users would have to know what PHP version is supported

Why, couldn’t we track that?

  • Cartisian product of bug reports (i.e., reporting errors in PatchDemo for problems with particular extensions and particular versions) — we'd want to flag non-standard versions as "use at your own risk" or something.

That’s true.