The $wgOAuth2PrivateKey and $wgOAuth2PublicKey settings are mentioned in https://www.mediawiki.org/wiki/Extension:OAuth#Configuration, but there is no documentation there explaining what the software actually expects for these settings.

If you dig around in the code long enough to figure out that the OAuth2 support is a wrapper around https://github.com/thephpleague/oauth2-server then you might eventually stumble on https://oauth2.thephpleague.com/installation/#generating-public-and-private-keys