Sending a REST API request with an invalid OAuth header returns in a generic rest-read-denied error. It should return an OAuth-specific error about why the authentication failed. That error is generated in SessionProvider::provideSessionInfo() but that runs too early to throw (the API framework is not set up yet) so it just caches the error and returns an anonymous session. The action API will fetch that error in ApiBeforeMain; the REST framework should do something comparable.