Summary: In Firefox, captions for both the original TimedMediaHandler audio player and the beta audio player are out-of-sync by approximately 100 to 400 milliseconds (this seems to vary). This primarily affects UX, since the delay can be noticeable enough to be irritating.
While testing these captions, I noticed that Firefox seemed to be persistently out-of-sync when playing back captions (this happened with both the old media player and the beta media player). Audacity tells me that the recording starts almost exactly on-beat, but even knowing that I wasn't able to synchronize the captions. Safari did noticeably better.
So I uploaded a test file¹ with the same tempo and added these captions. What seems to happen is that Firefox will play the captions with some delay (it doesn't seem to be always the same, and it's worse in the old player), whereas Safari doesn't seem to have much of a delay at all – if there is any – but buffers before playing the file. All of the caption timings should be exact to within 0.01 ms, since I derived them directly from the tempo. I haven't tested any other browsers yet.
¹ The link goes to the English Wikipedia to avoid T230650, since on Commons the issue causes the captions to be displayed in the wrong place.