https://gerrit.wikimedia.org/r/#/c/143407/ provides an RTL version of the 'help' icon, but doesn't work because the build process assumes that "file" maps to a string and doesn't resolve the object. Resolving the object would need OOjs UI, however, so… bootstrapping issue?
This would require us to generate per-language CSS files for distribution. The syntax you used is supported by MediaWiki's ResourceLoaderImageModule, but currently not by OOUI's image generator thingy.
For MediaWiki, we can use ResourceLoader and ResourceLoaderImageModule, which support this kind of thing natively.
For standalone OOjs UI, we should try out some magic with :lang() selectors.
We could also merge the LTR/RTL stuff together with :dir() selectors, which are coming 'soon' if the current draft of Selectors 4 remains true: http://dev.w3.org/csswg/selectors-4/#the-dir-pseudo.