This is a new feature in Gerrit 2.14+, so we can setup reply emails so that users can reply to gerrit emails.
I have no idea if this will work for us without any changes needed.
See feature doc at https://gerrit.wikimedia.org/r/Documentation/intro-user.html#reply-by-email