The SerializerFactory class is not a generic factory for serializers. It only supports the base model. In particular, it does not support custom entity types. BaseModelSerializerFactory would be a more descriptive name, that would avoid the confusion that led to T160426.
The same applies for DeserializerFactory.