Custom entity types need a mechanism to hook into RdfBuilder to provide mappings of parts of such entities that are not covered by the default code. The handlers for mapping entities to RDF should be defined in the Xxx.entitytypes.php, similar to the way that handlers for mapping custom data types are defined in Xxx.datatypes.php.
Just as for data types, the entity type definition could contain a 'rdf-builder-factory-callback' field, which could then be used to construct an EntityRdfBuilderFactory, simmilar to the ValueSnakRdfBuilderFactory.
EntityRdfBuilderFactory would provide EntityRdfBuilder instances for an entity type. Note that the current signature of addEntity defined by EntityRdfBuilder may be a bit awkward for the purpose, but it should be possibel to use it without changing much.