Change Details

NOTE: This is a technical subticket of {T390611} This ticket is about changing the donation bounded context to store donor data in normalized tables, additionally to storing it in the "data blob". ## Context We distinguish between **Domain Objects** and **Doctrine Entities**. Both are called `Donation`, but are in different namespace (`Domain` vs. `DataAccess`). We have a "Translation Layer" that converts one into the other. In the course of this ticket, we will store the different subclasses of the `Donor` Domain object as relations of the `Donation` Doctrine entity. This will //add// code to the translation layer. We can only remove code from the Translation Layer when all applications use the new tables. These changed are decribed in different tickets. ## Integration details This ticket creates a backwards-breaking change in the use case dependencies (`DonorIdRepository` added to `AddDonationUseCase` and `UpdateDonorUseCase`) and needs a major release and a manual update PR for the Fundraising Application after merging. This is described in {T369160} DO NOT MERGE the changes for this ticket right away, it needs {T390843} before it can be integrated in the Fundraising application. ## Implementation Notes ### Adding and generating IDs - In the next steps we'll need an ID when creating donors. Therefore, donors cannot be instantiated inside the `Donation` class any more. Change `Donation::scrubPersonalData` to receive a `ScrubbedDonor` instead of creating it inside the donation. You may move the donor creation to a static constructor in `ScrubbedDonor`. Change `DatabaseDonationAnonymizer` accordingly. - Create a `DonorIdRepository` interface (and a `DoctrineDonorIdRepository`) to generate new donor IDs. See the to `Donation` ID generation ( `DonationIdRepository` and `DoctrineDonationIdRepository`) as an example. ### Turning Donor interfaces into classes and mapped Doctrine entities - Convert the donor, address and name interfaces to classes with properties (including an integer ID property). Keep the existing accessors. Create an abstract `Donor` base class. You may already add house number and street name to the postal address and change the `getStreetAddress` to return a concatenated value. The classes should be in the `Domain\Model\Donor\` namespace or one of its sub-namespaces. - Change `AddDonationUseCase::getPersonalInfoFromRequest`: use the `DonorIdRepository` (injected in the use case constructor) when generating new donors. This is a backwards-breaking change in `AddDonationUseCase`. Reuse the ID for address and name entities. - Create Doctrine mapping XML files for the new classes. You may keep them in the `config/DoctrineClassMapping` directory. See https://www.doctrine-project.org/projects/doctrine-orm/en/3.3/reference/basic-mapping.html for documentation - Don't use autogenerated IDs, we'll generate the IDs in code (reusing the donor ID for address and name). Use `<generator strategy="NONE"/>` - Pay attention to the naming, it //must// match the namespace, the files should start with `WMDE.Fundraising.DonationContext.Domain.Model.` - Define the name and address entities as one-to-one unidirectional associations. They may have IDs - Add `persist` and `remove` to the name and address associations (see [Transitive persistence cascade](https://www.doctrine-project.org/projects/doctrine-orm/en/3.3/reference/working-with-associations.html#transitive-persistence-cascade-operations)) - Add a `donor` property to the existing `Donation` entity (in the `DoctrineEntities` namespace and as an [one-to-one association mapping](https://www.doctrine-project.org/projects/doctrine-orm/en/3.3/reference/association-mapping.html#association-mapping) in the XML mapping file for `Donation`). For maximum backwards compatibility (until we have migrated all the data, which is another ticket), make it nullable for now - Change the `DoctrineDonationRepository`: When updating donations, check the ID of the donors of the entity vs the domain object. if they don't match, it means that code has changed the donor type (e.g. through scrubbing or in the `UpdateDonor` use case). When you encounter this, you must delete the old entity. Ideally, calling `$oldDonation = $donationEntity->getDonor(); $donationEntity->setDonor($donation->getDonor()); $entityManager->persist($donationEntity); $entityManager->remove($oldDonation); $entityManager->flush();` should take care of all the steps and remove all the lingering dependent entities (through the persistence cascade). - Change `UpdateDonorUseCase::updateDonor`: You need to create a new donor with a new id. Use the `DonorIdRepository` interface as a dependency for the use case. Reuse the ID for address and name entities. - Change `DomainToLegacyConverter` to add the donor from the `Donation` (domain object) to the (entity) `Donation`. - Change `LegacyToDomainConverter` to read the Donor from the `Donation` entity and assign it to the Domain object `Donation`. You can drop the `DonorFactory`. Make sure the `DoctrineDonationTest` still works. If tests fail, make sure that you change the test code only in places where they check low-level data. If the donor is null, return an anoymous `ScrubbedDonor` (mark this default as deprecated and to be removed when the data has been migrated) - Change the `PersonalDataBackup` class to emit backup requests for the new tables, using the `BackupTableCondition` class. Hint: To select donors, you can use subqueries like `id IN (SELECT donor_id FROM spenden WHERE is_scrubbed=0)` - Use a throwaway script or the debugger to observe database behavior (executed SQL) of `DoctrineDonationRepository::getDoctrineDonationById` and the query behavior when you access the `donor` property of the entity. If the behavior shows that Doctrine lazy-loads the donor (multiple SQL statements), change the code of `getDoctrineDonationById` to use a DQL query that eagerly loads the donor data (see [Entity Object Graph Traversal documentation](https://www.doctrine-project.org/projects/doctrine-orm/en/3.3/reference/working-with-objects.html#entity-object-graph-traversal).

NOTE: This is a technical subticket of {T390611} This ticket is about changing the donation bounded context to store donor data in normalized tables, additionally to storing it in the "data blob". ## Context We distinguish between **Domain Objects** and **Doctrine Entities**. Both are called `Donation`, but are in different namespace (`Domain` vs. `DataAccess`). We have a "Translation Layer" that converts one into the other. In the course of this ticket, we will store the different subclasses of the `Donor` Domain object as relations of the `Donation` Doctrine entity. This will //add// code to the translation layer. We can only remove code from the Translation Layer when all applications use the new tables. These changed are decribed in different tickets. ## Integration details This ticket creates a backwards-breaking change in the use case dependencies (`DonorIdRepository` added to `AddDonationUseCase` and `UpdateDonorUseCase`) and needs a major release and a manual update PR for the Fundraising Application after merging. This is described in {T369160} DO NOT MERGE the changes for this ticket right away, it needs {T390843} before it can be integrated in the Fundraising application. ## Implementation Notes Ideally, these sub-tasks should be done in order ### 1. Adding and generating IDs - In the next steps we'll need an ID when creating donors. Therefore, donors cannot be instantiated inside the `Donation` class any more. Change `Donation::scrubPersonalData` to receive a `ScrubbedDonor` instead of creating it inside the donation. You may move the donor creation to a static constructor in `ScrubbedDonor`. Change `DatabaseDonationAnonymizer` accordingly. - Create a `DonorIdRepository` interface (and a `DoctrineDonorIdRepository`) to generate new donor IDs. See the to `Donation` ID generation ( `DonationIdRepository` and `DoctrineDonationIdRepository`) as an example. ### 2. Turning Donor interfaces into classes and mapped Doctrine entities - Convert the donor into an abstract base classe (with an integer `id` property for Doctrine). Keep the existing accessors. Create an abstract `Donor` base class. Change the implementations of the `Donor` interface to child classes of the abstract Donor base class. - We want to continue to use the [Null Object Pattern](https://en.wikipedia.org/wiki/Null_object_pattern) instead of doing null checks for addresses and names. Thewrefore, the Donor classes that don't use `PostalAddress` and `CompanyName`/`PersonName` must now intantiate the null object (e.g. `ScrubbedAddress`, `AnonymousAddress`, `NoName`, etc) in their getters. NOT in the constructor, because Doctrine does not call the constructor and the getter of instances that get read by Doctrine would not have the property instantiated. - For the PostalAddress You may already add house number and street name to the postal address and change the `getStreetAddress` to return a concatenated value. The classes should be in the `Domain\Model\Donor\` namespace or one of its sub-namespaces. to child classes of the abstract base classes. In case of Address and name, the abstract class may return empty values and the child class can inherit without implenting their own overrides. - Change `AddDonationUseCase::getPersonalInfoFromRequest`: use the `DonorIdRepository` (injected in the use case constructor) when generating new donors. This is a backwards-breaking change in `AddDonationUseCase`. Reuse the ID for address and name entities. - Create Doctrine mapping XML files for the new classes. You may keep them in the `config/DoctrineClassMapping` directory. See https://www.doctrine-project.org/projects/doctrine-orm/en/3.3/reference/basic-mapping.html for documentation - Don't use autogenerated IDs, we'll generate the IDs in code (reusing the donor ID for address and name). Use `<generator strategy="NONE"/>` - Pay attention to the naming, it //must// match the namespace, the files should start with `WMDE.Fundraising.DonationContext.Domain.Model.` - Define the name and address entities as one-to-one unidirectional associations. They may have IDs - Add `persist` and `remove` to the name and address associations (see [Transitive persistence cascade](https://www.doctrine-project.org/projects/doctrine-orm/en/3.3/reference/working-with-associations.html#transitive-persistence-cascade-operations)) - Add a `donor` property to the existing `Donation` entity (in the `DoctrineEntities` namespace and as an [one-to-one association mapping](https://www.doctrine-project.org/projects/doctrine-orm/en/3.3/reference/association-mapping.html#association-mapping) in the XML mapping file for `Donation`). For maximum backwards compatibility (until we have migrated all the data, which is another ticket), make it nullable for now ### TBD - Change the `DoctrineDonationRepository`: When updating donations, check the ID of the donors of the entity vs the domain object. if they don't match, it means that code has changed the donor type (e.g. through scrubbing or in the `UpdateDonor` use case). When you encounter this, you must delete the old entity. Ideally, calling `$oldDonation = $donationEntity->getDonor(); $donationEntity->setDonor($donation->getDonor()); $entityManager->persist($donationEntity); $entityManager->remove($oldDonation); $entityManager->flush();` should take care of all the steps and remove all the lingering dependent entities (through the persistence cascade). - Change `UpdateDonorUseCase::updateDonor`: You need to create a new donor with a new id. Use the `DonorIdRepository` interface as a dependency for the use case. Reuse the ID for address and name entities. - Change `DomainToLegacyConverter` to add the donor from the `Donation` (domain object) to the (entity) `Donation`. - Change `LegacyToDomainConverter` to read the Donor from the `Donation` entity and assign it to the Domain object `Donation`. You can drop the `DonorFactory`. Make sure the `DoctrineDonationTest` still works. If tests fail, make sure that you change the test code only in places where they check low-level data. If the donor is null, return an anoymous `ScrubbedDonor` (mark this default as deprecated and to be removed when the data has been migrated) - Change the `PersonalDataBackup` class to emit backup requests for the new tables, using the `BackupTableCondition` class. Hint: To select donors, you can use subqueries like `id IN (SELECT donor_id FROM spenden WHERE is_scrubbed=0)` - Use a throwaway script or the debugger to observe database behavior (executed SQL) of `DoctrineDonationRepository::getDoctrineDonationById` and the query behavior when you access the `donor` property of the entity. If the behavior shows that Doctrine lazy-loads the donor (multiple SQL statements), change the code of `getDoctrineDonationById` to use a DQL query that eagerly loads the donor data (see [Entity Object Graph Traversal documentation](https://www.doctrine-project.org/projects/doctrine-orm/en/3.3/reference/working-with-objects.html#entity-object-graph-traversal).

NOTE: This is a technical subticket of {T390611} This ticket is about changing the donation bounded context to store donor data in normalized tables, additionally to storing it in the "data blob". ## Context We distinguish between **Domain Objects** and **Doctrine Entities**. Both are called `Donation`, but are in different namespace (`Domain` vs. `DataAccess`). We have a "Translation Layer" that converts one into the other. In the course of this ticket, we will store the different subclasses of the `Donor` Domain object as relations of the `Donation` Doctrine entity. This will //add// code to the translation layer. We can only remove code from the Translation Layer when all applications use the new tables. These changed are decribed in different tickets. ## Integration details This ticket creates a backwards-breaking change in the use case dependencies (`DonorIdRepository` added to `AddDonationUseCase` and `UpdateDonorUseCase`) and needs a major release and a manual update PR for the Fundraising Application after merging. This is described in {T369160} DO NOT MERGE the changes for this ticket right away, it needs {T390843} before it can be integrated in the Fundraising application. ## Implementation Notes ###Ideally, these sub-tasks should be done in order ### 1. Adding and generating IDs - In the next steps we'll need an ID when creating donors. Therefore, donors cannot be instantiated inside the `Donation` class any more. Change `Donation::scrubPersonalData` to receive a `ScrubbedDonor` instead of creating it inside the donation. You may move the donor creation to a static constructor in `ScrubbedDonor`. Change `DatabaseDonationAnonymizer` accordingly. - Create a `DonorIdRepository` interface (and a `DoctrineDonorIdRepository`) to generate new donor IDs. See the to `Donation` ID generation ( `DonationIdRepository` and `DoctrineDonationIdRepository`) as an example. ### 2. Turning Donor interfaces into classes and mapped Doctrine entities - Convert the donor into an abstract base classe (with an integer `id` property for Doctrine). Keep the existing accessors. Create an abstract `Donor` base class. Change the implementations of the `Donor` interface to child classes of the abstract Donor base class. - We want to continue to use the [Null Object Pattern](https://en.wikipedia.org/wiki/Null_object_pattern) instead of doing null checks for addresses and names. Thewrefore, the Donor classes that don't use `PostalAddress` and `CompanyName`/`PersonName` must now intantiate the null object (e.g. `ScrubbedAddress`, address and name interfaces to classes with properties (including an integer ID property).`AnonymousAddress`, Keep the existing accessors.`NoName`, Create an abstract `Donor` base classetc) in their getters. You may already add house number and street name to the postal address and change the `getStreetAddress` to return a concatenated value.NOT in the constructor, The classes should be in the `Domain\Model\Donor\` namespace or one of its sub-namespacesbecause Doctrine does not call the constructor and the getter of instances that get read by Doctrine would not have the property instantiated. - For the PostalAddress You may already add house number and street name to the postal address and change the `getStreetAddress` to return a concatenated value. The classes should be in the `Domain\Model\Donor\` namespace or one of its sub-namespaces. to child classes of the abstract base classes. In case of Address and name, the abstract class may return empty values and the child class can inherit without implenting their own overrides. - Change `AddDonationUseCase::getPersonalInfoFromRequest`: use the `DonorIdRepository` (injected in the use case constructor) when generating new donors. This is a backwards-breaking change in `AddDonationUseCase`. Reuse the ID for address and name entities. - Create Doctrine mapping XML files for the new classes. You may keep them in the `config/DoctrineClassMapping` directory. See https://www.doctrine-project.org/projects/doctrine-orm/en/3.3/reference/basic-mapping.html for documentation - Don't use autogenerated IDs, we'll generate the IDs in code (reusing the donor ID for address and name). Use `<generator strategy="NONE"/>` - Pay attention to the naming, it //must// match the namespace, the files should start with `WMDE.Fundraising.DonationContext.Domain.Model.` - Define the name and address entities as one-to-one unidirectional associations. They may have IDs - Add `persist` and `remove` to the name and address associations (see [Transitive persistence cascade](https://www.doctrine-project.org/projects/doctrine-orm/en/3.3/reference/working-with-associations.html#transitive-persistence-cascade-operations)) - Add a `donor` property to the existing `Donation` entity (in the `DoctrineEntities` namespace and as an [one-to-one association mapping](https://www.doctrine-project.org/projects/doctrine-orm/en/3.3/reference/association-mapping.html#association-mapping) in the XML mapping file for `Donation`). For maximum backwards compatibility (until we have migrated all the data, which is another ticket), make it nullable for now ### TBD - Change the `DoctrineDonationRepository`: When updating donations, check the ID of the donors of the entity vs the domain object. if they don't match, it means that code has changed the donor type (e.g. through scrubbing or in the `UpdateDonor` use case). When you encounter this, you must delete the old entity. Ideally, calling `$oldDonation = $donationEntity->getDonor(); $donationEntity->setDonor($donation->getDonor()); $entityManager->persist($donationEntity); $entityManager->remove($oldDonation); $entityManager->flush();` should take care of all the steps and remove all the lingering dependent entities (through the persistence cascade). - Change `UpdateDonorUseCase::updateDonor`: You need to create a new donor with a new id. Use the `DonorIdRepository` interface as a dependency for the use case. Reuse the ID for address and name entities. - Change `DomainToLegacyConverter` to add the donor from the `Donation` (domain object) to the (entity) `Donation`. - Change `LegacyToDomainConverter` to read the Donor from the `Donation` entity and assign it to the Domain object `Donation`. You can drop the `DonorFactory`. Make sure the `DoctrineDonationTest` still works. If tests fail, make sure that you change the test code only in places where they check low-level data. If the donor is null, return an anoymous `ScrubbedDonor` (mark this default as deprecated and to be removed when the data has been migrated) - Change the `PersonalDataBackup` class to emit backup requests for the new tables, using the `BackupTableCondition` class. Hint: To select donors, you can use subqueries like `id IN (SELECT donor_id FROM spenden WHERE is_scrubbed=0)` - Use a throwaway script or the debugger to observe database behavior (executed SQL) of `DoctrineDonationRepository::getDoctrineDonationById` and the query behavior when you access the `donor` property of the entity. If the behavior shows that Doctrine lazy-loads the donor (multiple SQL statements), change the code of `getDoctrineDonationById` to use a DQL query that eagerly loads the donor data (see [Entity Object Graph Traversal documentation](https://www.doctrine-project.org/projects/doctrine-orm/en/3.3/reference/working-with-objects.html#entity-object-graph-traversal).