Change Details

==== Narratives * As an SRE, I want the ability to observe the ImageMatchingAlgorithm data pipeline metrics, so that I know how well it is performing compared to defined SLOs * As a PET data engineer, I want the ability to observe the ImageMatchingAlgorithm data pipeline metrics, so that I can have a baseline to ensure we have optimized how our pipeline performs. * As an Analytics Engineer, I want the ability to observe the ImageMatchingAlgorithm data pipeline metrics, so that I can be aware of how the data infrastructure resources are being utilized. ==== Acceptance Criteria [] As an PET Data Engineer, I want the ability to generate a csv file with the following metrics, so that I can have a baseline of how the pipeline performs. [] in/out records [] CPU Usage [] Memory Usage [] Executor Counts [] Runtime and resource utilization for algo ==== Notes * This is the initial part of how we start to collect metrics. We will iterate so that we have designated stores for our collected metrics. ==== Subtasks []

==== Narratives * As an SRE, I want the ability to observe the ImageMatchingAlgorithm data pipeline metrics, so that I know how well it is performing compared to defined SLOs * As a PET data engineer, I want the ability to observe the ImageMatchingAlgorithm data pipeline metrics, so that I can have a baseline to ensure we have optimized how our pipeline performs. * As an Analytics Engineer, I want the ability to observe the ImageMatchingAlgorithm data pipeline metrics, so that I can be aware of how the data infrastructure resources are being utilized. ==== Acceptance Criteria [X] As an PET Data Engineer, I want the ability to generate a csv file with the following metrics, so that I can have a baseline of how the pipeline performs. [X] in/out records [X] CPU Usage [X] Memory Usage [X] Executor Counts [X] Runtime and resource utilization for algo ==== Notes * This is the initial part of how we start to collect metrics. We will iterate so that we have designated stores for our collected metrics. ==== Subtasks []

==== Narratives * As an SRE, I want the ability to observe the ImageMatchingAlgorithm data pipeline metrics, so that I know how well it is performing compared to defined SLOs * As a PET data engineer, I want the ability to observe the ImageMatchingAlgorithm data pipeline metrics, so that I can have a baseline to ensure we have optimized how our pipeline performs. * As an Analytics Engineer, I want the ability to observe the ImageMatchingAlgorithm data pipeline metrics, so that I can be aware of how the data infrastructure resources are being utilized. ==== Acceptance Criteria [X] As an PET Data Engineer, I want the ability to generate a csv file with the following metrics, so that I can have a baseline of how the pipeline performs. [X] in/out records [X] CPU Usage [X] Memory Usage [X] Executor Counts [X] Runtime and resource utilization for algo ==== Notes * This is the initial part of how we start to collect metrics. We will iterate so that we have designated stores for our collected metrics. ==== Subtasks []