May 27 2022
Regarding the option of using a batch of queries to an external database; the issue is that what we are creating is a specialized index specifically for what we need. What we perform is a tree traversal were at each node a new decision is made. To do something like this with a database, one would have to fire a series of queries where each of the queries is dependent on all the results of the previous ones.
An update on the current status, mainly regarding the index file:
Apr 8 2022
I merged the pull request on github now.
Mar 28 2022
@Joe We now made the changes to use the bullseye distribution and the provided image with go installed See: https://github.com/martaannaj/RecommenderServer/pull/22/files
@QChris We just went ahead and added that file in our repository.
Mar 23 2022
@Joe : would it be fine if we include the index file into the image for the A/B testing phase? If that is successful, we can see whether it makes sense to move it to a different location.
@QChris I noticed the addition of the .gitreview file on gerrit. Is this file needed? If so, we would merge it into our github repository, so we can keep the active development there and synchronize with gerrit.
Mar 2 2022
Mar 1 2022
We'd be happy to receive a patch or pull request.
Feb 21 2022
@Joe for the base image, would you recommend our current approach of starting from an 'empty' image and downloading the latest go distribution ourselves, or should we take this one: https://docker-registry.wikimedia.org/golang1.17/tags/
I am not sure how stable these images are.
Feb 16 2022
@Joe we have created files for blubber before, I assume what is needed is very similar to that? I am not sure I know what the deployment-charts repository is . The code is under development on https://github.com/martaannaj/RecommenderServer/ also the blubber configuration files are there.
Feb 10 2022
Jan 12 2022
Dec 16 2021
@sbassett Is that something which should be checked now, during the security readiness review, or only later upon deployment?
Dec 15 2021
Thanks @sbassett . I didn't realize it was possible to run semgrep without posting the results to their service. They actually have a configuration available which can be used without that feature. I have now configured that.
I have now added gokart. The github action was not working out of the box, because of some missing configuration parameters in the example. I opened a pull request for that.
Dec 1 2021
Nov 28 2021
Nov 27 2021
We went forward fixing multiple aspects:
Nov 10 2021
For the security policy, I created a first draft. I would prefer to use phabricator for any security issues. Smaller issues can go as issues on github. https://github.com/martaannaj/RecommenderServer/blob/main/SECURITY.md
Aug 2 2021
I can see the property suggester events.
Aug 1 2021
Jul 31 2021
The example I was looking at fits the case you describe.
Jul 30 2021
With much appreciated help from @Ottomata , we got to see some events using kafkacat on deployment-kafka-jumbo-2 now. To check whether the events are correctly generated, this seems sufficient.
One thing which got done on the way is including the config of these streams from this patch https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/708592 into the main config.
Jul 29 2021
Updated a patch to resolve the issues.
Jul 28 2021
@Ottomata Is this a patch you can review?
Turned out the instructions which we had found before are outdated (now marked as deprecated). New ones are here: https://wikitech.wikimedia.org/wiki/Event_Platform/Instrumentation_How_To#Deployment
Accordingly, I created a patch to get our streams activated. https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/708592
At the moment, they are only in the -labs configuration. After making sure that works, we can move the configuration to main config and get it merged during a backport window.
We (@Michaelcochez and @Martaannaj ) should have access now. We still have to figure out where the events are logged. The /srv/log/eventlogging directory were logging should happen according to TestingOnBetaCluster seems empty.