Page MenuHomePhabricator
Feed Advanced Search

Sep 29 2022

bmansurov closed T318063: Failed to find any Kerberos tgt as Resolved.

This seems to have fixed the issue:

Sep 29 2022, 3:44 AM · Data-Engineering-Planning
bmansurov added a comment to T318063: Failed to find any Kerberos tgt.

Yep, as @otto indicated, could you check by running the following please?

sudo -u analytics-research kerberos-run-command analytics-researh yarn logs -appOwner analytics-research -applicationId application_1663082229270_26382
Sep 29 2022, 3:32 AM · Data-Engineering-Planning

Sep 27 2022

bmansurov updated the task description for T318063: Failed to find any Kerberos tgt.
Sep 27 2022, 1:01 PM · Data-Engineering-Planning

Sep 18 2022

bmansurov created T318063: Failed to find any Kerberos tgt.
Sep 18 2022, 10:27 PM · Data-Engineering-Planning

Sep 9 2022

bmansurov added a comment to T315374: airflow instances should use specific artifact cache directories.

Great!

Sep 9 2022, 9:46 AM · Data-Engineering, Data Pipelines

Sep 8 2022

bmansurov added a comment to T315374: airflow instances should use specific artifact cache directories.

@xcollazo thanks for the ping. If you mean the research instance on deploy1002, then I've pulled your changes, rebased on main and deployed. If you mean something else, please let me know. Thanks!

Sep 8 2022, 11:57 PM · Data-Engineering, Data Pipelines

Aug 24 2022

bmansurov placed T213761: Verify newly generated data before pushing it to production up for grabs.
Aug 24 2022, 4:51 PM · Recommendation-API
bmansurov closed T251587: Morelike API response is formatted badly as Resolved.
Aug 24 2022, 4:50 PM · Recommendation-API

Jun 30 2022

bmansurov closed T311336: Can't deploy airflow-dags/research anymore as Resolved.
Jun 30 2022, 2:20 AM · Scap, Data-Engineering
bmansurov added a comment to T311336: Can't deploy airflow-dags/research anymore.

Solved by running the following commands on airflow1002:

Jun 30 2022, 2:20 AM · Scap, Data-Engineering

Jun 25 2022

bmansurov renamed T311336: Can't deploy airflow-dags/research anymore from Can't deploy an airflow-dags/research anymore to Can't deploy airflow-dags/research anymore.
Jun 25 2022, 11:07 AM · Scap, Data-Engineering
bmansurov created T311336: Can't deploy airflow-dags/research anymore.
Jun 25 2022, 10:57 AM · Scap, Data-Engineering

Jun 7 2022

bmansurov closed T251732: wikiworkshop.org has Facebook button, external statcounter, https to http redirect as Resolved.

It seems the majority of the issues described in the task have been resolved. Closing it for now. Feel free to re-open it.

Jun 7 2022, 12:22 AM · SecTeam-Processed, Patch-For-Review, Security-Team, Privacy, Research, Privacy Engineering, Traffic, SRE

Jun 4 2022

bmansurov created T309922: Create `research` hive user.
Jun 4 2022, 2:42 PM · Data-Engineering-Kanban, Data-Engineering

May 7 2022

bmansurov added a comment to T251732: wikiworkshop.org has Facebook button, external statcounter, https to http redirect.

@BBlack thank you. Who would be a good person to review the patch? @Dzahn, would you be able to review the above patch? Thank you.

May 7 2022, 4:19 AM · SecTeam-Processed, Patch-For-Review, Security-Team, Privacy, Research, Privacy Engineering, Traffic, SRE

Apr 30 2022

bmansurov added a comment to T251732: wikiworkshop.org has Facebook button, external statcounter, https to http redirect.

Remaining items:

  1. Implement https://wikitech.wikimedia.org/wiki/Tool:Event_Metrics. I looked more into it. Unfortunately, it does not fit our use case as it's tied to specific Wikimedia projects (Wikipedia, Wiktionary, and Wikivoyage). Before looking more into it, I was under the false impression that we could use it as a pageview counter.
Apr 30 2022, 2:39 AM · SecTeam-Processed, Patch-For-Review, Security-Team, Privacy, Research, Privacy Engineering, Traffic, SRE

Apr 21 2022

bmansurov closed T306313: How to submit a Pyspark job with a keytab and principal? as Resolved.

Creating a spark session directly (as opposed to using wmfdata) seems to have fixed the issue.

Apr 21 2022, 4:44 PM · Data-Engineering

Apr 20 2022

bmansurov closed T305719: Add a new member to the airlfow-dags repository as Resolved.

@mforns, thanks! I'm able to push.

Apr 20 2022, 6:06 AM · Data Pipelines, Data-Engineering

Apr 17 2022

bmansurov created T306313: How to submit a Pyspark job with a keytab and principal?.
Apr 17 2022, 11:59 AM · Data-Engineering

Apr 8 2022

bmansurov created T305719: Add a new member to the airlfow-dags repository.
Apr 8 2022, 4:34 PM · Data Pipelines, Data-Engineering

Apr 7 2022

bmansurov closed T305457: PySpark is unable to find Hive tables as Resolved.

Thank you, both!

Apr 7 2022, 5:10 AM · Data-Engineering

Apr 6 2022

bmansurov added a comment to T305457: PySpark is unable to find Hive tables.

Hi @JAllemandou! Thanks for the reply and fixing the tag. I have followed the instructions from the link you shared. I've created a new stacked environment and activated it before running.

Apr 6 2022, 3:15 PM · Data-Engineering

Apr 5 2022

bmansurov renamed T305457: PySpark is unable to find Hive tables from PySpark is unable to find hive tables to PySpark is unable to find Hive tables.
Apr 5 2022, 12:26 PM · Data-Engineering
bmansurov created T305457: PySpark is unable to find Hive tables.
Apr 5 2022, 12:25 PM · Data-Engineering

Mar 20 2022

bmansurov committed rRWWS7ce4ff65ada9: Fix header on top on mobile.
Fix header on top on mobile
Mar 20 2022, 4:23 AM

Mar 19 2022

bmansurov committed rRWWS12a4641ea2d0: Add a speaker and fix the banner background color.
Add a speaker and fix the banner background color
Mar 19 2022, 1:19 PM

Mar 16 2022

bmansurov committed rRWWS2bdbbc4beecc: Update content and images; fix styling issues.
Update content and images; fix styling issues
Mar 16 2022, 2:27 AM

Feb 22 2022

bmansurov added a comment to T300450: Kerberos identity for bmansurov.

Hi @razzi, thanks for the instructions. I got my access.

Feb 22 2022, 10:36 PM · Data-Engineering-Kanban, Data-Engineering

Feb 13 2022

bmansurov added a comment to T301215: Give bmansurov access necessary to support Research Airflow jobs.

Thanks!

Feb 13 2022, 3:03 AM · SRE, Data-Engineering, SRE-Access-Requests

Feb 8 2022

bmansurov created T301215: Give bmansurov access necessary to support Research Airflow jobs.
Feb 8 2022, 1:24 AM · SRE, Data-Engineering, SRE-Access-Requests

Jan 30 2022

bmansurov created T300450: Kerberos identity for bmansurov.
Jan 30 2022, 1:55 AM · Data-Engineering-Kanban, Data-Engineering

Jan 20 2022

bmansurov added a comment to P18949 Resized image for 755807.

Lessig_Credit Jessica Scranton - small.jpg (441×390 px, 47 KB)

Jan 20 2022, 11:50 PM
bmansurov created P18949 Resized image for 755807.
Jan 20 2022, 11:50 PM

Dec 16 2021

bmansurov committed rRWWS1511e8aeec72: Add new content.
Add new content
Dec 16 2021, 3:05 AM

Dec 15 2021

bmansurov committed rRWWS1e40e54314c2: Add 2022.
Add 2022
Dec 15 2021, 5:33 AM

May 24 2021

bmansurov created T283469: Hovercard for NSA doesn't show anything.
May 24 2021, 2:47 AM · Product-Infrastructure-Team-Backlog-Deprecated (Kanban), Page-Previews (Tracking), Mobile-Content-Service

Apr 13 2021

bmansurov committed rRWWSee2013a30e90: Add papers.
Add papers
Apr 13 2021, 9:46 AM
bmansurov committed rRWWS7fb905c0a034: News and schedule updates.
News and schedule updates
Apr 13 2021, 1:05 AM

Apr 2 2021

bmansurov committed rRWWSa1c31a148129: Add schedule.
Add schedule
Apr 2 2021, 12:45 PM

Mar 20 2021

bmansurov committed rRWWS03b25fd5cb12: Add speakers and accepted papers.
Add speakers and accepted papers
Mar 20 2021, 10:48 PM
bmansurov added a comment to T271163: TranslationRecommendation* Schemas Event Platform Migration.

OK. Thanks!

Mar 20 2021, 9:30 PM · Data-Engineering, Research, Event-Platform, Analytics

Mar 15 2021

bmansurov added a comment to T271163: TranslationRecommendation* Schemas Event Platform Migration.

Indeed. Those schema tables are all empty. I visited the eventgate-validation dashboard on Logstash, but I couldn't find any such requests. Where can I find more about those missing events?

Mar 15 2021, 11:55 PM · Data-Engineering, Research, Event-Platform, Analytics

Mar 13 2021

bmansurov added a comment to T273535: GapFinder tool returns error message for every pair of languages.

I just saw this task. The above described bug was fixed in https://gerrit.wikimedia.org/r/c/research/recommendation-api/+/669595 .

Mar 13 2021, 2:10 AM · Recommendation-API, GapFinder
bmansurov added a comment to T271163: TranslationRecommendation* Schemas Event Platform Migration.

I think so. For example, here's what was sent:

Mar 13 2021, 2:08 AM · Data-Engineering, Research, Event-Platform, Analytics

Mar 8 2021

bmansurov added a comment to T271163: TranslationRecommendation* Schemas Event Platform Migration.

@Ottomata thanks for reviewing the patch. I've merged it and deployed it.

Mar 8 2021, 4:42 AM · Data-Engineering, Research, Event-Platform, Analytics

Feb 22 2021

bmansurov committed rRWWSba57e90ca6a8: Remove "Coming soon!".
Remove "Coming soon!"
Feb 22 2021, 1:19 PM
bmansurov committed rRWWSe32cb8b62b3b: Various updates.
Various updates
Feb 22 2021, 2:48 AM

Feb 2 2021

bmansurov added a comment to T270140: Release dataset on top search engine referrers by country, device, and language.

@Isaac thanks for the link. I've been working on this patch. This is the main file. Feel free to add comments to the patch with new additions or fixes.

Feb 2 2021, 12:46 AM · Data-Engineering, Privacy Engineering, Research

Jan 28 2021

bmansurov claimed T271163: TranslationRecommendation* Schemas Event Platform Migration.
Jan 28 2021, 10:53 PM · Data-Engineering, Research, Event-Platform, Analytics

Jan 13 2021

bmansurov added a comment to T270140: Release dataset on top search engine referrers by country, device, and language.

Thanks, both.

Jan 13 2021, 1:38 AM · Data-Engineering, Privacy Engineering, Research

Jan 11 2021

bmansurov added a comment to T270140: Release dataset on top search engine referrers by country, device, and language.

I think I created an Oozie job once ;) Would you say this is a good starting point: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Oozie? Any other links? Maybe you can share a similar job that I can take an inspiration from?

Jan 11 2021, 11:49 PM · Data-Engineering, Privacy Engineering, Research

Jan 6 2021

bmansurov committed rRWWSce04e30d85e8: 2021 updates.
2021 updates
Jan 6 2021, 12:51 AM

Dec 15 2020

bmansurov committed rRWWS567e322a936f: Add 2021 template and fix 2020 issues.
Add 2021 template and fix 2020 issues
Dec 15 2020, 5:24 AM

Nov 10 2020

bmansurov added a comment to T241230: Migrate recommendation-api to kubernetes.

@akosiaris Thanks for reviewing my patchsets. I was wondering if you've seen my last comment. My understanding is that the patch is very close to being merged, which would be great. Could you please take a look at my benchmark numbers and let me know what you think? Thanks. Here's the patch: https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/602527

Nov 10 2020, 12:47 AM · Product-Infrastructure-Team-Backlog-Deprecated, serviceops, Release-Engineering-Team, Services, Recommendation-API

Nov 9 2020

Aklapper awarded T210433: Identify and release data on similar Wikidata items a Like token.
Nov 9 2020, 6:09 PM · Recommendation-API

Sep 14 2020

bmansurov added a comment to T257843: Enable CI on research/landing-page repo.

Auto submit is working.

Sep 14 2020, 10:04 PM · Continuous-Integration-Config, Research

Sep 6 2020

bmansurov created P12498 ab results.
Sep 6 2020, 10:32 PM
bmansurov added a comment to T241230: Migrate recommendation-api to kubernetes.

@hashar o/ Yes, we're still working on it. I'll try to speed it up.

Sep 6 2020, 7:49 PM · Product-Infrastructure-Team-Backlog-Deprecated, serviceops, Release-Engineering-Team, Services, Recommendation-API

Aug 30 2020

bmansurov committed rRLPfdaa09ef46c9: Configure CI to run unit tests.
Configure CI to run unit tests
Aug 30 2020, 9:51 PM
bmansurov added a comment to T257843: Enable CI on research/landing-page repo.

@Reedy I've submitted a patch above. Could you review it? If I understood the tutorial correctly, this should be enough to auto-submit on +2.

Aug 30 2020, 9:51 PM · Continuous-Integration-Config, Research

Aug 23 2020

bmansurov added a comment to T257843: Enable CI on research/landing-page repo.

Are you still having to submit patches manually? If so, we should probably fix that before calling this bug done (as enabling CI config was done in this task)

Aug 23 2020, 2:23 AM · Continuous-Integration-Config, Research
bmansurov added a comment to T257843: Enable CI on research/landing-page repo.

Eslint has been added.

Aug 23 2020, 1:49 AM · Continuous-Integration-Config, Research
bmansurov committed rRLPd32b60ff233b: Add eslint.
Add eslint
Aug 23 2020, 1:17 AM

Jul 2 2020

bmansurov committed rMSRA858ae64c466f: Add ca-certificates to production blubber.
Add ca-certificates to production blubber
Jul 2 2020, 2:28 AM

Jul 1 2020

bmansurov committed rMSRA9e61a80079b8: Formatting fix.
Formatting fix
Jul 1 2020, 2:41 AM
bmansurov committed rMSRAbfe2fa615502: Log MW API errors.
Log MW API errors
Jul 1 2020, 2:41 AM

Jun 5 2020

bmansurov added a comment to T254143: Recommendation api always returns 404 when seed article is not supplied.

As soon as T254211: mostviewed generator not returning any results is fixed, the issue will go away.

Jun 5 2020, 10:45 PM · Article-Recommendation, Recommendation-API

Jun 4 2020

bmansurov added a comment to T254143: Recommendation api always returns 404 when seed article is not supplied.

@santhosh The mostviewed API started working again. Recommendations also are working for the language wikis where the mostviewed API is working. For example, I see some results on for Russian: https://en.wikipedia.org/api/rest_v1/data/recommendation/article/creation/translation/ru

Jun 4 2020, 11:39 PM · Article-Recommendation, Recommendation-API
bmansurov added a comment to T254211: mostviewed generator not returning any results.

Hmm, it's still not working on French Wikipedia: https://fr.wikipedia.org/w/api.php?action=query&generator=mostviewed&prop=pageviews

Jun 4 2020, 11:38 PM · Data-Engineering-Icebox, Analytics-Radar, MediaWiki-Action-API
bmansurov added a comment to T254211: mostviewed generator not returning any results.

o/ @Milimetric Thanks for looking into it. It's working for me too.

Jun 4 2020, 11:36 PM · Data-Engineering-Icebox, Analytics-Radar, MediaWiki-Action-API

Jun 2 2020

bmansurov added a comment to T241230: Migrate recommendation-api to kubernetes.

Thanks, @akosiaris!

Jun 2 2020, 11:16 PM · Product-Infrastructure-Team-Backlog-Deprecated, serviceops, Release-Engineering-Team, Services, Recommendation-API
bmansurov created T254211: mostviewed generator not returning any results.
Jun 2 2020, 2:02 AM · Data-Engineering-Icebox, Analytics-Radar, MediaWiki-Action-API
bmansurov added a comment to T254143: Recommendation api always returns 404 when seed article is not supplied.

@santhosh the issue is with the MediaWiki API. The mostviewed generator is not returning anything: https://en.wikipedia.org/w/api.php?action=query&generator=mostviewed&prop=pageviews

Jun 2 2020, 1:32 AM · Article-Recommendation, Recommendation-API

May 27 2020

bmansurov committed rRWWS4bb3d14aa83b: 2020: upgrade bootstrap.
2020: upgrade bootstrap
May 27 2020, 11:57 PM
bmansurov added a comment to T251732: wikiworkshop.org has Facebook button, external statcounter, https to http redirect.

@JFishback_WMF, issues mentioned at T251732#6158467 have been addressed.

May 27 2020, 11:57 PM · SecTeam-Processed, Patch-For-Review, Security-Team, Privacy, Research, Privacy Engineering, Traffic, SRE
bmansurov committed rRWWS2cfba8a48f15: 2019: Upgrade bootstrap.
2019: Upgrade bootstrap
May 27 2020, 2:00 AM
bmansurov committed rRWWS2cf262ff11fb: 2017: nav bg fix.
2017: nav bg fix
May 27 2020, 1:47 AM
bmansurov committed rRWWSb6885523f939: 2018: upgrade bootstrap.
2018: upgrade bootstrap
May 27 2020, 1:44 AM
bmansurov committed rRWWSfc4dd637c9be: 2017: Upgrade bootstrap.
2017: Upgrade bootstrap
May 27 2020, 1:34 AM

May 26 2020

bmansurov committed rRWWS1f90253f9cc2: 2016: Upgrade bootstrap.
2016: Upgrade bootstrap
May 26 2020, 2:26 AM

May 24 2020

bmansurov added a comment to T251732: wikiworkshop.org has Facebook button, external statcounter, https to http redirect.

Just a little status update: I've removed YUI and working on upgrading bootstrap. Since a lot changed between versions 2 and 4, it'll take some time to fully upgrade years 2016 through 2020. I'll let you know when I'm done.

May 24 2020, 6:02 PM · SecTeam-Processed, Patch-For-Review, Security-Team, Privacy, Research, Privacy Engineering, Traffic, SRE
bmansurov committed rRWWSb2f0700f7de6: 2015: remove YUI js and (most of) CSS.
2015: remove YUI js and (most of) CSS
May 24 2020, 5:14 PM
bmansurov committed rRWWS7f193567f684: Copy external assets to the repo.
Copy external assets to the repo
May 24 2020, 4:34 PM

May 22 2020

bmansurov committed rRWWS41754ca80f8b: Add video links.
Add video links
May 22 2020, 1:22 AM

May 21 2020

bmansurov closed T252956: Add Git LFS support for research/wikiworkshop as Resolved.

Thanks, @leila!

May 21 2020, 11:53 PM · git-lfs, SRE, Research

May 19 2020

bmansurov added a comment to T252956: Add Git LFS support for research/wikiworkshop.

@leila that sounds good.

May 19 2020, 12:44 AM · git-lfs, SRE, Research

May 18 2020

bmansurov added a comment to T252956: Add Git LFS support for research/wikiworkshop.

@Dzahn we have about 1.7 GB of video recordings of the recent workshop. We're trying to upload those files. Currently, there are only 4 files.

May 18 2020, 11:14 PM · git-lfs, SRE, Research

May 17 2020

bmansurov committed rRWWSe2216ca088e0: Fix links.
Fix links
May 17 2020, 1:47 AM
bmansurov created T252956: Add Git LFS support for research/wikiworkshop.
May 17 2020, 1:41 AM · git-lfs, SRE, Research
bmansurov committed rRWWSaed03a45cb82: Bring back missing people.
Bring back missing people
May 17 2020, 1:13 AM

May 13 2020

bmansurov committed rRWWSedf59099c5e9: Add Jess Wade slides.
Add Jess Wade slides
May 13 2020, 10:28 PM
bmansurov committed rRWWSda2abb4cbd15: remove more stat counters (authored by Dzahn).
remove more stat counters
May 13 2020, 10:24 PM
bmansurov committed rRWWS8f54476435cf: remove statcounter (authored by Dzahn).
remove statcounter
May 13 2020, 1:43 AM
bmansurov committed rRWWS8ca482966a34: Add papers and update speakers.
Add papers and update speakers
May 13 2020, 1:43 AM
bmansurov committed rRWWS5a47edab2152: remove Facebook button (authored by Dzahn).
remove Facebook button
May 13 2020, 1:43 AM

May 10 2020

bmansurov added a comment to T241230: Migrate recommendation-api to kubernetes.

@akosiaris could you please re-review this patch: https://gerrit.wikimedia.org/r/#/c/operations/deployment-charts/+/565788/

May 10 2020, 2:30 PM · Product-Infrastructure-Team-Backlog-Deprecated, serviceops, Release-Engineering-Team, Services, Recommendation-API
bmansurov updated subscribers of T251732: wikiworkshop.org has Facebook button, external statcounter, https to http redirect.

@leila head's up that one of the patches is removing the Facebook button. Another is removing statcounter. We probably want to use https://wikitech.wikimedia.org/wiki/Tool:Event_Metrics instead. What do you think?

May 10 2020, 2:26 PM · SecTeam-Processed, Patch-For-Review, Security-Team, Privacy, Research, Privacy Engineering, Traffic, SRE

May 3 2020

bmansurov added a comment to T247732: recommendation api's test on scb nodes are flapping.

@elukey I've deployed the fix. Let me know if you still see the issue.

May 3 2020, 9:39 PM · observability, SRE, Research
bmansurov committed rMSRAa02871729813: Health check: return 404 when MW API fails.
Health check: return 404 when MW API fails
May 3 2020, 9:37 PM
bmansurov committed rMSRD0c68d621d764: Update recommendation-api to a028717.
Update recommendation-api to a028717
May 3 2020, 9:36 PM