Page MenuHomePhabricator

dr0ptp4kt (Adam Baso)
Principal Software Engineer, Wikimedia Foundation

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Sunday

  • Clear sailing ahead.

User Details

User Since
Oct 7 2014, 6:35 PM (497 w, 2 d)
Availability
Available
IRC Nick
dr0ptp4kt
LDAP User
Unknown
MediaWiki User
ABaso (WMF) [ Global Accounts ]

Recent Activity

Today

dr0ptp4kt added a comment to T358345: [Epic] Search metrics 2024.

For those following along, have a look at the comment in T358349#9727873 to identify the notebook helping to fill a table in @EBernhardson's namespace and an example Superset.

Fri, Apr 19, 12:31 PM · Discovery-Search (Current work), Epic
dr0ptp4kt added a comment to T358352: Search Metrics - Number of user sessions using search.

Updated AC to say daily where it incorrectly said monthly within the Preferred section. It already said "estimated daily unique devices" so was hopefully sufficiently clear, but still. Sorry!

Fri, Apr 19, 11:50 AM · Discovery-Search (Current work)
dr0ptp4kt updated the task description for T358352: Search Metrics - Number of user sessions using search.
Fri, Apr 19, 11:48 AM · Discovery-Search (Current work)

Yesterday

dr0ptp4kt added a project to T362920: Benchmark Blazegraph import with increased buffer capacity (and other factors): Wikidata.
Thu, Apr 18, 6:28 PM · Wikidata, Wikidata-Query-Service
dr0ptp4kt renamed T362920: Benchmark Blazegraph import with increased buffer capacity (and other factors) from Benchmark Blazegraph import with increased buffer capacity to Benchmark Blazegraph import with increased buffer capacity (and other factors).
Thu, Apr 18, 6:18 PM · Wikidata, Wikidata-Query-Service
dr0ptp4kt created T362920: Benchmark Blazegraph import with increased buffer capacity (and other factors).
Thu, Apr 18, 6:18 PM · Wikidata, Wikidata-Query-Service
dr0ptp4kt awarded T336443: Investigate performance differences between wdqs2022 and older hosts a Burninate token.
Thu, Apr 18, 3:48 PM · Patch-For-Review, Data-Platform-SRE (2024.04.15 - 2024.05.05)

Wed, Apr 17

dr0ptp4kt added a comment to T358351: Search Metrics - Read traffic generated by Search.

@EBernhardson I had duplicated the verbiage "estimated daily unique devices, based on unique_devices_per_domain_monthly" (emphasis on incorrect "monthly" in Preferred section), but have now updated the Preferred section to say "estimated daily unique devices, based on unique_devices_per_domain_daily" to correct this glitch. I think you have this covered already, but just wanted to make sure the edit was obvious.

Wed, Apr 17, 7:18 PM · MW-1.43-notes (1.43.0-wmf.1; 2024-04-16), Patch-For-Review, Discovery-Search (Current work)
dr0ptp4kt added a comment to T358351: Search Metrics - Read traffic generated by Search.

@EBernhardson I had duplicated the verbiage "estimated daily unique devices, based on unique_devices_per_domain_monthly", but have now updated the Preferred section to say "estimated daily unique devices, based on unique_devices_per_domain_daily". I think you have this covered already, but just wanted to make sure the edit was obvious.

Wed, Apr 17, 7:17 PM · MW-1.43-notes (1.43.0-wmf.1; 2024-04-16), Patch-For-Review, Discovery-Search (Current work)
dr0ptp4kt updated the task description for T358351: Search Metrics - Read traffic generated by Search.
Wed, Apr 17, 7:16 PM · MW-1.43-notes (1.43.0-wmf.1; 2024-04-16), Patch-For-Review, Discovery-Search (Current work)

Tue, Apr 16

dr0ptp4kt created P60694 Graph split Spark refactor.
Tue, Apr 16, 8:28 PM
dr0ptp4kt added a comment to T362060: Generalize ScholarlyArticleSplitter.

Running time
Total Uptime: 55 min

Tue, Apr 16, 8:14 PM · Patch-For-Review, Discovery-Search (Current work), Wikidata
dr0ptp4kt added a comment to T362060: Generalize ScholarlyArticleSplitter.

I kicked off a run using the current version of the patch with the following command and backing table, and its status should be able to be followed here: https://yarn.wikimedia.org/cluster/app/application_1713178047802_16409

Tue, Apr 16, 4:49 PM · Patch-For-Review, Discovery-Search (Current work), Wikidata

Fri, Apr 12

dr0ptp4kt added a comment to T358352: Search Metrics - Number of user sessions using search.

@EBernhardson I updated the AC.

Fri, Apr 12, 8:33 PM · Discovery-Search (Current work)
dr0ptp4kt added a comment to T358345: [Epic] Search metrics 2024.

Short run we determined that the following are the initial focus:

Fri, Apr 12, 8:32 PM · Discovery-Search (Current work), Epic
dr0ptp4kt added a comment to T358350: Search Metrics - Successful searches.

(Updated previous comment. Do this in conjunction with the other tickets, not necessarily afterward.)

Fri, Apr 12, 8:27 PM · Discovery-Search (Current work)
dr0ptp4kt added a comment to T358351: Search Metrics - Read traffic generated by Search.

@EBernhardson I updated the AC to capture the essence of IRC discussion and the what we went over in Etherpad.

Fri, Apr 12, 8:21 PM · MW-1.43-notes (1.43.0-wmf.1; 2024-04-16), Patch-For-Review, Discovery-Search (Current work)
dr0ptp4kt updated subscribers of T358350: Search Metrics - Successful searches.

@EBernhardson I updated the AC to indicate that this should only be specified where there is high confidence signaling. For the near term, this notion of "successful searches" (satisifed searches) analysis comes after the other analysis.

Fri, Apr 12, 8:20 PM · Discovery-Search (Current work)
dr0ptp4kt updated subscribers of T358349: Search Metrics - Number of Searches.

@EBernhardson should we close this as a duplicate and move "(full text search, go bar, ...)" as a dimension in T358352: Search Metrics - Number of user sessions using search?

Fri, Apr 12, 8:17 PM · Discovery-Search (Current work)
dr0ptp4kt updated the task description for T358351: Search Metrics - Read traffic generated by Search.
Fri, Apr 12, 8:12 PM · MW-1.43-notes (1.43.0-wmf.1; 2024-04-16), Patch-For-Review, Discovery-Search (Current work)
dr0ptp4kt updated the task description for T358350: Search Metrics - Successful searches.
Fri, Apr 12, 8:10 PM · Discovery-Search (Current work)
dr0ptp4kt updated the task description for T358351: Search Metrics - Read traffic generated by Search.
Fri, Apr 12, 8:06 PM · MW-1.43-notes (1.43.0-wmf.1; 2024-04-16), Patch-For-Review, Discovery-Search (Current work)
dr0ptp4kt updated the task description for T358351: Search Metrics - Read traffic generated by Search.
Fri, Apr 12, 8:04 PM · MW-1.43-notes (1.43.0-wmf.1; 2024-04-16), Patch-For-Review, Discovery-Search (Current work)
dr0ptp4kt updated the task description for T358352: Search Metrics - Number of user sessions using search.
Fri, Apr 12, 7:46 PM · Discovery-Search (Current work)
dr0ptp4kt updated the task description for T358352: Search Metrics - Number of user sessions using search.
Fri, Apr 12, 7:38 PM · Discovery-Search (Current work)

Wed, Apr 10

dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

Good news. With the N-triples style scholarly entity graph files, with a buffer capacity of 1000000, a write retention queue capacity of 4000, and a heap size of 31g, on the gaming-class desktop, it took about 2.40 days. Recall that with buffer capacity of 100000 it took about 3.25 days on this desktop (and again, recall that it was 5.875 days on wdqs1024). So, there was about a 35% (1.35 minus 1) speed increase with the higher buffer capacity here on this gaming-class desktop.

Wed, Apr 10, 2:59 PM · Wikidata, Discovery-Search (Current work)

Mon, Apr 8

dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

Update: With the buffer capacity at 1000000, file number 550 of the scholarly graph was imported as of Mon Apr 8 03:22:08 PM CDT 2024 . So, under 28 hours so far (buffer capacity at 100000 was more than 36 hours).

Mon, Apr 8, 9:14 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt added a comment to T358350: Search Metrics - Successful searches.

Historically this was based on dwell time as a satisfied search. Plan would be to re-use that metrics if the source data points still hold.

Mon, Apr 8, 3:57 PM · Discovery-Search (Current work)
dr0ptp4kt added a project to T361246: scap deploy should not repool a wdqs node that is depooled: Discovery-Search (Current work).
Mon, Apr 8, 3:49 PM · Release-Engineering-Team, Data-Platform-SRE, Scap, Wikidata, Wikidata-Query-Service
dr0ptp4kt added a project to T361935: Adapt the WDQS Streaming Updater to update multiple WDQS subgraphs: Discovery-Search (Current work).
Mon, Apr 8, 3:49 PM · Patch-For-Review, Discovery-Search (Current work), Wikidata
dr0ptp4kt added a project to T361950: Ensure that WDQS query throttling does not interfere with federation: Discovery-Search (Current work).
Mon, Apr 8, 3:48 PM · Discovery-Search (Current work), Wikidata
dr0ptp4kt added a project to T362060: Generalize ScholarlyArticleSplitter: Discovery-Search (Current work).
Mon, Apr 8, 3:48 PM · Patch-For-Review, Discovery-Search (Current work), Wikidata
dr0ptp4kt edited projects for T358349: Search Metrics - Number of Searches, added: Discovery-Search (Current work); removed Discovery-Search.
Mon, Apr 8, 3:46 PM · Discovery-Search (Current work)
dr0ptp4kt edited projects for T358350: Search Metrics - Successful searches, added: Discovery-Search (Current work); removed Discovery-Search.
Mon, Apr 8, 3:46 PM · Discovery-Search (Current work)
dr0ptp4kt edited projects for T358351: Search Metrics - Read traffic generated by Search, added: Discovery-Search (Current work); removed Discovery-Search.
Mon, Apr 8, 3:45 PM · MW-1.43-notes (1.43.0-wmf.1; 2024-04-16), Patch-For-Review, Discovery-Search (Current work)
dr0ptp4kt moved T358352: Search Metrics - Number of user sessions using search from needs triage to Current work on the Discovery-Search board.
Mon, Apr 8, 3:44 PM · Discovery-Search (Current work)
dr0ptp4kt triaged T358349: Search Metrics - Number of Searches as High priority.
Mon, Apr 8, 3:43 PM · Discovery-Search (Current work)
dr0ptp4kt triaged T358350: Search Metrics - Successful searches as High priority.
Mon, Apr 8, 3:43 PM · Discovery-Search (Current work)
dr0ptp4kt triaged T358351: Search Metrics - Read traffic generated by Search as High priority.
Mon, Apr 8, 3:43 PM · MW-1.43-notes (1.43.0-wmf.1; 2024-04-16), Patch-For-Review, Discovery-Search (Current work)
dr0ptp4kt triaged T358352: Search Metrics - Number of user sessions using search as High priority.
Mon, Apr 8, 3:42 PM · Discovery-Search (Current work)
dr0ptp4kt triaged T359580: CirrusSearch should not send outdated cirrussearch-request events as Low priority.
Mon, Apr 8, 3:41 PM · Discovery-Search (Current work), Patch-For-Review, CirrusSearch
dr0ptp4kt set the point value for T361114: Alert Search Platform and/or DPE SRE when Wikidata is lagged to 2.
Mon, Apr 8, 3:37 PM · Data-Platform-SRE (2024.04.15 - 2024.05.05), Wikidata, Wikidata-Query-Service
dr0ptp4kt triaged T357066: CirrusSearch\BuildDocument\BuildDocumentException: ParserOutput cannot be obtained. as Medium priority.
Mon, Apr 8, 3:34 PM · MW-1.43-notes (1.43.0-wmf.2; 2024-04-23), Patch-For-Review, Discovery-Search (Current work), User-brennen, CirrusSearch, Wikimedia-production-error
dr0ptp4kt closed T356303: Review wikitech:Search and write processes for k8s world as Resolved.
Mon, Apr 8, 3:33 PM · Data-Platform-SRE (2024.03.25 - 2024.04.14), Documentation, Discovery-Search (Current work)
dr0ptp4kt assigned T356302: setup production Cirrus Streaming Updater alerts to bking.
Mon, Apr 8, 3:31 PM · Discovery-Search (Current work)
dr0ptp4kt moved T356302: setup production Cirrus Streaming Updater alerts from Ready for Dev -- SWE to Needs review on the Discovery-Search (Current work) board.
Mon, Apr 8, 3:31 PM · Discovery-Search (Current work)
dr0ptp4kt closed T350974: search/glent fails on Java 11 as Declined.

Closing this out until newer Java comes to the analytics cluster.

Mon, Apr 8, 3:29 PM · Discovery-Search (Current work), ci-test-error
dr0ptp4kt assigned T328330: Create SLI / SLO on Search update lag to pfischer.
Mon, Apr 8, 3:25 PM · Data-Platform-SRE, Discovery-Search (Current work)

Sun, Apr 7

dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

With bufferCapacity at 1000000, kicked it off again with the scholarly article entity graph files:

Sun, Apr 7, 5:15 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

Update. On the gaming-class machine it took about 3.25 days to import the scholarly article entity graph, using a buffer capacity of 100000 (compare this with 5.875 days on wdqs1024). This resulted in 7_643_858_078 triples as expected. Next up will be with a buffer capacity of 1000000 to see if there is any obvious difference in import time.

Sun, Apr 7, 4:34 PM · Wikidata, Discovery-Search (Current work)

Fri, Apr 5

dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

Just updating on how far along this run is, file 550 of the scholarly article entity side of the graph is being processed. There are files 0 through 1023 for this side of the graph. Note that I did think to tee output this time around so that generally/hopefully there's more info available to review output, stack traces (although hopefully there are none), and so on, should it be needed.

Fri, Apr 5, 3:23 PM · Wikidata, Discovery-Search (Current work)

Thu, Apr 4

dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

Following roughly the procedure in P54284 to rename the Spark-produced graph files (and updating loadData.sh with FORMAT=part-%05d-46f26ac6-0b21-4832-be79-d7c8709f33fb-c000.ttl.gz and still having a date call after each curl in it), I kicked off an import of the scholarly article entity graph like so to see how it goes with a buffer capacity of 100000:

Thu, Apr 4, 11:09 AM · Wikidata, Discovery-Search (Current work)

Wed, Apr 3

dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

This morning of April 3 around 6:25 AM I had SSH'd to check progress, and it was working, but going slowly, similar to the day before. It was on a file number in the 1200s, but I didn't write down the number or copy terminal output; I do remember seeing it was taking around 796 seconds for one of the files at that time. Look at the previous comment, you'll see those were going slow; not surprising as we know imports on these munged files are slower upon more stuff is imported.

Wed, Apr 3, 9:15 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt created P59389 BlazeGraph stack trace during munged file import, obtained from screen backscroll.
Wed, Apr 3, 7:52 PM

Tue, Apr 2

dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

Now this is interesting: we're now past 4 days (about 4 days and 1 hour) of this running, and with buffer capacity at 100000 instead of 1000000 (but this time without any gap between the batches of files), there's still a good way to go yet.

Tue, Apr 2, 7:57 PM · Wikidata, Discovery-Search (Current work)

Mon, Apr 1

dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

The run with with buffer at 1000000 and heap size at 31g and queue capacity at 4000 on the gaming-class desktop completed.

Mon, Apr 1, 8:44 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt added a comment to T361480: [citation-needed] "terms of use" and "privacy policy" links for first time use possibly not routed from "Generative AI experiment" overlay.

See attached. Here I was trying to click on the "terms of use" and "privacy policy" links, with no luck. Then see a click on the footer of the overlay working okay.

Mon, Apr 1, 4:06 PM · Future-Audiences
dr0ptp4kt created T361480: [citation-needed] "terms of use" and "privacy policy" links for first time use possibly not routed from "Generative AI experiment" overlay.
Mon, Apr 1, 4:02 PM · Future-Audiences
dr0ptp4kt added a comment to T346464: Experiment with InnoDB buffer pool size on clouddb1019.eqiad.wmnet.

Makes sense to shelve for now @Marostegui.

Mon, Apr 1, 3:13 PM · Data-Services, DBA

Fri, Mar 22

dr0ptp4kt added a comment to T305688: Make HTML Dumps available in hadoop.

I'm interested as well, as I intend to looking at some image dumping stuff, and the surrounding HTML will be important for understanding context.

Fri, Mar 22, 8:45 PM · Data-Engineering (Q4 2024 April 1st - June 30th), Structured-Data-Backlog
dr0ptp4kt added a comment to T252227: Mobile redirects drop provenance parameters.

Okay, if I understand correctly, then the idea would be to...

Fri, Mar 22, 7:09 PM · Data-Engineering, Data Pipelines, Traffic-Icebox, SRE

Thu, Mar 21

dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

By the way, I'm attempting a run for the first 1332 munged files (one shy of the 1333 where terminated last time around) with buffer at 1000000 and heap size at 31g and queue capacity at 4000 on the gaming-class desktop to see whether this imports smoothly and whether performance gains are noticeable.

Thu, Mar 21, 5:37 PM · Wikidata, Discovery-Search (Current work)

Wed, Mar 20

dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

The run to check with heap size of 31g, queue capacity of 8000, and buffer at 1000000 stalled at file 107.

Wed, Mar 20, 7:33 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

Attempting a run with a queue capacity of 8000 and buffer of 1000000 and heap size of 16g on the gaming-class desktop to mimic the MacBook Pro, things were slower than a queue capacity of 4000 and buffer of 1000000 and heap size of 31g on the gaming-class desktop.

Wed, Mar 20, 6:11 PM · Wikidata, Discovery-Search (Current work)

Mar 19 2024

dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

About Amazon Neptune

Mar 19 2024, 9:25 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

Going for much of the full import

Mar 19 2024, 8:37 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

More about bufferCapacity

Mar 19 2024, 8:14 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

More about NVMe versus SSD

Mar 19 2024, 8:10 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

AWS EC2 servers

Mar 19 2024, 7:36 PM · Wikidata, Discovery-Search (Current work)

Mar 8 2024

dr0ptp4kt updated subscribers of T359062: Assess Wikidata dump import hardware.

@ssingh would you mind if the following command is run on one of the newer cp#### hosts with a new higher write throughput NVMe? If so, got a recommended node? I don't have access, but I think @bking may.

Mar 8 2024, 3:42 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

Thanks @bking ! It looks like the NVMe in this one is not a higher speed one for writes (based on the reported model from lsblk I think this is a https://i.dell.com/sites/csdocuments/Shared-Content_data-Sheets_Documents/en/Dell-PowerEdge-Express-Flash-NVMe-Mixed-Use-PCIe-SSD.pdf ), and I'm also wondering if perhaps its write performance has degraded with age. I'll paste in the results here, but this was slower than the other servers, ironically (although not surprisingly because of the slower NVMe and slightly slower processor). This slower write speed is atypical of the other NVMes I've encountered. I believe the newer model ones are rated for 6000 MB/s for writes. But, I'm going to ping on task to see if we can get a comparative read of disk throughput from one of the newer and faster cp#### NVMes.

Mar 8 2024, 3:36 PM · Wikidata, Discovery-Search (Current work)

Mar 7 2024

dr0ptp4kt updated the task description for T359062: Assess Wikidata dump import hardware.
Mar 7 2024, 12:22 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt updated the task description for T359062: Assess Wikidata dump import hardware.
Mar 7 2024, 12:20 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt added a comment to T359062: Assess Wikidata dump import hardware.

First, adding some commands that were used for Blazegraph imports on Ubuntu 22.04. I had originally tried a good number of EC2 instance types, and then after that went back to focus on just four of them with a sequence of repeatable commands (this wasn't scripted, as I didn't want to spend time automating and also wanted to make sure I got the systems' feedback along the way). I forgot to grab RAM frequency as a routine step when running these commands (I recall checking on one server maybe in the original checks, and did look at my Alienware), but generally servers are DDR4 unless the documentation in AWS says DDR5 (for my 2018 Alienware and 2019 MacBook Pro they're DDR4, BTW).

Mar 7 2024, 12:09 PM · Wikidata, Discovery-Search (Current work)

Mar 6 2024

dr0ptp4kt updated the task description for T359062: Assess Wikidata dump import hardware.
Mar 6 2024, 9:38 PM · Wikidata, Discovery-Search (Current work)

Mar 5 2024

dr0ptp4kt added a comment to T252227: Mobile redirects drop provenance parameters.

Originally, the thought was to be able to simply count relative volume of these types of inbound taps/clicks. Although we want fidelity on whether a link actually resolves to a page (and I know there are Phabricator comments about this here and elsewhere), often a simple count is sufficient to know if there's any traction whatsoever. I see that it's considered desirable to have a definite mapping of bona fide pageviews or previews (or other things of that nature) to these wprov values - makes sense.

Mar 5 2024, 1:31 PM · Data-Engineering, Data Pipelines, Traffic-Icebox, SRE
dr0ptp4kt added a comment to T358727: Reclaim recently-decommed CP host for WDQS (see T352253).

@VRiley-WMF any pointers on how to iDRAC / iLO to this node and establish with a hostname of wdqs1025.eqiad.wmnet? I'm wondering if maybe there's a direct IP or IPs given that there don't seem to be DNS records for cp1086.eqiad.wmnet or cp1086.mgmt.eqiad.wmnet?

Mar 5 2024, 12:51 PM · Discovery-Search (Current work), Data-Platform-SRE (2024.03.04 - 2024.03.24), Wikidata, wmde-wikidata-tech, SRE, ops-eqiad

Mar 4 2024

dr0ptp4kt updated the task description for T359062: Assess Wikidata dump import hardware.
Mar 4 2024, 4:32 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt moved T359062: Assess Wikidata dump import hardware from Incoming to Current work on the Wikidata-Query-Service board.
Mar 4 2024, 4:30 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt updated the task description for T359062: Assess Wikidata dump import hardware.
Mar 4 2024, 4:29 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt updated the task description for T359062: Assess Wikidata dump import hardware.
Mar 4 2024, 4:17 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt moved T359062: Assess Wikidata dump import hardware from Incoming to In Progress on the Discovery-Search (Current work) board.
Mar 4 2024, 3:28 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt changed the status of T359062: Assess Wikidata dump import hardware from Open to In Progress.
Mar 4 2024, 3:28 PM · Wikidata, Discovery-Search (Current work)
dr0ptp4kt created T359062: Assess Wikidata dump import hardware.
Mar 4 2024, 3:24 PM · Wikidata, Discovery-Search (Current work)

Mar 1 2024

dr0ptp4kt added a comment to T358727: Reclaim recently-decommed CP host for WDQS (see T352253).

Thanks @VRiley-WMF ! @bking is up next for imaging, I think.

Mar 1 2024, 7:30 PM · Discovery-Search (Current work), Data-Platform-SRE (2024.03.04 - 2024.03.24), Wikidata, wmde-wikidata-tech, SRE, ops-eqiad

Feb 29 2024

dr0ptp4kt added a parent task for T358727: Reclaim recently-decommed CP host for WDQS (see T352253): T358533: Hardware requests for Search Platform FY2024-2025.
Feb 29 2024, 9:28 PM · Discovery-Search (Current work), Data-Platform-SRE (2024.03.04 - 2024.03.24), Wikidata, wmde-wikidata-tech, SRE, ops-eqiad
dr0ptp4kt added a subtask for T358533: Hardware requests for Search Platform FY2024-2025: T358727: Reclaim recently-decommed CP host for WDQS (see T352253).
Feb 29 2024, 9:28 PM · Data-Platform-SRE (2024.03.25 - 2024.04.14)
dr0ptp4kt added a parent task for T358727: Reclaim recently-decommed CP host for WDQS (see T352253): T336443: Investigate performance differences between wdqs2022 and older hosts.
Feb 29 2024, 9:26 PM · Discovery-Search (Current work), Data-Platform-SRE (2024.03.04 - 2024.03.24), Wikidata, wmde-wikidata-tech, SRE, ops-eqiad
dr0ptp4kt added a subtask for T336443: Investigate performance differences between wdqs2022 and older hosts: T358727: Reclaim recently-decommed CP host for WDQS (see T352253).
Feb 29 2024, 9:26 PM · Patch-For-Review, Data-Platform-SRE (2024.04.15 - 2024.05.05)
dr0ptp4kt added a comment to T252227: Mobile redirects drop provenance parameters.

Hi team - @lbowmaker asked if I could take a look at this and provide some context. I was having a think on this, and I'd like to ponder up to a few more days and provide some thoughts.

Feb 29 2024, 12:02 PM · Data-Engineering, Data Pipelines, Traffic-Icebox, SRE

Feb 28 2024

dr0ptp4kt added a comment to T352253: Decommission task for old cp hosts (cp1075-1090).

@bking , @RKemper , and I met today. @bking has an action on this here ticket (@bking LMK in case I need to chime in on anything!). Thanks!

Feb 28 2024, 8:14 PM · SRE, ops-eqiad, DC-Ops, Traffic

Feb 27 2024

dr0ptp4kt updated subscribers of T352253: Decommission task for old cp hosts (cp1075-1090).

After setup, I would be interested in using it for 6 weeks if that's okay (hopefully things would only take 4 weeks, but there's some PTO and real life stuff always comes up). Would that be okay?

Feb 27 2024, 10:50 PM · SRE, ops-eqiad, DC-Ops, Traffic

Feb 9 2024

dr0ptp4kt added a project to T357064: Use custom CDN if possible for Jupyter HTML exported notebooks: Security.
Feb 9 2024, 6:25 PM · Data-Platform-SRE, Security, Data-Engineering, Data-Engineering-Jupyter

Feb 8 2024

dr0ptp4kt updated the task description for T357064: Use custom CDN if possible for Jupyter HTML exported notebooks.
Feb 8 2024, 9:37 PM · Data-Platform-SRE, Security, Data-Engineering, Data-Engineering-Jupyter
dr0ptp4kt added a project to T357064: Use custom CDN if possible for Jupyter HTML exported notebooks: Data-Engineering-Jupyter.
Feb 8 2024, 9:07 PM · Data-Platform-SRE, Security, Data-Engineering, Data-Engineering-Jupyter
dr0ptp4kt added a project to T357064: Use custom CDN if possible for Jupyter HTML exported notebooks: Data-Platform-SRE.
Feb 8 2024, 9:07 PM · Data-Platform-SRE, Security, Data-Engineering, Data-Engineering-Jupyter
dr0ptp4kt created T357064: Use custom CDN if possible for Jupyter HTML exported notebooks.
Feb 8 2024, 9:02 PM · Data-Platform-SRE, Security, Data-Engineering, Data-Engineering-Jupyter
dr0ptp4kt awarded T349512: [Analytics] Collect multiple sets of SPARQL queries a Party Time token.
Feb 8 2024, 11:48 AM · Wikidata Analytics (Kanban), Discovery-Search (Current work), Wikidata, Wikidata-Query-Service

Feb 5 2024

dr0ptp4kt added a comment to T355037: Compare the performance of sparql queries between the full graph and the subgraphs.

I summarized at https://wikitech.wikimedia.org/wiki/Wikidata_Query_Service/Graph_split_IGUANA_performance . When we have a mailing list post during the next week or so, we'll want to move this to be a subpage of the target page of the post.

Feb 5 2024, 9:58 PM · Discovery-Search (Current work), Wikidata

Feb 2 2024

dr0ptp4kt added a comment to T355037: Compare the performance of sparql queries between the full graph and the subgraphs.

@dr0ptp4kt thanks! is the difference in the number of successful queries only explained by the improvement in query time or are there some improvements in the number of queries that timeout as well?

Feb 2 2024, 8:39 PM · Discovery-Search (Current work), Wikidata