Change Details

Over the last couple years, we have been forced to wage war to a new cadre of scrapers of the wikimedia projects, which have no respect for netiquette, policies, or robots.txt. These predatory scrapers, mostly born out of the "AI" gold rush, tend to use new, sneaky ways of scraping our content. In this short presentation, I'll explain how the new trend of residential proxies pose a novel challenge to the Wikimedia site reliability engineers, and to the community alike. What is a residential proxy? How do they get in your network? Do you have a residential proxy on your device right now? Why is your smart plug making 30 GB of internet traffic? Ok, what are we doing about it? All these questions, and many more you didn't want to know the answer to, will be answered in this session.

Over the last couple years, we have been forced to wage war to a new cadre of scrapers of the wikimedia projects, which have no respect for netiquette, policies, or robots.txt. These predatory scrapers, mostly born out of the "AI" gold rush, tend to use new, sneaky ways of scraping our content. In this short presentation, I'll explain how the new trend of residential proxies pose a novel challenge to the Wikimedia site reliability engineers, and to the community alike. What is a residential proxy? How do they get in your network? Do you have a residential proxy on your device right now? Why is your smart plug making 30 GB of internet traffic? Ok, what are we doing about it? All these questions, and many more you didn't want to know the answer to, will be answered in this session. My hope is that you'll leave it with a reduced trust in humanity. Duration: 20 minutes + space for Q&A

Over the last couple years, we have been forced to wage war to a new cadre of scrapers of the wikimedia projects, which have no respect for netiquette, policies, or robots.txt. These predatory scrapers, mostly born out of the "AI" gold rush, tend to use new, sneaky ways of scraping our content. In this short presentation, I'll explain how the new trend of residential proxies pose a novel challenge to the Wikimedia site reliability engineers, and to the community alike. What is a residential proxy? How do they get in your network? Do you have a residential proxy on your device right now? Why is your smart plug making 30 GB of internet traffic? Ok, what are we doing about it? All these questions, and many more you didn't want to know the answer to, will be answered in this session. My hope is that you'll leave it with a reduced trust in humanity. Duration: 20 minutes + space for Q&A