As we move closer to setting up the new data center in Brazil, we should discuss how we plan to ramp up traffic to the new data center, slowly and progressively to warm the caches, before turning it on for the rest of the continent. This task is meant to be a discussion about that with the aim of making a decision before April.
Currently and as per the [[ https://gerrit.wikimedia.org/r/plugins/gitiles/operations/dns/+/refs/heads/master/geo-maps | geo-maps ]] file in the operations/dns repository, we do not have a defined geo-map for any country in South America, so traffic from there currently goes to `codfw` (since the September 2023 switchover) or otherwise `eqiad`. This means whatever entry we add will be under a new [[ https://github.com/gdnsd/gdnsd/wiki/GdnsdPluginGeoip#geoip2-location-data-hierarchy | country, sudivision, city, or subnet]].
For a very high-level overview, traffic to South America as a percentage of our global traffic (30-days):
| North America | 35.95% |
| Europe | 33.63% |
| Asia | 21.75% |
| **South America** | **4.46%** |
| Africa | 1.72% |
| Oceania | 1.26% |
| Unknown | 1.24% |
| Antarctica | 0.00% |
The current traffic trends from South America indicate that we get the most traffic from Brazil and the least from Paraguay:
Brazil
Argentina
Colombia
Chile
Peru
Venezuela
Ecuador
Bolivia
Uruguay
Paraguay
{F42394055}
There is additionally one more important thing to keep in mind: Brazil is the only Portuguese-speaking country in South America, so there will be variations in user traffic between pt.wikipedia.org vs es.wikipedia.org, which should further affect our decision on how to warm up the site.
Within Brazil, we get the most traffic from:
Unknown
São Paulo
Rio de Janeiro
Brasília
Belo Horizonte
Curitiba
Fortaleza
Porto Alegre
Salvador
Campinas
Ignoring "Unknown" for which we can't do anything, not surprisingly, we get the most traffic from São Paulo and Rio.
{F42394057}
With the above information, we can turn on traffic for one of the cities above for Brazil that is not São Paulo or Rio*, and additionally, one or more Spanish-speaking country from where we don't get a lot of traffic. There has been some discussion on this topic and it is worth mentioning that perhaps we should turn on traffic for all of Brazil before moving anywhere else in the continent. So while that still means figuring out the answers to the above questions, the DNS changes won't happen at the same time.
(* - we can look into subdivision as well, however, the data looks comparable to the city-level data.)
Sometimes, the percantage of traffic helps visualize things better than graphs. For South America, percentage split of traffic (for just a given day but the trend holds true):
| Brazil | 37.81% |
| Argentina | 15.54% |
| Colombia | 14.44% |
| Chile | 9.28% |
| Peru | 8.02% |
| Venezuela | 6.27% |
| Ecuador | 3.85% |
| Bolivia | 2.68% |
| Uruguay | 1.46% |
| Paraguay | 0.65% |
Percentage of traffic ''excluding'' Brazil (thus, only Spanish speaking countries) from South America:
| Argentina | 24.99% |
| Colombia | 23.22% |
| Chile | 14.92% |
| Peru | 12.90% |
| Venezuela | 10.08% |
| Ecuador | 6.20% |
| Bolivia | 4.30% |
| Uruguay | 2.34% |
| Paraguay | 1.05% |
Once we have decided this, we can finalize a time as well for rolling out the change. Note that with the lowering of TTL in T140365, traffic will move over in five minutes instead of ten, so the change will be more "sudden", at least relative to how it has been in the past.
To summarize, the decisions we are looking to make are:
- Should we turn on the site for all of Brazil before moving on to the rest of the continent, or should we do it at the same time and pick Brazil and one other Spanish-speaking country?
- Regardless of the above, we should figure out which regions in Brazil and which country in South America we should consider for warming up the site.
- The northern part of South America -- Ecuador, Colombia, Venezuela, Guyana -- should traffic from there continue to go to eqiad/codfw (perhaps `codfw`) or should we send that traffic to `magru` as well? (This question might be better answered by actual data and I notice we have RIPE Atlas probes in each of these countries but perhaps some internal data as well?)