Page MenuHomePhabricator

Publish, and maintain ASPA records for valid AS14907 upstreams
Open, LowPublic

Description

Unlike RPKI ROA validation, which protects against origin AS manipulation, Autonomous System Provider Authorization (ASPA) provides path plausibility, increasing the likelihood of catching the valley-free routing violation type of route leaks. https://manrs.org/2023/02/unpacking-the-first-route-leak-prevented-by-aspa/ provides a summary of ASPA. In essence, an AS participating in ASPA must publish an ASPA record, defining the Set of Provider ASes, which is a list of valid Transit ASes for AS14907. This is the draft ASPA profile, and this explains the verification algorithm.

Unfortunately, broad support for ASPA validation is estimated no sooner than 2026/2027. Nevertheless, it should be possible to publish ASPA records in RPKI through the ARIN portal. I hope the more ASes publish such records, the sooner network vendors (and developers of BGP daemons, e.g. FRR and Bird) will implement ASPA validation.

In June 2024, 77 ASes had published ASPA records. At first glance, Wikimedia may become the Internet's largest resource to publish ASPA records :)

However, the ASPA record is yet another duplicate of the transit_provider list in Homer, and the export policies defined in our aut-num objects. Our export policies in the IRRs already do not match up what's in Homer (for instance, we removed a few Transit providers during the knams migration, but they're still present in our export policies), and adding an ASPA record will make it even harder to stay in sync. Perhaps we can script something that checks whether the IRRs and ASPA record(s) still match with the source of truth, being Homer?

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Nevertheless, it should be possible to publish ASPA records in RPKI through the ARIN portal

I looked a bit around Arin's RPKI's portal but couldn't find it, is there doc about it ?

However, the ASPA record is yet another duplicate of the transit_provider list in Homer, and the export policies defined in our aut-num objects. Our export policies in the IRRs already do not match up what's in Homer (for instance, we removed a few Transit providers during the knams migration, but they're still present in our export policies), and adding an ASPA record will make it even harder to stay in sync. Perhaps we can script something that checks whether the IRRs and ASPA record(s) still match with the source of truth, being Homer?

I think step 1 is to update the doc https://wikitech.wikimedia.org/wiki/Add_transit_provider to add the steps you mentioned. As well as add steps to do when removing a transit.

Before working on automating the verification (that we didn't forget any step) or the actual implementation, we should look at using Netbox as source of truth.

Follow-up from IRC: Wikimedia uses the Hosted RPKI, but we assume the ARIN portal just doesn't support anything else than ROAs. There is an ASPA record for AS11358, whose ASN is controlled by ARIN, and we think they either use Hybrid RPKI, where ARIN still hosts the RPKI objects through their Repository Publication Service. Technically, Krill could both act as the CA and create ASPA records, and it is possible that AS11358 and others are doing this.

RIPE NCC is working on ASPA support for their Hosted RPKI, but our ASN has been assigned by ARIN, so you would need some kind of cross-RIR resource certification to have RIPE NCC manage AS14907 RPKI objects, which probably doesn't make sense. If I read Section 5.12 of the RPKI-RTR v2 IETF draft correctly, it should be viable have multiple ASPA records with multiple CAs, in very unusual situations we probably do not qualify for. To conclude: RIPE NCC UI support is probably not what we looking for...

ARIN is not going to add support to their Hosted RPKI till the IETF standard has been ratified. Job Snijders predicted ratification to happen in 2024, and 2025 for RIRs adding ASPA support to the RPKI sections in their portals.

Unless we feel comfortable enough to move to a Hybrid RPKI, publishing ASPA records has to be postponed. And perhaps we don't feel comfortable hosting our own CA, do we?

[...]

However, the ASPA record is yet another duplicate of the transit_provider list in Homer, and the export policies defined in our aut-num objects. Our export policies in the IRRs already do not match up what's in Homer (for instance, we removed a few Transit providers during the knams migration, but they're still present in our export policies), and adding an ASPA record will make it even harder to stay in sync. Perhaps we can script something that checks whether the IRRs and ASPA record(s) still match with the source of truth, being Homer?

I think step 1 is to update the doc https://wikitech.wikimedia.org/wiki/Add_transit_provider to add the steps you mentioned. As well as add steps to do when removing a transit.

Ta da: https://wikitech.wikimedia.org/w/index.php?title=Adding_and_removing_transit_providers&diff=2218856&oldid=2042295. Can you verify this is correct? There are lots of references to private Phabricator tasks, and of course I have never dealed with WMF's transit providers before.

Before working on automating the verification (that we didn't forget any step) or the actual implementation, we should look at using Netbox as source of truth.

As far as I understand, the actual circuits are already managed in Netbox, but the Homer template for the Transit group (which is what actually manages the BGP sessions on the CRs) is expanded based on yaml configuration in config/devices.yaml. On top of that, we have site-wide policies, and BFD + FlowSpec configuration for transit providers in config/common.yaml. Using Netbox to manage the AS-specific import/export policies does not really make sense (the policies are free-form text), and I'm not sure what Netbox data model is suitable for modelling BGP sessions. Something I can think of is some kind of CI/test that checks whether the Homer transit and transit_provider keys contain ASes that are not "transit" Providers in Netbox, to at least have some kind of Netbox-Homer verification, and said data may also be useful for cross-checking the IRR databases and ASPA objects. If Wikimedia has some custom data model for any of the use cases listed, let me know!

Question for both of you (Arzhel/Cathal) is whether we feel OK with moving to a Hybrid RPKI model for ASPA support, or if we should wait till at least ARIN supports ASPA mutations through their portal.

Ta da: https://wikitech.wikimedia.org/w/index.php?title=Adding_and_removing_transit_providers&diff=2218856&oldid=2042295. Can you verify this is correct? There are lots of references to private Phabricator tasks, and of course I have never dealed with WMF's transit providers before.

That's great and sound much more pro :)
I did some tiny changes

Adjust AS14907 ASPA to reflect the transit AS (task T372161)

This should be left out for now as we don't have a documented way for doing so.

If Wikimedia has some custom data model for any of the use cases listed, let me know!

At some point we should look at using a real database to store those objects, Netbox through netbox-bgp could be a good option. As it increases our dependency on a 3rd party tool, we need to make sure it's worth it.

Question for both of you (Arzhel/Cathal) is whether we feel OK with moving to a Hybrid RPKI model for ASPA support, or if we should wait till at least ARIN supports ASPA mutations through their portal.

Could you ask ARIN/RIPE if they have an ETA on supporting ASPA on their portal ?
So far I'm on the fence about having to manage new services in house.

ASPA support has been added on RIPE's quarterly planning: https://www.ripe.net/manage-ips-and-asns/resource-management/rpki/rpki-planning-and-roadmap/#item-3-support-aspa. However, while IETF consensus seems to be close, there is still some discussion and [RIPE wants] to await formal consensus before implementing this, so it depends on promoting the IETF drafts, I suppose. An update has been published since then, and interestingly, an extension has been published a few months ago. Activity on the SIDROPS mailing list is fine, though.

Looking at this presentation from June 2024's SIDROPS meeting, one of the current issues lies in the "RPKI-to-Router v2 draft" (8210bis), seemingly the result of disagreement between the draft's editors and the SIDROPS working group members. And because most open issues as of december 2024 depend on changes to 8210bis, I'm wondering what the process will look like from now on.

[...]
Could you ask ARIN/RIPE if they have an ETA on supporting ASPA on their portal ?
So far I'm on the fence about having to manage new services in house.

I understand hosting our own CA setup is not the ideal way to go. Looks like we'll need to wait until the drafts have been published.

Thanks for keeping up to date on this @Southparkfan!

I understand hosting our own CA setup is not the ideal way to go. Looks like we'll need to wait until the drafts have been published.

Yeah the effort/benefit ratio doesn't really make sense to start hosting our own CA to do this. One also needs to be mindful that until the standards are fully agreed, supported by the RIRs, and generally implemented more widely, having such records is going to be of limited benefit.