IssueTracker

Hello, We are reviewing your request and need to verify it is from the organization which owns this address block. I have reached out to the owner of the address block, e.zuniga@libertycr.com, for verification.

Alternatively, if you can reply or send me an email from j.medrano@libertycr.com or ja.benavides@libertycr.com. This would allow us to move to the next steps.

Thanks!

Yes, I heard back from j.medrano@libertycr.com confirming this request.

We've moved forward with this request and the changes (increasing client QPS to 5,000) will roll out next week.

Running the test at https://cmdns.dev.dns-oarc.net/ when using Google Public DNS servers on the client shows that Google DNS does not verify BGP Route Validity of nameservers. This used to be the case in 2022, but not anymore. Cloudflare and other public DNS resolvers do implement RPKI ROA validation.

Hey there fdbrunetderochebrune@gmail.com, so that I can maybe better understand your question / statement:

 "Running the test at https://cmdns.dev.dns-oarc.net/ when using
  Google Public DNS servers on the client shows that Google DNS
  does not verify BGP Route Validity of nameservers."

I think you mean that the network elements that support the Google Public DNS service, AS15169 generally, used to do some form of RPKI validation, but does not for your test. Is this an accurate restatement of your ... statement (it's not really a question I suppose) ?

I think you really mean that the AS15169 network was performing something like 'Route Origin Validation' using the RPKI (Route Origin Authorization) data published by many networks (to include AS15169).

I apologize for not really knowing what that test website does... do you know when you may have tested last and seen more positive results? (it would be good to know when this might have changed state, and knowing what/how the test works might help us diagnose where we went sideways in AS15169's views of the world)

thanks! -chris

(added jay for visibility as well)

When performing the test, a subdomain is generated with an authoritative nameserver that has an invalid ROA. An example test I just did showed the following domains being queried (click on the transport graph and then expand RPKI IPv4):

Q	2a00:1450:4013:c16::110	64193	48124	udp	IN	HTTPS	opi8Nc0U9H7qlATL9sK2c33KNC.cmdNS.dEv.Dns-OArC.NeT.

The nameserver for this domain is:

$ dig ns opi8Nc0U9H7qlATL9sK2c33KNC.cmdNS.dEv.Dns-OArC.NeT. +short
ns.opi8Nc0U9H7qlATL9sK2c33KNC.cmdNS.dEv.Dns-OArC.NeT.

The IP address is:

$ dig ns.opi8Nc0U9H7qlATL9sK2c33KNC.cmdNS.dEv.Dns-OArC.NeT. +short
185.49.142.4

Checking this IP in HE's BGP viewer shows that this block is ROA Signed but Invalid: https://bgp.he.net/ip/185.49.142.4

My ISP (KPN NL AS1136) drops invalid prefixes correctly, which makes me unable to perform the query myself:

$ dig ns.opi8Nc0U9H7qlATL9sK2c33KNC.cmdNS.dEv.Dns-OArC.NeT. +short @185.49.142.4
;; communications error to 185.49.142.4#53: timed out
;; communications error to 185.49.142.4#53: timed out
;; communications error to 185.49.142.4#53: timed out

; <<>> DiG 9.18.25 <<>> ns.opi8Nc0U9H7qlATL9sK2c33KNC.cmdNS.dEv.Dns-OArC.NeT. +short @185.49.142.4
;; global options: +cmd
;; no servers could be reached

The query from Google came from IP 2a00:1450:4013:c16::110 (AS15169) and while it is signed and valid, it does not seem to filter invalid ROA itself. Hence why Google Public DNS returns a response:

dig ns.opi8Nc0U9H7qlATL9sK2c33KNC.cmdNS.dEv.Dns-OArC.NeT. +short @8.8.8.8
185.49.142.4

Cloudflare also does not give a result, meaning they check for valid ROA. It throws EDE 22 (No Reachable Authority):

dig ns.opi8Nc0U9H7qlATL9sK2c33KNC.cmdNS.dEv.Dns-OArC.NeT. @1.1.1.1

; <<>> DiG 9.18.25 <<>> ns.opi8Nc0U9H7qlATL9sK2c33KNC.cmdNS.dEv.Dns-OArC.NeT. @1.1.1.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 20835
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; EDE: 9 (DNSKEY Missing): (no SEP matching the DS found for opi8nc0u9h7qlatl9sk2c33knc.cmdns.dev.dns-oarc.net.)
; EDE: 22 (No Reachable Authority): (at delegation opi8nc0u9h7qlatl9sk2c33knc.cmdns.dev.dns-oarc.net.)
;; QUESTION SECTION:
;ns.opi8Nc0U9H7qlATL9sK2c33KNC.cmdNS.dEv.Dns-OArC.NeT. IN A

OpenDNS returns a SERVFAIL for this domain.

howdy fdbrunetderochebrune !

Your example here talks about v4 prefixes and then v6 queries :( but.. sure, ok the gist is:

A test sets up a situation where queries should be destined to an
invalid prefix, those should not succeed

For your v4 example I see the /23 that covers the /24 in our view, and that the same origin AS is announcing both the /23 and /24 (and that the /23 is RPKI valid, actually).

For the v6 path I can't tell because it looks (to me) like the infra that sets up the tests tears it down already :( Do you happen to have the v6 prefixes handy?

All of this aside, I failed to notice that the issue here was opened in 2021 ;( I think our timelines for our RPKI deployment(s) are mostly here:

https://cloud.google.com/blog/products/networking/how-google-is-working-to-improve-internet-routing-security

though I admit we slipped a bit on the timing :( (bugs, resources, pandemic, etc) I believe we're still pushing forward to a conclusion that has 'drop invalids' as a goal, though.

Apologies, I indeed showed a query by a v6 address. The Google DNS makes a request on v4 and v6 simultaneously it seems:

 Q	173.194.170.67	55389	37141	udp	IN	A	m6LfnLpAJl5I91r1nK5gFb668o.cMDnS.Dev.dNS-oaRc.Net.
 Q	2a00:1450:4013:c06::106	1775	62224	udp	IN	AAAA	m6LFNlpAjL5i91r1Nk5gFb668o.CMDnS.deV.DNS-OaRc.nEt.
 R	173.194.170.67	55389	37141	udp	IN	A	m6LfnLpAJl5I91r1nK5gFb668o.cMDnS.Dev.dNS-oaRc.Net.
 R	2a00:1450:4013:c06::106	1775	62224	udp	IN	AAAA	m6LFNlpAjL5i91r1Nk5gFb668o.CMDnS.deV.DNS-OaRc.nEt.

The nameserver for v4 is 185.49.142.4 and the /23 prefix is valid, but shouldn't the more specific prefix /24 be used?

The nameserver for the v6 test is at 2a04:b907::4, again we see a valid /47 but a more specific /48 that's invalid (https://bgp.he.net/ip/2a04:b907::4).

Google Public DNS used to pass the test, but now seems not to. Is this because some logic has been changed? Like it ignores invalid more specific announcements when there is a valid less specific announcement? No other AS that implements RPKI does this AFAIK.

"The nameserver for v4 is 185.49.142.4 and the /23 prefix is valid, but shouldn't the more specific prefix /24 be used?"

it might, but if it's filtered then the /23 would be followed. The v6 prefix has the same sort of setup it seems, same pattern of valid/invalid and both prefixes are visible inside 15169, on the same set of as-paths actually.

sorry I didn't reply to this question:

"Google Public DNS used to pass the test, but now seems not to. Is this because some logic has been changed? Like it ignores invalid more specific announcements when there is a valid less specific announcement? No other AS that implements RPKI does this AFAIK."

Possibly in the past we didn't see the less specific prefixes, I can't tell from ~2yrs back :(

" No other AS that implements RPKI does this AFAIK."

I don't understand this statement, sorry. In the extreme, using your prefix as an example: "If 15169 ignored 185.49.142.0/24 and all less specifics (the /23) we would also not follow a default"

that seems like unexpected behavior... The routing system SHOULD (in the rpki/roa/ov endstate) ignore your invalid /24 and follow the valid /23. I expect 'all' bgp implementations follow that logic.

I did a little experiment and tried to reach 185.49.142.4 via a couple of AS that implement RPKI:

1136   KPN NL
13335  CLOUDFLARENET
25369  Hydra Communications Ltd
33915  Vodafone Libertel B.V.
136787 TEFINCOM S.A.
198890 Albanian Fiber Telecommunications
203020 HostRoyale
207137 PacketHub S.A.

and was not able to reach it. Via Google One VPN I am able to connect via AS36492, with 15169 and 36385 being its only peers. I will make an assumption that the RPKI validation/router is the same as on 15169 (is that the case?). Via 36492 I am able to reach 185.49.142.4. Furthermore, using https://isbgpsafeyet.com/ I am getting: "Your ISP (AS36492) does not implement BGP safely. It should be using RPKI to protect the Internet from BGP hijacks." This test tries to reach invalid.rpki.cloudflare.com, which has addresses:

103.21.244.8
103.21.244.9
2606:4700:7000::6715:f408
2606:4700:7000::6715:f409

The v4 addresses come from a single /24 announcement which is signed and invalid. The v6 addresses come from a single /48 announcement which is signed and invalid as well. Should these announcements not be ignored?

Your test using 'not google' networks I can't really speak to with any authority at all :( I'd guess they don't see the same view of the Internet that AS15169 does :( I do see that https://www.as25369.net/ lookingglass sees the /23 path:

185.49.142.0/23    *[BGP/170] 19w0d 08:07:57, MED 0, localpref 100, from 217.146.92.34
                      AS path: 3257 20473 211321 I, validation-state: unverified

I don't know what peering/transit AS36492 has.. 'not all google is google' :( Google also operates at least 2 routers, for redundancy reasons, of course. :) so I'm sure there are more than 1 router doing rpki things around here.

AS15169 does not set routing policy for these downstream ASN (AS36385 && AS36492), in the cases where these may use AS15169 for transit, though, the AS15169 path to the /23 would still be usable.

As to the cloudflare prefixes, I see this in the RPKI system today:

• { "asn": "AS0", "prefix": "103.21.244.0/23", "maxLength": 23, "ta": "apnic" }
• { "asn": "AS0", "prefix": "2606:4700:7000::/48", "maxLength": 48, "ta": "arin" }

I'd point back to my previous comment though :(

Okay, let's focus on the IPv6 address 2606:4700:7000::1, announced by AS13335 in 2606:4700:7000::/48.

Google AS15169 sadly does not have a public looking glass, are you able to check ROA status from your end?

Checking 2606:4700:7000::1 via the looking glass https://www.as25369.net/ shows that it's being discarded in London - Telehouse North 2.

My ISP also discards it.

Google's AS36492 does not discard it.

While AS36492 is an ASN that Google/Alphabet uses, it's not an ASN that has the same routing policies (nor operations) as AS15169.

For that v6 prefix the rpki data I see shows:

{ "asn": "AS0", "prefix": "2606:4700:7000::/48", "maxLength": 48, "ta": "arin" }

I believe our(15169) rpki-ov process is still rolling along to conclusion. I don't have a great answer for 'when is it done?' because as I noted previously we've run into problems along the way and expect to see more as we progress.

As to the other associated ASN (36492 for instance), I think we're (15169 folsk) pointing out how the process works and that it has benefit to the customers of these ASN, but we can't set the policy.

203676739

https://issuetracker.google.com/issues?q=id%3A%28203676739%7C309581930%7C40727384%7C385170288%29