Inconsistent responses for a CNAME query (sometimes an answer, sometimes the SOA record) [183705439]

Customer Issue

Status Update

No update yet.

Description

ro...@envato.com

created issue #1

Mar 26, 2021 01:01AM

We have recently seen inconsistent results when querying 8.8.8.8 for a handful of DNS names. Repeated queries for the same record will sometimes return the expected answer, an identical query moments later will return just the domain's SOA record.

Examples:

Query with answer:


; <<>> DiG 9.16.12 <<>> -tcname elements-assets.envato.com @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63955
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;elements-assets.envato.com.	IN	CNAME

;; ANSWER SECTION:
elements-assets.envato.com. 299	IN	CNAME	d1lrs95zs5ox0o.cloudfront.net.

;; Query time: 39 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Mar 26 13:54:56 NZDT 2021
;; MSG SIZE  rcvd: 98

Query with only SOA response:

$ dig -tcname elements-assets.envato.com @8.8.8.8

; <<>> DiG 9.16.12 <<>> -tcname elements-assets.envato.com @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7576
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;elements-assets.envato.com.	IN	CNAME

;; AUTHORITY SECTION:
envato.com.		5	IN	SOA	dell.ns.cloudflare.com. dns.cloudflare.com. 2036832436 10000 2400 604800 3600

;; Query time: 33 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Mar 26 13:54:56 NZDT 2021
;; MSG SIZE  rcvd: 114

Testing today indicates that about 70% of queries return the answer, while 30% do not. Neither the domain nor record have had recent changes.

I've tested against other public DNS services (Cloudflare and OpenDNS), which return the answer for 100% of queries. I expect the occasional failure or error, but 30% seems excessive.

Any suggestions on further troubleshooting steps, or ideas on what could be the cause would be appreciated.

Cheers! Ross

I affirm that I have looked at all the other public issue templates at https://issuetracker.google.com/components/191657/manage#templates and restricted issue templates at https://issuetracker.google.com/components/191885/manage#templates and none of them apply.

Comments

ro...@envato.com <ro...@envato.com> #2Mar 28, 2021 11:27PM

Automated by Blunderbuss job honestdns_tickets_blunderbuss_autoassigner for 2nd config honestdns_defects_customer_issues for component 191657.

al...@google.com <al...@google.com> #3Mar 29, 2021 03:30PM

Hello, We are reviewing your request and need to verify it is from the organization which owns this address block. I have reached out to the owner of the address block, e.zuniga@libertycr.com, for verification.

Alternatively, if you can reply or send me an email from j.medrano@libertycr.com or ja.benavides@libertycr.com. This would allow us to move to the next steps.

Thanks!

ro...@envato.com <ro...@envato.com> #4Mar 29, 2021 09:26PM

Hello

We have already answered the email with the mail j.medrano@libertycr.com

Thank you for contacting us.

Let's continue the process from the other email account.

Regards

ro...@envato.com <ro...@envato.com> #5Mar 29, 2021 09:28PM

Yes, I heard back from j.medrano@libertycr.com confirming this request.

We've moved forward with this request and the changes (increasing client QPS to 5,000) will roll out next week.

al...@google.com <al...@google.com> #6Mar 30, 2021 07:41PM

Hi
Dear Google team.

Thank you very much for the update.

We will keep an eye on the increase over the next week to monitor our monitoring graphs.

Regards.

ro...@envato.com <ro...@envato.com> #7Apr 1, 2021 08:09AM

I confirm what you're seeing, requesting an A record always returns an answer for me, while specifically requesting the CNAME intermittently returns no answer.

We don't have a direct customer-facing use case for the CNAME query, no. It was raised because we have a CI job that validates that all our DNS records are published and distributed correctly, which started failing on (mostly) just these two records last week. We will look at potentially rewriting these tests to fall back to checking the A record. Our end users are unlikely to be affected by this problem, given that web browsers will not be looking up the CNAME specifically but the A record directly (confirmed with a packet capture).

Given that it no longer appears to be customer impacting, it's a lower priority for us now. We'll focus on more accurate tests (though it'd be interesting to understand the cause of the CNAME resolution issue).

Thanks for your time!

al...@google.com <al...@google.com> #8Apr 1, 2021 01:24PM

Thanks for your response. I agree that this is surprising and incorrect, but low priority due to the low likelihood of end user impact. I've created an internal issue to track this and should be able to update you here once it's resolved.