Food Fight Over Encrypting DNS

Oh great. ANOTHER technical Food Fight in progress.

I’d noticed this in my usual “turn off all the insecure crap” first boot of browsers on a new (Buster?) install on one of my SBCs (Single Board Computers). I think the SBC in question was the RockPro64, maybe.

Wandering through all the settings and turning off camera, microphone, auto-run sound & video, location… I discovered one setting was to do DNS over HPPTS, or not. WT? I want to use MY carefully made and ad blocking DNS servers (that also reduce DNS traffic out of my site and hide particular machine identity from external DNS providers…)

So I turned it off, checked that I still had blocking working, and moved on.

Then I ran into an issue on the Odroid N2 running plain Armbian where ad blocking wasn’t working (and found out it was very hard to get it to accept a change of DNS server…). Eventually enough crowbar got it to work ;-)

Along the way found that OpenDNS was bought by CISCO (who I’m pretty sure was happy with the Prism Program and likes working with the TLAs of the world… so I’m not feeling all secure about that DNS info). This caused me to think maybe it was time to run my internal / house DNS servers over a VPN (to change geolocation information) and via a TLS encrypted pipe (to keep DNS lookups private) to some trustworthy upstream.

Looking into this, I found out just why the browser was suddenly wanting to steal my DNS lookups. “It’s complicated” and comes down to a Food Fight between Privacy and Security and over “Who has control” and “Who gets the information about you to sell”.

The short form is that if you put HTTPS DNS into the browser, it bypasses and site local DNS servers (so local network managers lose visibility and control) and sends the information to an “authoritative” DNS provider who is beyond local control and beyond Telco visibility (so that one site gets all the lucrative information about what DNS lookups are happening AND gets all that tactical information too…). It also lets your browser bypass all the local filtering by DNS for things like blocking porn sites ( I’m sure that was not a consideration… /sarc;) and other forbidden stuff.

Now Ideal for me would be allowing me to choose. Put HTTPS DNS in the browser, but let me turn it on / off AND let me chose what DNS Server it points at. Let me as Site Admin put my DNS on TLS secure pipes if I so choose. Also, let a local admin lock OUT browser DNS IFF the company so chooses (otherwise the network admin has no control and all manner of malware blocking fails).

Here’s the IEEE article about it:

https://spectrum.ieee.org/tech-talk/telecom/security/the-fight-over-encrypted-dns-boils-over

From Nov. 2019, so I”m only a year late getting to this. Likely as things get to the ARM ports about a year after they hit the Intel chip ports… IMHO.

27 Nov 2019 | 15:52 GMT
The Fight Over Encrypted DNS: Explained
DNS over TLS and DNS over HTTPS both do what they are designed to do. So what’s all the fighting about?
[…]
Pros and Cons
For the privacy-minded, DNS over TLS isn’t good enough because anyone monitoring the network will know that any activity on Port 853 must be DNS-related. While an observer won’t know the actual contents of the query because both the response and request are encrypted, the fact that anyone could know that queries are being made is enough to raise warnings flags for some. While secure, DNS over TLS isn’t as privacy-friendly as DNS over HTTPS.

Another downside of DNS over TLS—as far as opponents are concerned—is that it requires software developers and device makers to make changes so that their applications and hardware support the protocol. The resolver may have the certificates to handle the encryption, but if the program or device can’t (or won’t) establish the connection, DNS over TLS isn’t protecting the user.

DNS over HTTPS is more democratic, as anyone using a supported web browser automatically gets encrypted DNS. DNS over HTTPS stops all third parties—bad actors, Internet service providers, government agencies, law enforcement, and network operators (including corporate IT staff)—from seeing anything about what sites viewers are browsing. That’s exactly what privacy advocates want, but it’s the opposite of what network administrators and security teams need.

Privacy vs Security
DNS over HTTPS treats privacy as absolute—but parental control applications, antivirus and security software, enterprise firewalls, and other networking tools don’t share that ethos. Having DNS over HTTPS turned on by default in the web browser means all DNS queries are relayed to the designated DNS server, which may not be the organization’s own DNS server, or even the ISP’s.

Mozilla has announced that DNS over HTTPS will be the default for Firefox users in the United States, and that change is currently being rolled out. Firefox will automatically relay all DNS traffic to Cloudflare’s 1.1.1.1 service and ignore the user’s existing DNS settings. That bypasses all associated network-based filtering rules, such as those that block malware from communicating with command-and-control servers, or that stop users from accessing malicious or illegal sites.

Google also turned on DNS over HTTPS for Chrome users, but that situation is slightly different as the browser defaults to DNS over HTTPS only if the user has a DNS over HTTPS-compatible service.

Microsoft is trying to have it both ways, as it plans to support DNS over HTTPS in Windows, but will allow Windows administrators to maintain some control.

DNS is a “reasonable place to restrict access” to bad entities, Akamai’s April said. Network operators block hostnames used by Wannacry and other malware or redirect users who try to access malicious or banned sites. Operators of public Wi-Fi networks modify DNS queries to first load a network sign-on page for new users. DNS over HTTPS breaks these use cases.

This is partly why Mozilla is not turning on DNS over HTTPS for Firefox users in the United Kingdom, as UK law requires ISPs to block access to illegal websites, such as those related to child pornography. Losing visibility over the network is dangerous, April said.

DNS over HTTPS versus DNS over TLS is also a battle over the user’s web browsing data and who gets to access it. DNS queries from Firefox will go to Cloudflare, which means Cloudflare is going to be sitting on top of a lot of DNS data. The tech companies that operate centralized DNS servers—such as Cloudflare and Google—will ultimately benefit with DNS over HTTPS, as they will be the ones with more visibility into what people are browsing, Vixie said.

Privacy advocates believe that users should be in charge of their web browsing, not ISPs. But Mozilla’s decision forces Firefox users to use Cloudflare regardless of their own preferences.

Most users won’t notice any difference when encrypted DNS becomes the default—which has already happened for Chrome users. For enterprises and ISPs, the socially-conscious, user-oriented approach forces a tradeoff: the price for increased privacy is reduced security.

The reality of the modern Internet is that ISPs and enterprises play a role in keeping threats away from users. Ideally, web browsers should let users choose whether or not to use DNS over TLS or DNS over HTTPS, and give users the option to control which DNS provider to use.

Oh Joy. Yet Another Security Breaking move that we all must adapt to. But where most folks will be blissfully unaware and uncaring.

So, OK, I can see this is going to be an issue. I expect that the Chromium Open Source version of Chrome will have more optional control than the official Google Chrome, but when? How hard?

So I can turn it off in FireFox (done). And I may have to do something else with new Chromium (whenever it gets to my old SBCs in an eventual software update…). While I don’t really want to get into the business of hacking Chromium, that it IS open source means I can if need be. Hopefully others more engaged will do that hacking before I get motivated enough ;-)

Until then:

Just Be Aware that YOUR browser may well be bypassing any DNS settings you have made in your machine, or your site, or your corporate standards; and any made by your Telco if you depend on their defaults.

Welcome to the Browser DNS Wars…

Sigh.

Why is it so hard to just get folks to stop screwing around with stuff that works?

Subscribe to feed

About E.M.Smith

A technical managerial sort interested in things from Stonehenge to computer science. My present "hot buttons' are the mythology of Climate Change and ancient metrology; but things change...
This entry was posted in Security & Privacy, Tech Bits. Bookmark the permalink.

16 Responses to Food Fight Over Encrypting DNS

  1. E.M.Smith says:

    Oh Dear!

    Another gem from the Lords Of Unfortunate Acronyms! Just like STD – Star Trek Discovery or S Transmitted Disease… we get. for DNS over HTTPS:

    DOH!

    (Shades of Homer of The Simpsons…)

    https://www.chromium.org/developers/dns-over-https

    DNS over HTTPS (aka DoH)

    I mean, really… Just “Oh DOH! Why?…”
    ;-)

    We’ve enabled an experiment in Chrome 83 for a fraction of our users with the following scope.

    Platforms: Windows, Mac, Chrome OS. (Support for Android and Linux is planned for after M83)
    Default experience: same-provider auto-upgrade (i.e. the behavior outlined above and tested in Chrome 79).
    Setting in the “Privacy and Security” section to allow users to control and configure the feature in the rare cases where the default behavior wouldn’t meet their needs. This will include the ability to completely disable DoH, as well as manual configuration options. The setting will default to an automatic mode which corresponds to the “same provider, auto-upgrade” behavior we’ve been testing in the Chrome 79 experiment.
    For users who fall into any of the following conditions, the setting will be locked down and DoH related command line options (if any) will be ignored:
    If OS parental controls are detected -> locked in “off” (subject to the availability of APIs provided by the OS).
    DoH set via enterprise policies -> locked with a state reflecting the enterprise policies
    Detected managed environment (but no DoH enterprise policies set) -> locked in “off”

    So, OK, check if Chromium is MORE THAN 83 or more. Then you can work on that whole “Enterprise Policies” stuff. Provided the Open Source Chromium gets the same options of control.

  2. E.M.Smith says:

    Well, it’s supposedly in AFTER Armbian Buster:

    Version 83.0.4103.116 (Developer Build) built on Debian 10.4, running on Debian 10.6 (64-bit)

    But I don’t find a control under the Security & Privacy where they say it will be. I suspect the developer just removed the behaviour as I’m running it now and it is using my ad blocking DNS.

    Maybe next release…

    UPDATE:

    Just sunk into my pea brain. It is AFTER 83.x.x..x not after 83.0.0.0 So not in this 83.x.x.x release ’cause we’re not at 84.x.x.x yet…

  3. andysaurus says:

    I use Chrome on Windows 10. I realise I’ve sold my soul to Google. I reckoned it was a toss up between them and Apple unless I wanted to get into the weeds like you do. I have just started getting this message on some sites (Such as the Dilbert Cartoon):
    This page isn’t working
    dsldevice.lan redirected you too many times.
    Try clearing your cookies.
    ERR_TOO_MANY_REDIRECTS
    Do you think it’s related?

  4. andysaurus says:

    I am pretty sure it is related. I get that error message if I follow the link from an Email as it is embedded in an Email that I read in Outlook (My Email server of choice that does not actually display the cartoon). If I follow the same link from the same message displayed in my Gmail account, then it works!

  5. Taz says:

    Yeah, I’m sick of it too. Worse, with this alphabet soup of standards – no one seems to be minding the store. I get a lot DNS issues these days :( especially annoying since I’m almost always on a VPN (who should have staff to take care of these things).

    Something to look at? IPFire. Supports DNSMASQ and DNS over https “out of the box”. Unfortunately, if you hate Cloudflare (as I do) your only good DNS source will likely be a non profit in Germany or Luxembourg. There is a new paid DNS service (NextDNS) which might also be worthwhile.

    Warning. *IF* you activate IPFire’s built in Tor relay, some of those Roku services will block you since some idiot updates all relay IP addresses to GitHUB weekly. Doesn’t matter if you’re just running relays – they will still block you.

    The fix? Either settle for running an I2P relay OR subscribe to a smart DNS service with a real proxy (not all are same). Then buy a GL-Inet open source mini router (many models GL-MT300N is common) for each Roku, carefully screening out Google’s DNS and fixing the DNS source to your SmartDNS service. Long ago, you could do such blocking from your main router – but not any more.

  6. E.M.Smith says:

    Odd… dsl is a WAN connectivity now. Putting dsl and lan together is “odd”.
    Looks like it relates to a particular kind of router:
    https://community.plus.net/t5/ADSL-Broadband/dsldevice-lan-EDITED-to-add-Q-about-firewall-error-message/td-p/971659

    So OK… your router is saying it’s not happy with a tendency for some site you are trying to reach to be “redirected” too much.

    There’s 2 most likely cases:

    1) It’s a foreign hacker site trying to poach your traffic and getting caught from trying to do the redirect too much.

    2) The thing you are trying to reach is down and you are being bounced around trying to find an “up” version.

    Neither one is good, but #1 is really bad.

    Could it be related?

    1) Are you trying to reach the sites using HTTPS ? If no, then no. If yes, then maybe. Look for that setting in “Security & Privacy” that the above link about Chromium says lets you turn it off. Turn it off and see if anything changes.

    2) If you are using HTTPS-DNS, then it could be that the resolver chosen by your browser is having issues. Unfortunately, you can’t see it, fix it, diagnose it, etc. Most likely you don’t even get to know what resolver is being used. (Though for Chrome, being Google, I’d expect it to be their resovlers like 4.4.4.4 or 8.8.8.8 ) Given the high use of the Google resolvers, I doubt they are screwed up (but might have traffic issues).

    My best guess would be that some hacker is trying to pwn your router and failing, but occasionally screws up the performance. Second best guess would be that it IS something about the HTTPS DNS (and you ought to be able to test that as noted above). Third up is that the DNS attack is happening to your regular Telco resolver and you are not using HTTPS DNS.

    OK, that’s what I came up with “blind”. Now lets test me. Doing a web search on the error message pulls up an interesting web site:

    https://www.keycdn.com/support/err_too_many_redirects

    However, sometimes those cookies can contain faulty data, and this can lead to the redirect error. When you clear your browser cache, it gets rid of those cookies and can often fix the redirect loop you’re getting stuck in.

    implies it is more about incorrectly configured web sites and / or broken cookies. You can try dumping cookies to see if that helps. ( I hate that advice due to several sites I use storing my authentication in a cookie and dumping cookies has me need to do the 2 factor authentication again – thing sent to cell phone and keyed in…) So if you don’t have any cookies you care about, try dumping them. Also test that particular site / URL from some other computer / device. That will tell you if it is you, or them.

    Could that be related? Perhaps. IF the cookie has one resolution and using HTTP DNS for some reason resolved to a different place, it might be confused…

    They also talk about HTTPS issues:

    A common cause of a redirect loop is a problem with HTTPS settings. This is especially true if you’ve recently transitioned your website from HTTP. Often, users forget to finish something in the process, or they make a simple mistake during setup.

    You should also be sure that you have an SSL certificate installed on your website before you complete the HTTPS process. Without an up to date SSL certificate, forcing your page to load over HTTPS will automatically give you a redirect loop error.

    So if you are suddenly doing HTTPS and the destination has a cert that doesn’t match, you get this error. Check the DATE on your computer. Certs expire at a given date and time. I often get grief when I bring up a chip / image that has been idle for a while and find the clock set to 2001 or something and EVERY cert is “not yet valid” as they have a start date too…

    So there’s my best guesses on how to approach debugging this. Good luck.

    FWIW, I’d probably check the date first (easy and necessary first step), then dump cookies, then look for more exotic stuff.

  7. E.M.Smith says:

    Oh, and note that folks have been shifting to ever newer / tighter settings for encryptions that are allowed. I also ran into problems when a site was still using a weaker encryption and my OS decided it was not going to accept that old weak encryption anymore. IF this is a very new OS install AND the web site target is running an older encryption that’s discouraged, you can get a failed connection.

  8. E.M.Smith says:

    @Andysaurus:

    OK, you have an added data point. Since it works in one place, it isn’t the site. Since it works in one software, it’s something about the other software.

    You are talking email though, not browser… Does your email launch a browser to look at the cartoon? (Sorry, it’s been a decade+ since I ran Windows more than just a boot up / shut down on my old Windows 7 to see if it still works ;-)

    @Taz:

    It’s rude, but one could likely point at one of the dozen or so root servers with their own private DNS server.

    I could easily see a “consortium” of Linux Users gang together and have one guy be Tier 2, use the root server, and then fan out to all the rest.

    Nothing wrong with setting up your own DNS world and just touch the outside at one point ;-) (How most companies work and how I run my home network too, but my outside is a major DNS provider. Until now I often used OpenDNS, but with Cisco buying them I may need a re-think… I have a long list of “The Usual” DNS providers. Started doing that at work decades back; so when trying to bring up “whatever” “whereever” with no infrastructure, I had numbers I could plug in…)

    FWIW on the “relay” issue:

    I’ve been thinking of using the free individual AWS tranche to build my own VPN server. As I’d be the only one using it, I can assure no logging is done. It also ought not show up anywhere as a “relay” as nobody but me knows about it ;-) Just an idea…

  9. E.M.Smith says:

    @Taz:

    Yeah, I got that nag at first fire on a new install of Armbian ( I think it was on the RockPro64) and chose the “opt out” without fully knowing why I was getting the nag.

    Part of what sent me down the road leading to this article…

  10. andysaurus says:

    Thank you for your advice.

  11. E.M.Smith says:

    You know, the more I think about it, with Outlook not working and Gmail working, it just sounds more and more like a difference over encryption level allowed. That could be confirmed by checking what Gmail allows (IIRC, Gmail runs on Google and only displays on your end) vs what your site / system allows as Outlook runs locally.

    I could easily see that being the root cause.

    Does DNS inside the browser tickle that? Not sure how…

  12. E.M.Smith says:

    @Andysaurus:

    You are most welcome. I just hope my scattergun ideas are helpful in some way.
    By definition, at most one is right, so most are wrong…

    The tyranny of debugging. By definition you will be wrong in all your ideas until, if you are lucky and solve it, the last idea may be correct.

  13. andysaurus says:

    Thanks E.M. I was telling the guys at the Men’s Shed the other day, after successfully searching for a tool, that “It’s always the last place you look” is a truism. There is little value in continuing your search after finding that for which you are looking.

    In similar vein, all the reports that “Too much salt/fat/sunshine/ etc. is bad for you” make me laugh. That’s what “too much” means.

  14. andysaurus says:

    E.M. It was my router. It seems that when it boots it selects a DNS server and it had picked what we would call here a wrong ‘un. All I had to do was reboot.

    I have to give a shout out to you and to the tech support of my ISP Internode. I called them and they fixed me up within 10 minutes, which is normal for them. Having been in the industry myself, I know how hard it is to get competent, friendly staff and they always seem to do it. And they speak English.

    One of the things they share with you is that they treat me like a rational person who is really trying his best to give them all the information they need to solve the problem. Most tech support is sneering and supercilious.

    Anyway,

    Case closed.

Comments are closed.