If you violate the SPF record with "includes" and go over the magical 10 "includes" you will notice that some bounce with errors and others will end up in spam folders - these symptoms are a sign that there is something wrong with your SPF record.
After digging into the logs and running some tests, I discovered our SPF record was at capacity, years of adding new email services, IP addresses, and third-party integrations had turned our SPF record into an unmanageable monster that was on the brink of violating DNS lookup limits.
The Problem: When SPF Records Get Too Big
SPF (Sender Policy Framework) records have strict limitations that can bite you as your email infrastructure grows:
- 10 DNS lookup limit: Each
include:anda:mechanism counts as a DNS lookup - 255 character limit per TXT record string
- 10 void lookup limit: Failed DNS lookups also count against you
My original SPF record looked something like this monster:
v=spf1 ip4:203.45.78.92 ip4:198.51.100.47 ip4:172.16.254.1/24 ip4:10.0.0.0/8
ip4:192.168.1.0/24 ip4:203.0.113.45 ip4:198.51.100.0/24 ip4:203.45.78.0/24
ip4:172.20.10.5 ip4:192.0.2.100 ip4:198.51.100.200 ip4:203.45.78.150
include:_spf.google.com include:spf.protection.outlook.com
include:spf.mailchimp.com include:amazonses.com include:spf.sendinblue.com
include:_spf.salesforce.com include:spf.constantcontact.com
a:mail.bythepowerofgreyskull.com.com ~all
This is the fun part that caught me off guard: even though I only had 8 include: statements, I was still hitting the DNS lookup limit.
Note: Some of those includes (like _spf.google.com and spf.protection.outlook.com) contain their own includes internally, and each of those counts toward your 10-lookup limit too! So what looks like 8 lookups on the surface was actually triggering 12-15 DNS lookups behind the scenes.
Does that error show in email headers?
Yes, absolutely as an example from the message headers it would look something like this:
Received-SPF: permerror (google.com: too many DNS lookups)
client-ip=74.125.176.26;
envelope-from=lee@bythepowerofgreyskull.com;
helo=mail-server.bythepowerofgreyskull.com;
Authentication-Results: mx.google.com;
spf=permerror (google.com: too many DNS lookups) smtp.mailfrom=lee@bythepowerofgreyskull.com;
dkim=pass (signature was verified) header.i=@bythepowerofgreyskull.com;
dmarc=fail (p=quarantine dis=none) header.from=bythepowerofgreyskull.com
X-Spam-Score: 5.2
X-Spam-Level: *****
X-Spam-Status: Yes, score=5.2 required=5.0 tests=SPF_PERMERROR,DMARC_FAILIf you were looking on the server logs, you will probably see something more like this:
spf=permerror (bythepowerofgreyskull.com: error in processing during lookup of bythepowerofgreyskull.com: DNS timeout)
spf=permerror (bythepowerofgreyskull.com: too many void lookups)
spf=permerror (bythepowerofgreyskull.com: error in processing during lookup)The Hidden DNS Lookup Problem
Here's something that trips up a lot of people: you don't need 10 visible include: statements to hit the lookup limit. Large email providers often have complex SPF records with nested includes.
For example:
_spf.google.comitself contains multiple includes internallyspf.protection.outlook.comhas its own chain of DNS lookups- Each of these nested lookups counts toward your 10-lookup limit
So even if your SPF record only shows 6-7 includes, you might actually be triggering 12+ DNS lookups behind the scenes. This was exactly my problem – what looked manageable on the surface was causing validation failures due to hidden complexity in the third-party SPF records.
SPF Flattening
This involves converting include: statements to ip4: addresses, which don't count against the DNS lookup limit.
For example, instead of this include uses 5 DNS lookups:
include:_spf.google.com
I flattened it to:
ip4:209.85.128.0/17 ip4:64.233.160.0/19 ip4:66.102.0.0/20 ip4:66.249.80.0/20
ip4:72.14.192.0/18 ip4:74.125.0.0/16
This converts from 5 DNS lookups to 0 DNS lookups - a massive improvement, but there is consequences to this process as the dynamic updates are no longer dynamic meaning if new entries are added or removed this will not be updated dynamically.
The SPF Flattening Trade-off
Benefits:
- Dramatically reduces DNS lookup count to zero for flattened includes
- Can fit more services within the 10-lookup limit
- Faster SPF evaluation (no additional DNS queries needed)
Major Downside:
- Static snapshots : If Google adds new IP ranges for their mail servers, my SPF record won't automatically include them
- Maintenance burden: I need to manually monitor and update IP ranges
- Potential delivery failures: Emails from new IPs would fail SPF until I update my record
This is where automation becomes critical to being informed about what has changed and when.
Automating SPF Flattening Maintenance
Since manually monitoring IP changes isn't realistic, I created a PowerShell script that automatically checks for changes and alerts me via email this is the process flow:
Process Flow
- Lookup my current SPF record: Using
nslookup -q=txt bythepowerofgreyskull.com - Check each flattened include: Using
nslookup -q=txt _spf.google.comfor each service I've flattened - Compare IPs: Find any new IP addresses in the includes that aren't in my flattened SPF record
- Send email alerts: When changes are detected, I get a clean email showing exactly what's changed
Example email Alert
When discrepancies are detected you will receive an alert like this:
Script : spf-include-checker.ps1
This is the full script for the monitoring solution that includes the e-mail notification as well, remeber to update the items in bold.
# SPF Flattening Monitor Script - Dynamic Lookup
# Monitors upstream SPF includes for IP changes by looking up live DNS records
# =============================================================================
# CONFIGURATION VARIABLES - MODIFY THESE FOR YOUR ENVIRONMENT
# =============================================================================
# Email configuration
$SmtpServer = "smtp.bear.local"
$SmtpPort = 25
$FromEmail = "spf-monitor@bythepowerofgreyskull.com"
$ToEmail = "lee@bythepowerofgreyskull.com"
$Subject = "SPF Record Change Alert"
# Domain to monitor
$YourDomain = "bythepowerofgreyskull.com"
# List of include domains to monitor
$IncludesToMonitor = @(
"_spf.google.com"
)
# =============================================================================
# FUNCTIONS
# =============================================================================
function Get-SPFRecordViaNslookup {
param([string]$Domain)
try {
Write-Host "Running: nslookup -q=txt $Domain" -ForegroundColor Gray
$nslookupResult = & nslookup -q=txt $Domain 2>$null
if ($LASTEXITCODE -ne 0) {
Write-Warning "nslookup failed for $Domain"
return $null
}
# Parse nslookup output to find SPF record
$spfRecord = ""
$inRecord = $false
foreach ($line in $nslookupResult) {
$line = $line.Trim()
# Look for lines containing v=spf1
if ($line -like '*"v=spf1*' -or $line -like "*v=spf1*") {
$spfRecord = $line
$inRecord = $true
}
# Handle multi-line SPF records
elseif ($inRecord -and $line -like '*"*') {
$spfRecord += " " + $line
}
# Stop if we hit a new record or empty line
elseif ($inRecord -and ($line -eq "" -or $line -like "*text =*")) {
break
}
}
if ($spfRecord) {
# Clean up the record - remove quotes and extra whitespace
$spfRecord = $spfRecord -replace '"', '' -replace '\s+', ' '
$spfRecord = $spfRecord.Trim()
# Ensure it starts with v=spf1
if ($spfRecord -like "v=spf1*") {
return $spfRecord
}
}
return $null
}
catch {
Write-Warning "Failed to resolve SPF record for $Domain : $_"
return $null
}
}
function Get-IncludesFromSPF {
param([string]$SPFRecord)
$includes = @()
if ($SPFRecord) {
# Extract include: statements
$includeMatches = [regex]::Matches($SPFRecord, "include:([^\s]+)")
foreach ($match in $includeMatches) {
$includes += $match.Groups[1].Value
}
}
return $includes
}
function Get-IP4AddressesFromSPF {
param([string]$SPFRecord)
$ipAddresses = @()
if ($SPFRecord) {
# Extract ip4: addresses
$ip4Matches = [regex]::Matches($SPFRecord, "ip4:([^\s]+)")
foreach ($match in $ip4Matches) {
$ipAddresses += $match.Groups[1].Value
}
}
return $ipAddresses | Sort-Object
}
function Get-AllIPsFromIncludeDomain {
param([string]$IncludeDomain)
$allIPs = @()
$spfRecord = Get-SPFRecordViaNslookup -Domain $IncludeDomain
if ($spfRecord) {
Write-Host " SPF record for $IncludeDomain : $spfRecord" -ForegroundColor Gray
# Get direct IP addresses
$directIPs = Get-IP4AddressesFromSPF -SPFRecord $spfRecord
$allIPs += $directIPs
# Recursively process nested includes (be careful of the 10 lookup limit)
$nestedIncludes = Get-IncludesFromSPF -SPFRecord $spfRecord
foreach ($nestedInclude in $nestedIncludes) {
Write-Host " Processing nested include: $nestedInclude" -ForegroundColor Gray
$nestedIPs = Get-AllIPsFromIncludeDomain -IncludeDomain $nestedInclude
$allIPs += $nestedIPs
}
}
else {
Write-Warning "Could not retrieve SPF record for $IncludeDomain"
}
return $allIPs | Sort-Object | Get-Unique
}
function Compare-IncludeWithCurrentSPF {
param(
[string]$IncludeDomain,
[array]$IncludeIPs,
[array]$CurrentSPFIPs
)
# Find IPs that exist in the include but not in our current SPF record
$added = $IncludeIPs | Where-Object { $_ -notin $CurrentSPFIPs }
# Find IPs that exist in our SPF record but are no longer in the include
# This is trickier - we need to identify which IPs in our SPF might belong to this include
# For now, we'll just report what's missing from the include
$removed = @()
# Note: Detecting removed IPs is complex because we don't know which IPs in the current
# SPF record originally came from which include. This would require storing historical mappings.
# For now, we'll focus on detecting new IPs that need to be added.
return @{
Added = $added
Removed = $removed
HasChanges = ($added.Count -gt 0 -or $removed.Count -gt 0)
}
}
function Send-AlertEmail {
param(
[string]$IncludeDomain,
[array]$AddedIPs,
[array]$RemovedIPs,
[array]$CurrentIncludeIPs
)
# Create HTML email with clean, card-based design
$htmlBody = @"
<!DOCTYPE html>
<html>
<head>
<style>
body { font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif; margin: 0; padding: 20px; background-color: #f5f5f5; }
.container { max-width: 600px; margin: 0 auto; }
.header { background-color: #ffffff; padding: 20px; border-radius: 8px 8px 0 0; border-left: 4px solid #0078d4; }
.header h1 { margin: 0; color: #323130; font-size: 24px; font-weight: 600; }
.header p { margin: 5px 0 0 0; color: #605e5c; font-size: 14px; }
.card { background-color: #ffffff; margin: 1px 0; padding: 20px; border-radius: 0; }
.card:last-child { border-radius: 0 0 8px 8px; }
.card h2 { margin: 0 0 15px 0; color: #323130; font-size: 18px; font-weight: 600; }
.alert-added { border-left: 4px solid #107c10; }
.alert-removed { border-left: 4px solid #d13438; }
.alert-info { border-left: 4px solid #0078d4; }
.ip-list { background-color: #f8f9fa; padding: 15px; border-radius: 4px; margin: 10px 0; }
.ip-item { font-family: 'Courier New', monospace; margin: 5px 0; color: #323130; }
.ip-added { color: #107c10; font-weight: 600; }
.ip-removed { color: #d13438; font-weight: 600; }
.footer { background-color: #f8f9fa; padding: 15px 20px; border-radius: 0 0 8px 8px; color: #605e5c; font-size: 12px; }
.metadata { display: flex; justify-content: space-between; margin: 10px 0; }
.metadata span { color: #605e5c; font-size: 14px; }
</style>
</head>
<body>
<div class="container">
<div class="header">
<h1>SPF Record Change Detected</h1>
<div class="metadata">
<span><strong>Domain:</strong> $YourDomain</span>
<span><strong>Time:</strong> $(Get-Date -Format "yyyy-MM-dd HH:mm:ss")</span>
</div>
<p>Changes detected for include: <strong>$IncludeDomain</strong></p>
</div>
"@
if ($AddedIPs.Count -gt 0) {
$htmlBody += @"
<div class="card alert-added">
<h2>New IP Addresses Detected</h2>
<p>The following IP addresses are now present in <strong>$IncludeDomain</strong> but are NOT in your current SPF record:</p>
<div class="ip-list">
"@
foreach ($ip in $AddedIPs) {
$htmlBody += " <div class='ip-item ip-added'>+ ip4:$ip</div>`n"
}
$htmlBody += @"
</div>
<p><strong>Action:</strong> Consider adding these IP addresses to your SPF record if this include was previously flattened.</p>
</div>
"@
}
# Show all current IPs from the include for reference
if ($CurrentIncludeIPs.Count -gt 0) {
$htmlBody += @"
<div class="card alert-info">
<h2>Current IPs in $IncludeDomain</h2>
<p>All IP addresses currently returned by <strong>$IncludeDomain</strong>:</p>
<div class="ip-list">
"@
foreach ($ip in $CurrentIncludeIPs) {
$htmlBody += " <div class='ip-item'>ip4:$ip</div>`n"
}
$htmlBody += @"
</div>
</div>
"@
}
$currentSPF = Get-SPFRecordViaNslookup -Domain $YourDomain
$htmlBody += @"
<div class="card alert-info">
<h2>Your Current SPF Record</h2>
<div class="ip-list">
<div class="ip-item">$currentSPF</div>
</div>
</div>
<div class="footer">
<strong>Action Required:</strong> Review the changes and update your SPF record if necessary to maintain email deliverability.<br>
Generated by SPF Flattening Monitor Script
</div>
</div>
</body>
</html>
"@
try {
Send-MailMessage -SmtpServer $SmtpServer -Port $SmtpPort -From $FromEmail -To $ToEmail -Subject "$Subject - $IncludeDomain" -Body $htmlBody -BodyAsHtml -ErrorAction Stop
Write-Host "Alert email sent successfully for $IncludeDomain" -ForegroundColor Green
}
catch {
Write-Error "Failed to send email alert: $_"
}
}
# =============================================================================
# MAIN EXECUTION
# =============================================================================
Write-Host "SPF Flattening Monitor Started - $(Get-Date)" -ForegroundColor Cyan
Write-Host "Monitoring domain: $YourDomain" -ForegroundColor Cyan
# Get current SPF record for your domain using nslookup
$currentSPFRecord = Get-SPFRecordViaNslookup -Domain $YourDomain
if (-not $currentSPFRecord) {
Write-Error "Could not retrieve SPF record for $YourDomain"
exit 1
}
Write-Host "`nCurrent SPF Record: $currentSPFRecord" -ForegroundColor Yellow
# Extract current IP4 addresses from your SPF record
$currentSPFIPs = Get-IP4AddressesFromSPF -SPFRecord $currentSPFRecord
Write-Host "`nCurrent ip4 addresses in your SPF record:" -ForegroundColor Yellow
$currentSPFIPs | ForEach-Object { Write-Host " $_" -ForegroundColor Yellow }
# Check each include domain specified in the monitor list
foreach ($includeDomain in $IncludesToMonitor) {
Write-Host "`n" + "="*60 -ForegroundColor Cyan
Write-Host "Checking include: $includeDomain" -ForegroundColor Cyan
Write-Host "="*60 -ForegroundColor Cyan
# Get current IPs from the include domain using nslookup
$includeIPs = Get-AllIPsFromIncludeDomain -IncludeDomain $includeDomain
if ($includeIPs.Count -eq 0) {
Write-Warning "No IP addresses found for $includeDomain"
continue
}
Write-Host "`nAll IPs currently in $includeDomain :" -ForegroundColor Green
$includeIPs | ForEach-Object { Write-Host " $_" -ForegroundColor Green }
# Compare the include's current IPs with what's in your SPF record
$comparison = Compare-IncludeWithCurrentSPF -IncludeDomain $includeDomain -IncludeIPs $includeIPs -CurrentSPFIPs $currentSPFIPs
if ($comparison.HasChanges) {
Write-Host "`nCHANGES DETECTED for $includeDomain !" -ForegroundColor Red
if ($comparison.Added.Count -gt 0) {
Write-Host "New IPs in include (not in your SPF):" -ForegroundColor Red
$comparison.Added | ForEach-Object { Write-Host " + $_" -ForegroundColor Red }
}
# Send alert email
Send-AlertEmail -IncludeDomain $includeDomain -AddedIPs $comparison.Added -RemovedIPs $comparison.Removed -CurrentIncludeIPs $includeIPs
}
else {
Write-Host "`nNo new IPs detected for $includeDomain" -ForegroundColor Green
Write-Host "All IPs from this include appear to be covered in your current SPF record." -ForegroundColor Green
}
}
if ($IncludesToMonitor.Count -eq 0) {
Write-Host "`nNo include domains specified to monitor." -ForegroundColor Yellow
Write-Host "Please add domains to the `$IncludesToMonitor variable." -ForegroundColor Yellow
}
Write-Host "`n" + "="*60 -ForegroundColor Cyan
Write-Host "SPF Flattening Monitor Completed - $(Get-Date)" -ForegroundColor Cyan
Write-Host "="*60 -ForegroundColor Cyan
I schedule this to run daily, and it catches any IP changes before they affect email delivery.
The Results
Email delivery improved immediately. No more SPF validation errors, no more mysterious bounces, and I have a much cleaner setup that's easier to manage. The monitoring script gives me peace of mind - I know I'll be alerted if any of the flattened services add new IP ranges.
Key Takeaways
- The 10 DNS lookup limit applies to the entire SPF evaluation - not per subdomain or include
- IP4 addresses don't count against the lookup limit, making flattening highly effective
- Automation is essential - manual monitoring of IP changes isn't sustainable
- SPF flattening is more effective than complex include structures for most scenarios
- Monitor what you flatten - services change IPs, and you need to know about it (hence the script)
Conclusion
Don't let an oversized SPF record break your email delivery. SPF flattening, combined with automated monitoring, provides a robust solution that's been part of the SPF standard for nearly two decades. The key is converting those DNS-lookup-heavy includes into static IP addresses while maintaining automated oversight of changes.