The Problem with Entra MFA Reports
If you download the Entra MFA report, it's a bit of a mess. When you look at the authentication method column, everything a user has registered is listed, but it's not clear which one the user is actually using. I found this report quite unreliable for gathering concrete conclusions about actual authentication method usage.
What's more concrete is to look at the sign-in logs of users in Entra. Under authentication details, you can see the actual authentication method used. Many of these entries will be marked as "previously satisfied", but there will be one login that will have an authentication method, which confirms what the user is actually using rather than what they have registered but aren't actually using.
The Technical Challenge
I had a requirement to figure out who was using SMS authentication. This doesn't include who has a phone registered because SMS is no longer separated - it registers as "phone". I wanted to see the actual data in the sign-in audit logs.
Step 1: Identifying Authentication Method Types
First, I needed to identify my login types and see what they report under the authentication method. For me, it was "Mobile app" which means I'm using password login with the Authenticator app. I needed to ignore all events that were "previously satisfied" because that wasn't helpful.
Here's how the authentication details appear in the sign-in logs:
# Sample sign-in log structure
$signIn = @{
id = "xxxxx-xxxx-xxxx-xxxx"
userPrincipalName = "user@croucher.cloud"
authenticationDetails = @(
@{
authenticationMethod = "Previously satisfied"
succeeded = $true
},
@{
authenticationMethod = "Text message"
authenticationMethodDetail = "+1XXXXXXX23"
succeeded = $true
}
)
}
Step 2: Single User SMS Detection
I took my initial script and modified it to only look for "Text message" in the authentication method. Here's the core logic:
# Connect to Graph API
function Get-GraphToken {
$body = @{
client_id = $clientId
scope = "https://graph.microsoft.com/.default"
client_secret = $clientSecret
grant_type = "client_credentials"
}
$tokenUrl = "https://login.microsoftonline.com/$tenantId/oauth2/v2.0/token"
$resp = Invoke-RestMethod -Method Post -Uri $tokenUrl -Body $body -ContentType "application/x-www-form-urlencoded"
return $resp.access_token
}
# Query for specific user
$filter = "userPrincipalName eq '$($UPN.ToLower())' and createdDateTime ge $startIso"
$uri = "https://graph.microsoft.com/beta/auditLogs/signIns?`$filter=$([uri]::EscapeDataString($filter))&`$top=500"
# Process results
foreach ($signIn in $response.value) {
foreach ($authDetail in $signIn.authenticationDetails) {
if ($authDetail.authenticationMethod -eq "Text message") {
# Found SMS authentication
$results += [PSCustomObject]@{
UserPrincipalName = $UPN
SignInId = $signIn.id
CreatedDateTime = $signIn.createdDateTime
AuthenticationMethod = $authDetail.authenticationMethod
Succeeded = $authDetail.succeeded
}
}
}
}
My user correctly reported "no SMS logins in the last 30 days".
Step 3: Scaling to Whole Tenant with Pagination
Applying this to the whole tenant introduced pagination challenges. The Graph API returns a maximum of 999 results per page, and I had tens of thousands of sign-ins to process.
Here's how I implemented pagination:
# Initial query without user filter
$filter = "createdDateTime ge $startIso"
$uri = "https://graph.microsoft.com/beta/auditLogs/signIns?`$filter=$([uri]::EscapeDataString($filter))&`$top=999"
# Pagination loop
$allSignIns = @()
$nextLink = $uri
$pageNumber = 0
while ($nextLink) {
$pageNumber++
Write-Host ("Fetching page " + $pageNumber + "...") -NoNewline
$response = Invoke-RestMethod -Uri $nextLink -Headers $headers -Method Get
if ($response.value) {
$pageRecordCount = $response.value.Count
$allSignIns += $response.value
Write-Host (" Retrieved " + $pageRecordCount + " records. Total: " + $allSignIns.Count)
}
# CRITICAL: Check for next page
$nextLink = $response.'@odata.nextLink'
}
The problem: Processing 80,000+ sign-ins was taking an extraordinary amount of time and generating massive result sets.
Step 4: Optimizing for Unique Users Only
I realized I only needed to know WHO was using SMS, not every instance. So I modified the script to track unique users:
# Use hashtable for O(1) lookup performance
$uniqueUsers = @{}
while ($nextLink) {
$response = Invoke-RestMethod -Uri $nextLink -Headers $headers -Method Get
foreach ($signIn in $response.value) {
# Skip if we already have this user - huge performance gain
if ($uniqueUsers.ContainsKey($signIn.userPrincipalName)) {
continue
}
# Check authentication details
if ($signIn.authenticationDetails) {
foreach ($authDetail in $signIn.authenticationDetails) {
if ($authDetail.authenticationMethod -eq "Text message") {
# Store only first occurrence
$uniqueUsers[$signIn.userPrincipalName] = [PSCustomObject]@{
UserPrincipalName = $signIn.userPrincipalName
UserDisplayName = $signIn.userDisplayName
FirstSMSAuthDate = $signIn.createdDateTime
LastKnownLocation = "$($signIn.location.city), $($signIn.location.state)"
LastKnownIP = $signIn.ipAddress
}
break # Stop checking this user's other auth methods
}
}
}
}
$nextLink = $response.'@odata.nextLink'
}
Step 5: Handling 429 Rate Limits
Entra started returning 429 errors (Too Many Requests). I had to implement intelligent retry logic:
function Invoke-GraphRequestWithRetry {
param(
[string]$Uri,
[hashtable]$Headers,
[int]$MaxRetries = 5
)
$retryCount = 0
$baseDelay = 1
while ($retryCount -lt $MaxRetries) {
try {
$response = Invoke-RestMethod -Uri $Uri -Headers $Headers -Method Get -ErrorAction Stop
return $response
}
catch {
$statusCode = $_.Exception.Response.StatusCode.value__
if ($statusCode -eq 429) {
# Extract Retry-After header if present
$retryAfter = 60 # Default
if ($_.Exception.Response.Headers.RetryAfter) {
$retryAfter = [int]$_.Exception.Response.Headers.RetryAfter.TotalSeconds
}
# Exponential backoff with jitter
$delay = [Math]::Min($retryAfter, $baseDelay * [Math]::Pow(2, $retryCount))
$jitter = Get-Random -Minimum 0 -Maximum 5
$totalDelay = $delay + $jitter
Write-Host (" Rate limited! Waiting " + $totalDelay + " seconds...")
Start-Sleep -Seconds $totalDelay
$retryCount++
}
else {
throw $_
}
}
}
throw "Max retries exceeded"
}
Step 6: Making it Resumable
Since the process could take hours and might get interrupted, I added resume capability using a JSON progress file:
function Save-Progress {
param(
[string]$NextLink,
[hashtable]$Users,
[int]$TotalProcessed,
[int]$PageNumber
)
$progress = @{
NextLink = $NextLink
TotalProcessed = $TotalProcessed
PageNumber = $PageNumber
UserCount = $Users.Count
LastUpdated = (Get-Date).ToString("yyyy-MM-dd HH:mm:ss")
}
$progress | ConvertTo-Json | Out-File ".\sms-auth-progress.json" -Force
# Also save users to temp CSV
if ($Users.Count -gt 0) {
$Users.Values | Export-Csv -Path ".\sms-auth-temp.csv" -NoTypeInformation
}
}
function Load-Progress {
if (Test-Path ".\sms-auth-progress.json") {
$savedProgress = Get-Content ".\sms-auth-progress.json" | ConvertFrom-Json
# Reload users from temp CSV
$savedUsers = @{}
if (Test-Path ".\sms-auth-temp.csv") {
$csvUsers = Import-Csv ".\sms-auth-temp.csv"
foreach ($user in $csvUsers) {
$savedUsers[$user.UserPrincipalName] = $user
}
}
return @{
Progress = $savedProgress
Users = $savedUsers
}
}
return $null
}
Putting It All Together
The complete flow now looks like this:
# Main processing loop with all features
$uniqueUsers = @{}
$pageNumber = 0
# Check for resume
if ($Resume) {
$savedData = Load-Progress
if ($savedData) {
$uniqueUsers = $savedData.Users
$nextLink = $savedData.Progress.NextLink
$pageNumber = $savedData.Progress.PageNumber
}
}
try {
while ($nextLink) {
$pageNumber++
# Make request with retry
$response = Invoke-GraphRequestWithRetry -Uri $nextLink -Headers $headers
# Process page
$newUsersThisPage = 0
foreach ($signIn in $response.value) {
if (-not $uniqueUsers.ContainsKey($signIn.userPrincipalName)) {
foreach ($authDetail in $signIn.authenticationDetails) {
if ($authDetail.authenticationMethod -eq "Text message") {
$uniqueUsers[$signIn.userPrincipalName] = [PSCustomObject]@{
# User details...
}
$newUsersThisPage++
break
}
}
}
}
Write-Host ("Page " + $pageNumber + ": Found " + $newUsersThisPage + " new SMS users. Total: " + $uniqueUsers.Count)
# Get next page
$nextLink = $response.'@odata.nextLink'
# Save progress
if ($nextLink) {
Save-Progress -NextLink $nextLink -Users $uniqueUsers -TotalProcessed $totalProcessed -PageNumber $pageNumber
Start-Sleep -Milliseconds 500 # Prevent rate limiting
}
}
}
catch {
Write-Host "Error occurred. Progress saved. Use -Resume to continue." -ForegroundColor Yellow
exit 1
}
# Export final results
$uniqueUsers.Values | Export-Csv -Path $OutCsv -NoTypeInformation
Key Technical Details
- Authentication methods in logs: "Text message" is the exact string to search for SMS authentication
- Skip "Previously satisfied": These are not actual authentication events
- Hashtable for performance: Using
$uniqueUsers.ContainsKey()is O(1) vs array searching at O(n) - @odata.nextLink: This property contains the URL for the next page - don't construct it manually
- Rate limit headers: The Retry-After header tells you exactly how long to wait
- Progress persistence: Save both the next URL and discovered data for true resume capability
Conclusion
Running this on our tenant of ~40,000 users:
- Processed ~181,000 sign-in records over 81 pages
- Found 2580 unique users still using SMS authentication
- Took approximately 95 minutes with rate limit delays
- Successfully resumed twice after 429 errors
The final CSV provides a clean list of users actually using SMS, not just those who have it registered.