prod@blog:~$

From Daily Alerts to Weekly Insights: Reducing Notification Fatigue in Queue Monitoring

In my previous guide in this post here, I covered implementing daily reporting with heatmap-styled emails that summarized queue activity patterns. However, after running the system for a while, I realized I'd created an alerting overload problem. Not only was I getting emails every time a queue exceeded zero, but I was also receiving daily summary emails at 7 AM. The constant stream of notifications was becoming counterproductive.

Visual Images

This is the report after 7 days, first this time lets start with the heat map, where you can see that the queuing emails over the whole week around 06:00 and up to 17:00 at the latest on this example below:


Then at the top of the email you get all the statistics and a breakdown day-by-day so you can easily see where the larger amount of email has come from on which day:


The Alert Fatigue Problem

The original implementation had two notification streams:

  1. Real-time alerts whenever queues > 0 (potentially dozens per day)
  2. Daily summary emails with heatmaps

During busy periods, this could mean 20+ emails per day. While the information was valuable, the signal-to-noise ratio was poor. I needed a way to maintain visibility without the constant interruptions.

This amendment is also handy. If you have other platforms that can monitor accuses and generate alert, officially via the SIEM Into your ticket management system of choice.

The Weekly Summary Solution

I modified the script to support weekly pattern analysis with optional real-time alert suppression. The key insight: most queue patterns repeat weekly, so a Thursday-to-Thursday view provides better strategic insights than daily snapshots.

Configurable Alert Behavior

The first major change was adding a toggle for real-time notifications, this means if another solution can monitor the queue you can keep the weekly heatmap without the daily alerts:

# Notification Settings
$enableRealtimeNotifications = $false  # Set to $false to disable real-time alerts

This simple boolean completely changes the script's behavior:

  • $true : Original behavior - alerts for every queue event
  • $false : Silent monitoring - only weekly summaries

The implementation wraps the existing alert logic:

# Send email if values > 0 AND notifications are enabled
if ($enableRealtimeNotifications -and ($FireEmail -or ($attachmentValue -gt 0 -or $messageValue -gt 0))) {
    # ... send alert email ...
} else {
    if (-not $enableRealtimeNotifications) {
        Write-Host "[INFO] Real-time notifications disabled. Data recorded only."
    } else {
        Write-Host "[INFO] No queues with non-zero values; email not sent."
    }
}

Weekly Tracking Implementation

Dynamic Week Boundaries

Unlike daily summaries that reset at midnight, weekly summaries need to track from the script's start day. If you start monitoring on a Thursday, your weeks run Thursday-to-Thursday:

function Initialize-WeeklyTracking {
    if (-not (Test-Path $weeklyStartFile)) {
        $startDate = (Get-Date).ToString("yyyy-MM-dd")
        $startDate | Set-Content $weeklyStartFile -Force
        Write-Host "[INFO] Weekly tracking initialized. Start date: $startDate"
    }
}

Week Completion Detection

The script checks if exactly 7 days have passed before sending summaries:

function Should-SendWeeklySummary {
    $currentHour = (Get-Date).Hour
    
    # Check if it's 7 AM
    if ($currentHour -ne $summaryHour) {
        return $false
    }
    
    # Check if a week has passed
    $startDateStr = Get-Content $weeklyStartFile
    $startDate = [DateTime]::ParseExact($startDateStr, "yyyy-MM-dd", $null)
    $daysSinceStart = ((Get-Date).Date - $startDate.Date).Days
    
    # Send summary if exactly 7 days have passed
    if ($daysSinceStart -eq 7) {
        return $true
    }
    
    return $false
}

Enhanced Data Structure

Weekly tracking required additional metadata in each reading:

$newEntry = @{
    Timestamp = (Get-Date).ToString("yyyy-MM-dd HH:mm:ss")
    Date = (Get-Date).ToString("yyyy-MM-dd")
    DayOfWeek = (Get-Date).DayOfWeek.ToString()  # New field for weekly grouping
    Hour = (Get-Date).Hour
    AttachmentQueue = $AttachmentValue
    MessageQueue = $MessageValue
}

The DayofWeek field enables proper grouping in the weekly heatmap.

Building the Weekly Heatmap

The weekly visualization required a complete redesign. Instead of 24 columns, I needed 7 rows × 24 columns = 168 cells:

# Build hourly heatmap data for the entire week
$weeklyHeatmap = @{}
foreach ($day in $daysOfWeek) {
    $weeklyHeatmap[$day] = @{}
    0..23 | ForEach-Object { 
        $weeklyHeatmap[$day][$_] = @{
            Attachment = 0
            Message = 0
        }
    }
}

Correct Day Ordering

A critical detail: the heatmap must respect the actual week boundaries. If monitoring starts Thursday, the display order should be Thursday through Wednesday:

# Start from the week start day
$startDayIndex = [Array]::IndexOf($daysOfWeek, $startDate.DayOfWeek.ToString())
for ($i = 0; $i -lt 7; $i++) {
    $dayIndex = ($startDayIndex + $i) % 7
    $day = $daysOfWeek[$dayIndex]
    # ... render day in correct order ...
}

Daily Peak Summary Cards

I added a new visualization showing daily maximums at a glance:

<div class="daily-summary">
    <div class="day-card">
        <div class="day-name">Monday</div>
        <div class="day-max" style="color: $attColor;">A: 15</div>
        <div class="day-max" style="color: $msgColor;">M: 12</div>
    </div>
    <!-- ... more days ... -->
</div>

This provides quick daily insights without scrolling through the full heatmap.

JSON File for Time stamping and Tracking

To support the data for the support I use a JSON file for monitoring history in this section here:

# Weekly monitoring
$trendingDataFile = Join-Path $trendingDataFolder "QueueTrendsWeekly.json"

Status Communication

The script now clearly communicates its configuration:

Write-Host "[INFO] Real-time notifications are: $(if ($enableRealtimeNotifications) { "ENABLED" } else { "DISABLED" })"
Write-Host "[INFO] Current weekly period: $weekStart to $([DateTime]::ParseExact($weekStart, 'yyyy-MM-dd', $null).AddDays(6).ToString('yyyy-MM-dd'))"

Key Implementation Improvements

1. Metric Calculations

Weekly metrics scale appropriately:

# Active hours for the week (out of 168 hours)
$attachmentActiveHours = ($attachmentReadings.Count / [Math]::Max($totalReadings, 1)) * 168

2. HTML Table Structure

Each day gets two rows in the heatmap:

<tr>
    <td rowspan='2' class='day-header'>Monday</td>
    <td style='font-weight: 600;'>Attach</td>
    <!-- 24 hour cells -->
</tr>
<tr>
    <td style='font-weight: 600;'>Msg</td>
    <!-- 24 hour cells -->
</tr>

3. Disclaimer Updates

The disclaimer now indicates notification status:

Real-time notifications are $(if ($enableRealtimeNotifications) { "enabled" } else { "disabled" }).

Operational Tips

I've found the following setup works well:

  • Run weekly monitoring as the default ($enableRealtimeNotifications = $false)
  • Enable real-time alerts only during high-risk periods (deployments, holidays)
  • Review weekly summaries in Monday morning meetings for planning

The complete solution provides the visibility needed for capacity planning while eliminating the notification fatigue that made the original system less effective. Sometimes the best enhancement is knowing when to show less, not more.