Building a Secure Alternative to CIS-CAT Pro Dashboard: From WinRM Woes to PowerShell Solutions

When I first looked into deploying CIS-CAT Pro Dashboard for tracking compliance across our infrastructure, I was excited about the potential. The tool uses MariaDB and a Java runtime to execute CIS compliance checks on remote servers, with the promise of tracking security scores over time through an intuitive dashboard interface. What could go wrong?

Well, quite a lot, as it turns out.

The WinRM Dilemma

The first red flag was CIS-CAT Pro Dashboard's reliance on Windows Remote Management (WinRM). While WinRM can be configured in two ways:

• Port 5985: Non-HTTPS (insecure)
• Port 5986: HTTPS with TLS encryption (secure)

The obvious choice would be the encrypted option, but that's where the complications began. Even with TLS encryption, the authentication options presented their own challenges. The tool supports basic authentication and certificate-based authentication, but the documentation seemed to steer away from basic authentication for good reason.

Plain Text Credentials: A Non-Starter

The real deal-breaker came when I discovered how credentials are handled. The tool requires you to specify a username (typically a service account) in a configuration file, stored in plain text. While a plain text username might be acceptable, the password handling was equally problematic:

1. Option 1: Store the password in clear text alongside the username
2. Option 2: Leave the password field empty and get prompted during execution

Neither option sat well with me from a security perspective. The first is obviously terrible practice, while the second introduces too much potential for human error and doesn't scale well for automated deployments.

Group Policy : Blocks these communications

Here's where things got really interesting. In our environment—and likely in most security-conscious organizations—Group Policy has disabled:

• WinRM services
• Basic authentication for remote connections
• Non-encrypted remote connections

This configuration aligns perfectly with CIS baseline recommendations, but it completely invalidates the CIS-CAT Pro Dashboard approach. Even if you tried to configure trusted hosts, the existing Group Policy Objects would simply override those settings.

It's ironic that a tool designed to check CIS compliance can't function in an environment that follows CIS recommendations.

Dashboards are "ideal" for visualising data

Despite these obstacles, I genuinely liked the core concept of the dashboard. The ability to track compliance scores over time, identify trending issues, and visualize security posture improvements is incredibly valuable for any security program.

So I thought to myself: "This sounds like a challenge. What if I could recreate this functionality using local checks and PSExec, then extract the scores with PowerShell and display them on a clean, easy-to-read website?"

The Solution: Building my own Solution!

That's exactly what I set out to do. Instead of fighting against security best practices to make a flawed tool work, I decided to build a solution that:

• Respects existing security configurations
• Uses local execution instead of problematic remote protocols
• Leverages PowerShell for data extraction and processing
• Provides the same visual tracking capabilities through a custom web interface

In the following sections, I'll walk you through the code and architecture that makes this possible, showing how you can achieve the same compliance tracking goals without compromising your security posture.

Visual Results

Lets look at the final Dashboard, there you can easily see the trend of the scores with the up/down arrow:

You then get the summary of all the tests:

Finally you can search for servers:

Phase 1 : Secure Assessment Deployment - DeployRemote.ps1

Now let's examine the PowerShell solution I built to replace the problematic WinRM-based approach.

The DeployRemote.ps1 script accomplishes everything the original dashboard promised, but does it securely and reliably.

The script follows a four-step process:

1. Interactive Benchmark Selection - Automatically detects the target OS and presents appropriate CIS benchmarks
2. Secure File Deployment - Uses administrative shares to deploy CIS-CAT locally on the target server
3. Local Execution - Runs the assessment directly on the target server using PSExec
4. Report Collection - Retrieves results and cleans up temporary files

Step 1: Intelligent OS Detection and Benchmark Selection

Instead of requiring manual configuration, the script automatically the target system's OS and role and asks you to confirm that selection to then build the remote configuration file:

# Try Active Directory first for accurate OS information
Import-Module ActiveDirectory -ErrorAction Stop
$adComputer = Get-ADComputer -Identity $computerName -Properties Name, OperatingSystem, DistinguishedName

$osName = $adComputer.OperatingSystem
$distinguishedName = $adComputer.DistinguishedName

# Determine OS version from AD OS Name
if ($osName -like "*Server 2022*") { $detectedOSVersion = "2022" }
elseif ($osName -like "*Server 2019*") { $detectedOSVersion = "2019" }
elseif ($osName -like "*Server 2016*") { $detectedOSVersion = "2016" }

# Determine server role from OU location
if ($distinguishedName -like "*OU=Domain Controllers*") {
    $serverRole = "Domain Controller"
}

This approach leverages Active Directory metadata to make intelligent decisions about which benchmarks and profiles to present, falling back to WMI queries if AD isn't available.

Step 2: Secure File Deployment

The script uses the tried-and-true administrative shares approach to copy the files to the target computer system from the folder defined in the $AssessorPath variable this also includes the license file for executution

# Test admin access to target server
$adminShare = "\\$TargetServer\C$"
try {
    $null = Get-ChildItem $adminShare -ErrorAction Stop | Select-Object -First 1
    Write-Host "✓ Administrative access confirmed" -ForegroundColor Green
} catch {
    Write-Host "ERROR: Cannot access $adminShare" -ForegroundColor Red
    exit 1
}

# Use robocopy for reliable copying with progress
$robocopyArgs = @(
    "`"$AssessorPath`"",
    "`"$fullRemotePath`"",
    "/E",           # Copy subdirectories including empty ones
    "/Z",           # Copy files in restartable mode
    "/R:3",         # Retry 3 times on failure
    "/MT:8"         # Multi-threaded copying (8 threads)
)

This method respects existing Group Policy configurations while providing reliable file transfer with built-in retry logic and progress monitoring.

Step 3: Local Execution with PSExec

The key insight was to run CIS-CAT locally on each target server rather than trying to manage remote connections. The script creates a dynamic batch file with the selected benchmark and profile, copies it to the target server, and then executes it:

# Create a dynamic batch file with the selected benchmark and profile
$batchContent = @"
@echo off
echo Starting CIS-CAT Assessment...
cd /d "$RemotePath"
Assessor-CLI.bat -html -b "$RemotePath\benchmarks\$selectedBenchmark" -p "$selectedProfile"
echo Assessment completed with exit code: %ERRORLEVEL%
"@

# Save the batch file locally first, then copy it to remote server
$localBatchFile = ".\run_cis_assessment.bat"
$remoteBatchFile = "$fullRemotePath\run_cis_assessment.bat"

$batchContent | Out-File -FilePath $localBatchFile -Encoding ASCII -Force
Copy-Item $localBatchFile -Destination $remoteBatchFile -Force

# Run the copied batch file via PSExec
$psexecArgs = @(
    "\\$TargetServer",
    "-h",
    "cmd.exe",
    "/c",
    "$RemotePath\run_cis_assessment.bat"
)

$process = Start-Process -FilePath $PSExecPath -ArgumentList $psexecArgs -Wait -PassThru -NoNewWindow

This approach eliminates all the WinRM complexity while providing full output visibility and proper error handling. The dynamic batch file ensures that each assessment uses the correct benchmark and profile for the target system.

Step 4: Automated Report Collection

Once the assessment completes, the script automatically retrieves the results:

# Automatically copy reports back from remote server
$localReportsPath = ".\Reports\$TargetServer"
$remoteReportsPath = "$fullRemotePath\reports"

Copy-Item "$remoteReportsPath\*" -Destination $localReportsPath -Recurse -Force

# Show what was copied
$reportFiles = Get-ChildItem $localReportsPath -File
$htmlFiles = $reportFiles | Where-Object { $_.Extension -eq ".html" }
$xmlFiles = $reportFiles | Where-Object { $_.Extension -eq ".xml" }

Write-Host "Summary: $($htmlFiles.Count) HTML report(s), $($xmlFiles.Count) XML report(s)"

Step 5 : Automatic Cleanup

The script offers to clean up temporary files after report collection, maintaining good hygiene:

$cleanup = Read-Host "Clean up remote CIS-CAT files? (y/n) [n]"
if ($cleanup -eq 'y') {
    Remove-Item $fullRemotePath -Recurse -Force
    Write-Host "✓ Remote files cleaned up" -ForegroundColor Green
}

Phase 2 : Data Extraction - Score-Extractor.ps1

Once we have HTML reports from multiple servers, we need to extract the key metrics for dashboard visualization. The Score-Extractor.ps1 script transforms individual HTML reports into structured CSV data that can be easily consumed by web dashboards or reporting tools.

The Challenge: Parsing Unstructured HTML Reports

CIS-CAT generates comprehensive HTML reports with detailed findings, but extracting just the compliance scores requires careful parsing of the HTML structure. The reports contain nested tables, CSS styling, and various formatting that makes simple text parsing unreliable.

Step 1: Intelligent Report Discovery and Cleanup

The script starts by cleaning up the reports directory and identifying HTML files to process:

# Step 1: Delete any XML files (keep only HTML reports)
$xmlFiles = Get-ChildItem -Path $ReportsPath -Filter "*.xml" -ErrorAction SilentlyContinue
if ($xmlFiles) {
    foreach ($xmlFile in $xmlFiles) {
        Remove-Item $xmlFile.FullName -Force
        Write-Host "  ✓ Deleted: $($xmlFile.Name)" -ForegroundColor Green
    }
}

# Step 2: Process HTML files
$htmlFiles = Get-ChildItem -Path $ReportsPath -Filter "*.html" -ErrorAction SilentlyContinue
Write-Host "Found $($htmlFiles.Count) HTML file(s) to process" -ForegroundColor Green

This cleanup step ensures we're only processing the relevant HTML reports while maintaining a clean working environment.

Step 2: Robust HTML Content Parsing

The core of the script uses regex patterns to extract specific data points from the HTML structure:

# Extract server name from coverPageTitle section
$serverName = ""
if ($htmlContent -match '<h1>for\s+([^<]+)</h1>') {
    $serverName = $matches[1].Trim()
}
elseif ($htmlContent -match 'for\s+([^\s<]+)') {
    $serverName = $matches[1].Trim()
}

# Extract OS version, test level, and date from structured sections
if ($htmlContent -match '<h2>(CIS[^<]+)</h2>\s*<ul>\s*<li>([^<]+)</li>\s*<li>([^<]+)</li>') {
    $fullOSString = $matches[1].Trim()
    $testLevel = $matches[2].Trim()
    $testDate = $matches[3].Trim()
    
    # Parse OS version from full CIS benchmark name
    if ($fullOSString -match 'Windows\s+(Server\s+\d{4}|\d+|11|10)') {
        $osVersion = $matches[1].Trim()
    }
}

The script uses multiple fallback patterns to handle variations in HTML formatting, ensuring reliable extraction even if CIS-CAT changes their report structure slightly.

Step 3: CIS Score Extraction

The most important piece of data is the compliance percentage score. The script uses precise regex patterns to locate the score in the report's summary table:

# Extract percentage score - look for the pattern before "actual scores" note
if ($htmlContent -match '<td class="numeric bold">(\d+\.?\d*%)</td>\s*</tr>\s*</tbody>\s*</table>\s*<p class="caption"><b>Note</b>:\s*Actual scores') {
    $scorePercentage = $matches[1].Trim()
}
# Fallback: Look for any td with numeric bold class containing percentage
elseif ($htmlContent -match '<td class="numeric bold">(\d+\.?\d*%)</td>') {
    $scorePercentage = $matches[1].Trim()
}

This two-tier approach ensures we capture the summary score accurately, with a fallback pattern for edge cases.

Step 4: Structured Data Output in CSV

The extracted data is organized into a clean CSV format perfect for dashboard consumption:

# Create structured record for each server
$record = [PSCustomObject]@{
    ServerName = $serverName
    OS = $osVersion
    TestLevel = $testLevel
    Date = $testDate
    ScorePercentage = $scorePercentage
    FileName = $htmlFile.Name
}

$csvData += $record

# Export to timestamped CSV file
$csvFileName = "CIS-CAT-Report-Summary-$(Get-Date -Format 'yyyyMMdd-HHmmss').csv"
$csvData | Export-Csv -Path $csvPath -NoTypeInformation -Encoding UTF8

Phase 3: Visual Dashboard Creation - Dashboard-Generator.ps1

The final piece of our solution is transforming the extracted CSV data into a beautiful, interactive web dashboard. The Dashboard-Generator.ps1 script creates a modern HTML dashboard that provides all the visual tracking capabilities we wanted from the original CIS-CAT Pro Dashboard.

Step 1 : Intelligent Data Processing and Trend Analysis

The script starts by automatically finding the latest CSV file and processing it for dashboard visualization:

# Find the latest CSV file automatically
$csvFiles = Get-ChildItem -Path "." -Filter "*CIS*CAT*Report*Summary*.csv" | Sort-Object LastWriteTime -Descending
$latestCsv = $csvFiles[0]

# Read and clean the data
$csvData = Import-Csv -Path $latestCsv.FullName
$csvData = $csvData | ForEach-Object {
    $_.ServerName = $_.ServerName.Trim()
    $_.OS = $_.OS.Trim()
    $_.ScorePercentage = $_.ScorePercentage.Trim()
    $_
}

The real intelligence comes in trend calculation. The script analyzes historical assessments for each server to determine if compliance scores are improving, declining, or staying steady:

# Group assessments by server and calculate trends
$serverGroups = $csvData | Group-Object ServerName

foreach ($group in $serverGroups) {
    $allAssessments = $group.Group | Sort-Object { [DateTime]::Parse($_.Date) }
    
    # Separate Level 1 and Level 2 assessments
    $level1Assessments = @($allAssessments | Where-Object { $_.TestLevel -match "Level 1" })
    $level2Assessments = @($allAssessments | Where-Object { $_.TestLevel -match "Level 2" })
    
    # Calculate trend for Level 1
    if ($level1Assessments.Count -gt 1) {
        $latest = [int]($level1Assessments[-1].ScorePercentage -replace '%', '')
        $previous = [int]($level1Assessments[-2].ScorePercentage -replace '%', '')
        
        if ($latest -gt $previous) {
            $level1Trend = "up"
            $level1Icon = "▲"
            $level1Color = "green"
        } elseif ($latest -lt $previous) {
            $level1Trend = "down"
            $level1Icon = "▼"
            $level1Color = "red"
        } else {
            $level1Trend = "same"
            $level1Icon = "►"
            $level1Color = "orange"
        }
    }
}

Step 2 : Web Dashboard Generation

The script generates a fully self-contained HTML dashboard with modern CSS styling and interactive JavaScript:

$htmlContent = @"
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>CIS-CAT Assessment Dashboard</title>
    <style>
        body {
            font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
            min-height: 100vh;
            padding: 20px;
        }
        
        .server-card {
            background: white;
            border-radius: 15px;
            padding: 25px;
            box-shadow: 0 10px 30px rgba(0,0,0,0.2);
            transition: transform 0.3s ease, box-shadow 0.3s ease;
        }
        
        .server-card:hover {
            transform: translateY(-5px);
            box-shadow: 0 15px 40px rgba(0,0,0,0.3);
        }
"@

Step 3 : Interactive Features

The dashboard includes real-time search functionality that allows users to quickly find specific servers:

function filterServers() {
    const searchInput = document.getElementById('serverSearch');
    const serverCards = document.querySelectorAll('.server-card');
    const searchText = searchInput.value.toLowerCase().trim();
    
    let visibleCount = 0;
    
    serverCards.forEach(card => {
        const serverName = card.getAttribute('data-server-name');
        
        if (searchText === '' || serverName.includes(searchText)) {
            card.classList.remove('hidden');
            visibleCount++;
        } else {
            card.classList.add('hidden');
        }
    });
    
    // Update results counter
    searchResults.textContent = 'Showing ' + visibleCount + ' of ' + totalCount + ' servers';
}

Step 4 : Visual Trend Indicators

Each server card displays visual trend indicators using colored arrows and icons:

# Generate server cards with trend indicators
foreach ($server in $serverData) {
    $htmlContent += @"
    <div class="server-card" data-server-name="$($server.ServerName.ToLower())">
        <div class="levels-container">
            <div class="level-section">
                <div class="level-trend" style="color: $($server.Level1Color)">$($server.Level1Icon)</div>
                <div class="level-title">Level 1</div>
                <div class="level-score">$($server.Level1Score)</div>
                <div class="level-date">$($server.Level1Date)</div>
            </div>
            <div class="level-section">
                <div class="level-trend" style="color: $($server.Level2Color)">$($server.Level2Icon)</div>
                <div class="level-title">Level 2</div>
                <div class="level-score">$($server.Level2Score)</div>
                <div class="level-date">$($server.Level2Date)</div>
            </div>
        </div>
    </div>
"@
}

The visual indicators use:

• Green ▲ for improving scores
• Red ▼ for declining scores
• Orange ► for stable scores
• Gray ■ for insufficient data

Step 5 : Assessment History Tracking

Each server card includes a detailed history section showing recent assessments:

# Add recent assessment history (last 5 assessments)
$recentAssessments = $server.AllAssessments | Sort-Object { [DateTime]::Parse($_.Date) } -Descending | Select-Object -First 5
foreach ($assessment in $recentAssessments) {
    $htmlContent += @"
    <div class="assessment-item">
        <div>
            <div class="assessment-date">$($assessment.Date)</div>
            <div style="font-size: 0.8rem; color: #6c757d;">$($assessment.TestLevel)</div>
        </div>
        <div class="assessment-score">$($assessment.ScorePercentage)</div>
    </div>
"@
}