AWS Glue Data Quality for MCP Servers

DQDL ruleset syntax

DQDL (Data Quality Definition Language) is a declarative language for expressing data quality rules. A ruleset is a named collection of rules applied to a single DynamicFrame or Glue table.

# DQDL ruleset for MCP tool call events
# Syntax: Rules = [ RuleType "column_name" [operator threshold] ]

Rules = [
    # Completeness: fraction of non-null values >= threshold
    Completeness "server_id" >= 0.99,
    Completeness "tool_name" >= 0.99,
    Completeness "timestamp_ms" >= 1.0,   # timestamp must always be present

    # Uniqueness: fraction of unique values in the column
    Uniqueness "event_id" >= 0.999,       # event_id should be near-unique

    # IsPositive: all values in column are > 0
    IsPositive "duration_ms",
    IsPositive "timestamp_ms",

    # ColumnValues: custom threshold on a specific value range
    ColumnValues "duration_ms" <= 60000,  # No tool call should take > 60 seconds

    # IsComplete: equivalent to Completeness >= 1.0 (all values non-null)
    IsComplete "session_id",

    # RowCount: dataset should have at least N rows per partition run
    RowCount >= 1000,

    # CustomSql: arbitrary SQL predicate — fails if the WHERE clause matches any row
    CustomSql "SELECT count(*) FROM primary WHERE error = true AND duration_ms < 10"
        BETWEEN 0 AND 100,    # Allow up to 100 fast errors (e.g., auth failures); more is suspicious
]

Rule reference table:

Evaluation modes: EVALUATE, FAIL, QUARANTINE

# PySpark Glue script — Data Quality with QUARANTINE mode
from awsglue.transforms import EvaluateDataQuality
from awsglue.context import GlueContext
from awsgluedi.transforms import dq

glueContext = GlueContext(sc)

# Define the ruleset inline or reference a catalog ruleset by ARN
RULESET = """
Rules = [
    Completeness "server_id" >= 0.99,
    IsPositive "duration_ms",
    Uniqueness "event_id" >= 0.999,
    RowCount >= 1000
]
"""

# QUARANTINE mode: splits the frame into passing and failing rows
dq_results = EvaluateDataQuality.apply(
    frame=source_df,
    ruleset=RULESET,
    publishing_options={
        "dataQualityEvaluationContext": "mcp_tool_calls_dq",
        "enableDataQualityResultsPublishing": True,  # Publish to Glue Data Quality console
        "enableDataQualityCloudWatchMetrics": True,  # Emit CloudWatch metrics
    },
    additional_options={
        "observations.scope": "ALL",
        "performanceTuning.caching": "CACHE_NOTHING",
    },
)

# Extract the good and bad row frames
passing_rows = dq_results.select_fields(["rowResults"]).select_from_collection(
    dataset="rowLevelResults",
    filterExpression="allRulesPassed == true",
)
failing_rows = dq_results.select_fields(["rowResults"]).select_from_collection(
    dataset="rowLevelResults",
    filterExpression="allRulesPassed == false",
)

# Write good rows to the main target
glueContext.write_dynamic_frame.from_options(
    frame=passing_rows,
    connection_type="s3",
    connection_options={"path": "s3://mcp-data/processed/tool_calls/"},
    format="parquet",
    transformation_ctx="write_good",
)

# Write bad rows to quarantine for investigation
glueContext.write_dynamic_frame.from_options(
    frame=failing_rows,
    connection_type="s3",
    connection_options={"path": "s3://mcp-data/quarantine/tool_calls/"},
    format="json",
    transformation_ctx="write_quarantine",
)

Choosing the right mode:

• EVALUATE: use during development or for non-critical pipelines where you want visibility into data quality without breaking anything. All rows flow to the target; rules just emit metrics.
• FAIL: use when downstream consumers cannot tolerate bad data (e.g., a billing or SLA report). The Glue job raises an exception and terminates if any rule fails below its threshold.
• QUARANTINE: use in production for operational pipelines. Bad rows are isolated for analysis; the pipeline continues processing good rows without interruption. This is the operational sweet spot for MCP log pipelines where occasional malformed events should not halt the entire daily transform.

Anomaly detection rules

Anomaly detection rules use a statistical model trained on historical evaluation results for a column. They detect when a metric (null rate, min value, max value, mean, histogram distribution) deviates significantly from the historical baseline.

# Anomaly detection DQDL — detects statistical drift without fixed thresholds
# Requires >= 10 prior evaluation runs to build a stable baseline

AnomalyRules = [
    # Detect sudden increase in null rate for server_id
    # (would indicate a bug in the upstream ingestion pipeline)
    AnomalyDetection "Completeness" "server_id",

    # Detect unusual spikes or drops in total row count
    # (e.g., Firehose delivery outage dropping entire hours)
    AnomalyDetection "RowCount",

    # Detect distribution shift in duration_ms
    # (e.g., a performance regression causing all tool calls to take 10x longer)
    AnomalyDetection "Mean" "duration_ms",
]

Warm-up requirement: anomaly detection requires a training window. The rule will return PASS for the first 10 evaluations regardless of the data values — it is learning the baseline during this period. After the warm-up, any deviation beyond 3 standard deviations from the mean triggers a FAIL. For new pipelines, run a backfill of historical partitions to fast-forward the training window before enabling anomaly detection in a FAIL or QUARANTINE mode.

Configuration options: the sensitivity of anomaly detection can be tuned via the anomalyDetectionConfig in the Data Quality settings (Glue console or API). The default threshold is 3σ; reducing to 2σ increases sensitivity but also false positive rate for naturally bursty MCP traffic patterns.

Publishing results to CloudWatch

When enableDataQualityCloudWatchMetrics: true is set, Glue publishes Data Quality rule evaluation results as CloudWatch metrics under the Glue/DataQuality namespace.

# CloudWatch metric structure
Namespace: Glue/DataQuality
Dimensions:
  - JobName: mcp-log-transform
  - JobRunId: jr_abc123
  - RulesetName: mcp_tool_calls_dq
  - RuleName: Completeness_server_id  # auto-generated from rule type + column

Metrics emitted per rule:
  - EvaluationResult: 1.0 (PASS) or 0.0 (FAIL)
  - ActualScore: the measured value (e.g., 0.987 = 98.7% completeness)

# CloudFormation: CloudWatch alarm on overall Data Quality score drop
import boto3
cw = boto3.client("cloudwatch")

cw.put_metric_alarm(
    AlarmName="mcp-pipeline-dq-completeness-drop",
    AlarmDescription="server_id completeness dropped below 99% in MCP log ETL",
    Namespace="Glue/DataQuality",
    MetricName="EvaluationResult",
    Dimensions=[
        {"Name": "JobName", "Value": "mcp-log-transform"},
        {"Name": "RuleName", "Value": "Completeness_server_id"},
    ],
    Statistic="Minimum",
    Period=3600,           # 1-hour window
    EvaluationPeriods=1,
    Threshold=1.0,         # Alert if any evaluation in the period was a FAIL
    ComparisonOperator="LessThanThreshold",
    TreatMissingData="notBreaching",  # Don't alarm if job didn't run
    AlarmActions=["arn:aws:sns:us-east-1:123456789012:mcp-alerts"],
)

Glue Studio visual ruleset builder

Glue Studio includes a point-and-click interface for building DQDL rulesets without writing the DSL syntax manually. It samples the DynamicFrame from a live job run, shows column statistics (null rate, distinct values, min/max), and lets you set rule thresholds visually.

The visual builder is useful for the initial ruleset bootstrap — explore column statistics, click "recommend rules" to auto-generate a baseline ruleset based on the sampled data, then export the DQDL and add it to version control. Do not use the visual builder as the primary edit path for production rulesets — keep DQDL files in Git so schema changes are reviewed in PRs.

# Export a ruleset from the Glue Data Quality API for version control
import boto3, json

dq = boto3.client("glue")

ruleset = dq.get_data_quality_ruleset(
    Name="mcp_tool_calls_dq"
)

# Save to version-controlled file
with open("glue/rulesets/mcp_tool_calls_dq.dqdl", "w") as f:
    f.write(ruleset["Ruleset"])

Common data quality failures in MCP log pipelines