AWS Transfer Family for MCP Servers: Multi-Tenant SFTP Intake, Custom Identity Providers, and Production Observability

Pattern 1 — Multi-Tenant SFTP File Intake

Server creation decisions and the immutability trap

A Transfer Family server is the top-level resource that binds together the endpoint type, identity provider, and protocols. Two settings are immutable after server creation: EndpointType (PUBLIC vs VPC) and IdentityProviderType (SERVICE_MANAGED, AWS_LAMBDA, API_GATEWAY, or AWS_DIRECTORY_SERVICE). Protocols (SFTP, FTPS, FTP) and the logging role can be changed after creation. If you create a PUBLIC server and later need VPC-level IP filtering or Elastic IPs for partner firewall allowlisting, you must create a new server, migrate all users, and delete the old one.

# Create a public SFTP server — internet accessible, no IP filtering
aws transfer create-server \
  --protocols SFTP \
  --endpoint-type PUBLIC \
  --identity-provider-type SERVICE_MANAGED \
  --logging-role arn:aws:iam::123456789012:role/TransferLoggingRole \
  --tags Key=Project,Value=mcp-file-intake

# Create a VPC SFTP server with Elastic IPs for static internet IPs
EIP1=$(aws ec2 allocate-address --domain vpc --query 'AllocationId' --output text)
EIP2=$(aws ec2 allocate-address --domain vpc --query 'AllocationId' --output text)

aws transfer create-server \
  --protocols SFTP \
  --endpoint-type VPC \
  --endpoint-details "{
    \"VpcId\": \"vpc-0abc123\",
    \"SubnetIds\": [\"subnet-0aaa\", \"subnet-0bbb\"],
    \"SecurityGroupIds\": [\"sg-0transfer\"],
    \"AddressAllocationIds\": [\"$EIP1\", \"$EIP2\"]
  }" \
  --identity-provider-type SERVICE_MANAGED \
  --logging-role arn:aws:iam::123456789012:role/TransferLoggingRole

The VPC endpoint type provisions an NLB in the specified subnets — one node per AZ. Elastic IPs are optional: without them, the NLB is internal-only (reachable from the VPC, VPN, and Direct Connect but not the public internet). With Elastic IPs, partners get static IP addresses they can whitelist in their firewalls. The AddressAllocationIds array is positional — the first EIP maps to the first subnet. Adding a third AZ later requires a third EIP; you cannot add a subnet without also adding a corresponding EIP to the array.

The NLB preserves client source IPs — Transfer Family logs and custom identity provider Lambdas see the real partner IP, not a NAT gateway IP. This makes IP-based access control meaningful for VPC endpoints in a way that isn't possible with HTTP-proxied services.

S3 bucket setup: BucketOwnerEnforced and the bucket policy trap

The S3 bucket backing your Transfer Family server needs two non-default configurations before you create the first user. First, set ObjectOwnership to BucketOwnerEnforced — this disables ACLs entirely and makes the bucket owner account the owner of every object regardless of who uploaded it. Without this, uploaded files are owned by the Transfer Family service principal, and downstream Lambda functions reading those files via the bucket owner's IAM role get AccessDenied responses even with the correct IAM policy. Second, block all public access.

# Configure S3 bucket for Transfer Family intake
aws s3api create-bucket \
  --bucket mcp-file-intake \
  --region us-east-1

# Block all public access (required for intake buckets)
aws s3api put-public-access-block \
  --bucket mcp-file-intake \
  --public-access-block-configuration \
    BlockPublicAcls=true,IgnorePublicAcls=true,\
    BlockPublicPolicy=true,RestrictPublicBuckets=true

# BucketOwnerEnforced: disables ACLs, bucket owner owns all objects
aws s3api put-bucket-ownership-controls \
  --bucket mcp-file-intake \
  --ownership-controls '{"Rules": [{"ObjectOwnership": "BucketOwnerEnforced"}]}'

# Enable versioning for recovery of overwritten files
aws s3api put-bucket-versioning \
  --bucket mcp-file-intake \
  --versioning-configuration Status=Enabled

The bucket policy trap: many tutorials add an Allow * for transfer.amazonaws.com as the bucket policy principal. This grants every Transfer Family user on every server in your AWS account access to the bucket — not just users on your specific server. The correct approach is no bucket policy entry for Transfer Family at all. Access comes entirely through the IAM role attached to each user, scoped further by the session policy. If you do add a bucket policy (to enforce TLS, deny HTTP, or restrict to specific VPC endpoints), use aws:SourceAccount and aws:SourceArn conditions to scope it to your specific server ARN.

IAM role and session policy: the intersection model

Transfer Family applies two-layer IAM controls per user. The user role grants broad S3 access to the bucket. The session policy, passed as a JSON string in the --policy parameter at create-user time, is intersected with the role policy by AWS STS at session creation — the effective permission is only what both policies allow. This STS intersection is what provides multi-tenant isolation: Alice's session policy scoped to bucket/alice/* cannot access bucket/bob/* even though the shared role grants access to all prefixes.

# Shared IAM role permission policy — uses ${transfer:UserName} variable
# for automatic per-user scoping without per-user role creation
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListBucket",
      "Effect": "Allow",
      "Action": ["s3:ListBucket", "s3:GetBucketLocation"],
      "Resource": "arn:aws:s3:::mcp-file-intake",
      "Condition": {
        "StringLike": {
          "s3:prefix": ["${transfer:UserName}/*", "${transfer:UserName}"]
        }
      }
    },
    {
      "Sid": "ObjectAccess",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:GetObjectVersion"],
      "Resource": "arn:aws:s3:::mcp-file-intake/${transfer:UserName}/*"
    }
  ]
}

# Create user with LOGICAL home directory and session policy
aws transfer create-user \
  --server-id s-0abc \
  --user-name alice \
  --role arn:aws:iam::123456789012:role/TransferUserRole \
  --home-directory-type LOGICAL \
  --home-directory-mappings '[
    {"Entry": "/", "Target": "/mcp-file-intake/alice"},
    {"Entry": "/shared", "Target": "/mcp-file-intake/shared-read"}
  ]' \
  --policy '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Action": ["s3:*"],
      "Resource": [
        "arn:aws:s3:::mcp-file-intake",
        "arn:aws:s3:::mcp-file-intake/alice",
        "arn:aws:s3:::mcp-file-intake/alice/*",
        "arn:aws:s3:::mcp-file-intake/shared-read/*"
      ]
    }]
  }'

The ${transfer:UserName} policy variable in the role policy resolves to the authenticated username at session creation. This means you can use one role for all users — the role's s3:prefix condition automatically restricts listing to the requesting user's prefix. However, session policies passed at create-user time cannot use policy variables — they must contain literal usernames. This is why the session policy says mcp-file-intake/alice/* while the role policy says mcp-file-intake/${transfer:UserName}/*.

The HomeDirectoryType LOGICAL with HomeDirectoryMappings provides the user experience layer on top of the security isolation. Alice's SFTP root (/) maps to s3://mcp-file-intake/alice/ — she cannot see the bucket name, cannot traverse above /, and the /shared mapping gives her access to a second S3 prefix without exposing the full key hierarchy. ABSOLUTE mode is simpler but only supports a single S3 path and exposes raw key names. Use LOGICAL when you have multiple tenants, when users should not see S3 bucket names, or when you need to mount files from multiple prefixes or buckets into a single SFTP directory tree.

Lifecycle policies for long-term intake buckets

SFTP intake buckets accumulate objects indefinitely without lifecycle management. The standard pattern for MCP file processing pipelines: standard S3 storage for 30 days (active processing window), transition to S3-IA at 30 days (low-access archive), transition to Glacier Instant Retrieval at 90 days, expire at 365 days. Apply lifecycle to user-prefix paths, not the bucket root, to avoid archiving processed outputs in other prefixes.

# S3 lifecycle for SFTP intake prefixes
aws s3api put-bucket-lifecycle-configuration \
  --bucket mcp-file-intake \
  --lifecycle-configuration '{
    "Rules": [{
      "ID": "sftp-intake-lifecycle",
      "Status": "Enabled",
      "Filter": {"Prefix": ""},
      "Transitions": [
        {"Days": 30, "StorageClass": "STANDARD_IA"},
        {"Days": 90, "StorageClass": "GLACIER_IR"}
      ],
      "Expiration": {"Days": 365},
      "NoncurrentVersionExpiration": {
        "NoncurrentDays": 30,
        "NewerNoncurrentVersions": 5
      }
    }]
  }'

The NoncurrentVersionExpiration rule is critical when versioning is enabled. Without it, a partner who uploads the same filename daily (common batch patterns: orders-2026-10-03.csv overwriting orders-2026-10-02.csv) accumulates an unbounded number of noncurrent versions. The rule above retains the 5 most recent noncurrent versions and deletes older ones after 30 days — a 30-day recovery window without indefinite storage growth.

Triggering MCP processing on file arrival

Transfer Family writes files directly to S3. Two event mechanisms trigger downstream MCP tool execution: S3 event notifications (millisecond latency, simpler setup, no user context) and EventBridge File Transfer Complete events (slightly higher latency, includes username and session ID). Use S3 events when latency matters and you can derive context from the S3 key path. Use EventBridge when you need user context in the processing trigger without parsing the S3 key.

# S3 event notification — trigger Lambda on new uploads from any user
aws s3api put-bucket-notification-configuration \
  --bucket mcp-file-intake \
  --notification-configuration '{
    "LambdaFunctionConfigurations": [{
      "Id": "mcp-file-processor",
      "LambdaFunctionArn": "arn:aws:lambda:us-east-1:123:function:mcp-process-upload",
      "Events": ["s3:ObjectCreated:*"]
    }]
  }'

# Lambda handler — extract user from S3 key prefix
def handler(event, context):
    for record in event['Records']:
        key = record['s3']['object']['key']  # "alice/reports/q1.csv"
        username = key.split('/')[0]          # "alice"

        # Idempotency check before processing
        output_key = f"processed/{key}"
        try:
            s3.head_object(Bucket='mcp-processed', Key=output_key)
            print(f"Already processed: {key} — skipping")
            continue
        except s3.exceptions.ClientError as e:
            if e.response['Error']['Code'] != '404':
                raise

        # Process and write atomically (PutObject is atomic)
        process_and_write(key, output_key, username)

The HeadObject idempotency check prevents double-processing when Transfer Family retries a failed upload (the same S3 key gets a new version) or when Lambda event delivery fires more than once for the same object — Lambda guarantees at-least-once delivery. The guard reads: if the output already exists, skip. Since s3:PutObject is atomic, the output key either exists completely or doesn't — there is no partial-write state that would cause the guard to incorrectly skip an unfinished processing job.

Pattern 2 — Custom Identity Provider

AWS_LAMBDA vs the legacy API_GATEWAY type

Transfer Family originally required an API Gateway in front of the authentication Lambda — the API_GATEWAY identity provider type passes through the API Gateway invocation. The newer AWS_LAMBDA type invokes the Lambda directly without API Gateway. For new deployments, always use AWS_LAMBDA — it removes the API Gateway cost and complexity, and the Lambda receives the same event structure either way. The identity provider type is immutable, so if you have an existing server with API_GATEWAY type, you cannot migrate it in place.

# Create server with Lambda identity provider (direct invocation)
aws transfer create-server \
  --protocols SFTP \
  --endpoint-type PUBLIC \
  --identity-provider-type AWS_LAMBDA \
  --identity-provider-details '{
    "Function": "arn:aws:lambda:us-east-1:123456789012:function:sftp-auth"
  }' \
  --logging-role arn:aws:iam::123456789012:role/TransferLoggingRole

# Grant Transfer Family permission to invoke the Lambda
# source-arn scopes permission to this specific server
aws lambda add-permission \
  --function-name sftp-auth \
  --statement-id transfer-family-invoke \
  --action lambda:InvokeFunction \
  --principal transfer.amazonaws.com \
  --source-arn arn:aws:transfer:us-east-1:123456789012:server/s-0abc

# Test the identity provider without a real SFTP session
aws transfer test-identity-provider \
  --server-id s-0abc \
  --user-name alice \
  --user-password secretpassword123 \
  --source-ip 203.0.113.1

The test-identity-provider command is your primary debugging tool. It invokes the Lambda with the test credentials and returns the exact ServerResponse JSON the Lambda returned, the HTTP status code, and whether Transfer Family treated the response as success or failure. Run it after every Lambda change before testing with a real SFTP client — SFTP client errors like "Permission denied (publickey)" or "Authentication failed" give you no indication of what went wrong inside the Lambda.

Lambda event structure: password vs SSH key authentication

Transfer Family calls the identity provider Lambda on every connection. For password authentication, the event contains the username and password. For SSH key authentication, Transfer Family calls the Lambda first with an empty password field to retrieve the user's authorized public keys, then performs the key exchange locally — the Lambda never handles the private key material.

# Lambda event for password authentication
{
  "username": "alice",
  "password": "secretpassword123",
  "protocol": "SFTP",
  "serverId": "s-0abc123",
  "sourceIp": "203.0.113.1"
}

# Lambda event for SSH key authentication (password is empty string or absent)
{
  "username": "alice",
  "password": "",
  "protocol": "SFTP",
  "serverId": "s-0abc123",
  "sourceIp": "203.0.113.1"
}

# Python handler — detecting authentication type
def handler(event, context):
    username = event['username']
    password = event.get('password', '')
    source_ip = event.get('sourceIp', '')

    is_key_auth = not password  # Empty password = SSH key auth request

    user_config = get_user_config(username)
    if not user_config:
        return {}  # Unknown user — deny

    if is_key_auth:
        # Return config including PublicKeys for key verification
        return build_response(username, user_config)
    else:
        if not verify_password(password, user_config):
            return {}  # Bad password — deny
        return build_response(username, user_config)

For SSH key authentication, the PublicKeys array in the Lambda response contains the user's authorized public keys in OpenSSH format (ssh-rsa AAAA... comment). Transfer Family verifies the client's private key against these returned public keys locally — no cryptographic operations happen in the Lambda. The Lambda acts as a key-lookup service: given a username, return the associated public keys. Cache the Secrets Manager client outside the handler to amortize the ~50–100ms connection setup across warm invocations.

ServerResponse schema: the json.dumps() gotcha

The ServerResponse schema has two fields that must be JSON-serialized strings, not native Python dicts or lists. Passing HomeDirectoryDetails as a Python list causes Transfer Family to reject the response as malformed and deny authentication — but the Lambda returns HTTP 200, so the error looks like a credential failure rather than a schema error. The same applies to Policy: pass a json.dumps() string, not a Python dict.

import json
import boto3

sm = boto3.client('secretsmanager')

def build_response(username, user_config):
    return {
        # Required: IAM role ARN the user session assumes
        "Role": user_config['iam_role'],

        # Policy MUST be a JSON string — not a Python dict
        "Policy": json.dumps({
            "Version": "2012-10-17",
            "Statement": [{
                "Effect": "Allow",
                "Action": ["s3:*"],
                "Resource": [
                    "arn:aws:s3:::mcp-file-intake",
                    f"arn:aws:s3:::mcp-file-intake/{username}/*"
                ]
            }]
        }),

        # HomeDirectoryType LOGICAL for virtual directory isolation
        "HomeDirectoryType": "LOGICAL",

        # HomeDirectoryDetails MUST be a JSON string — not a Python list
        "HomeDirectoryDetails": json.dumps([
            {"Entry": "/", "Target": f"/mcp-file-intake/{username}"},
            {"Entry": "/shared", "Target": "/mcp-file-intake/shared-read"}
        ]),

        # PublicKeys for SSH key authentication (OpenSSH format)
        "PublicKeys": user_config.get('ssh_public_keys', [])
    }

# Auth denial: return empty dict (not None, not HTTP 4xx)
# Non-200 Lambda responses also deny, but Transfer Family may log
# "Unable to invoke function" rather than "authentication failed"
def deny():
    return {}

A useful mental model: HomeDirectoryDetails is the API equivalent of the --home-directory-mappings CLI argument, which takes a JSON string. The name ends in "Details" precisely because it is a stringified JSON blob, not a first-class nested object. If you forget json.dumps(), the Python dict gets serialized as the string "[{'Entry': '/', 'Target': '/bucket/alice'}]" — Python's str() representation, which is not valid JSON, and Transfer Family rejects it. Always verify with test-identity-provider after any change to the response structure.

Credentials lookup from Secrets Manager

Secrets Manager is the recommended backend for per-user SFTP credentials. Store each user's configuration as a secret at a predictable path like SFTP/{username}, containing the hashed password, salt, IAM role ARN, SSH public keys, and any per-user configuration (IP allowlist, home directory overrides). Use PBKDF2 with hmac.compare_digest for timing-safe password verification.

import hashlib, hmac, json
import boto3

sm = boto3.client('secretsmanager')

def handler(event, context):
    username = event['username']
    password = event.get('password', '')
    source_ip = event.get('sourceIp', '')

    try:
        secret = sm.get_secret_value(SecretId=f"SFTP/{username}")
        user_config = json.loads(secret['SecretString'])
    except sm.exceptions.ResourceNotFoundException:
        return {}  # Unknown user

    # IP allowlist check before credential verification
    allowed_cidrs = user_config.get('allowed_cidrs', [])
    if allowed_cidrs and not is_ip_allowed(source_ip, allowed_cidrs):
        print(f"IP {source_ip} blocked for user {username}")
        return {}

    is_key_auth = not password
    if is_key_auth:
        return build_response(username, user_config)

    # Password verification: PBKDF2 with timing-safe comparison
    stored_hash = user_config.get('password_hash', '')
    salt = user_config.get('salt', '').encode()
    input_hash = hashlib.pbkdf2_hmac(
        'sha256', password.encode(), salt, 100_000
    ).hex()

    if not hmac.compare_digest(input_hash, stored_hash):
        return {}  # Wrong password

    return build_response(username, user_config)

IP-based access control at authentication time

The sourceIp field in the Lambda event lets you enforce per-user IP allowlists at authentication time. This complements, but does not replace, security group rules on the VPC endpoint. Security groups block TCP connections before the Lambda is invoked — they provide coarse network-level filtering. The Lambda IP check runs after the TCP handshake and provides fine-grained per-user policy: Alice can only connect from her company's datacenter CIDR; Bob can connect from anywhere. The distinction matters for audit logs — a security group block leaves no application-level trace, while a Lambda denial produces a log entry with the username and blocked IP.

import ipaddress

def is_ip_allowed(source_ip, allowed_cidrs):
    try:
        client_ip = ipaddress.ip_address(source_ip)
        return any(
            client_ip in ipaddress.ip_network(cidr, strict=False)
            for cidr in allowed_cidrs
        )
    except ValueError:
        return False  # Unparseable IP — deny

Store the per-user allowed_cidrs list in the Secrets Manager secret alongside the password hash. When a user's office IP range changes, update the secret — no Lambda redeployment needed. For users with no IP restriction, store an empty list [] and the Lambda skips the check, preserving the opt-in-only model.

Pattern 3 — Production Observability

The logging role requirement: zero metrics without it

Transfer Family observability is entirely opt-in. A server created without a --logging-role produces no CloudWatch metrics, no structured session logs, and no authentication event records. The gap is invisible — the server operates normally from the client's perspective, but your observability stack sees nothing. Attach the logging role at server creation time; add it to an existing server with update-server. Metrics only appear after the role is attached — there is no backfill of historical data.

# IAM logging role — Transfer Family service assumes this to write logs
# Trust policy
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"Service": "transfer.amazonaws.com"},
    "Action": "sts:AssumeRole"
  }]
}

# Permission policy — scoped to Transfer Family log groups
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": [
      "logs:CreateLogGroup",
      "logs:CreateLogStream",
      "logs:DescribeLogStreams",
      "logs:PutLogEvents"
    ],
    "Resource": "arn:aws:logs:*:*:log-group:/aws/transfer/*"
  }]
}

# Attach logging role to existing server
aws transfer update-server \
  --server-id s-0abc \
  --logging-role arn:aws:iam::123456789012:role/TransferLoggingRole

# Verify
aws transfer describe-server \
  --server-id s-0abc \
  --query 'Server.LoggingRole'

Transfer Family creates the CloudWatch log group /aws/transfer/<server-id> automatically on first transfer. Each SFTP session creates a separate log stream named after the session ID. Log entries are structured JSON containing: username, source_ip, session_id, protocol, action (AUTH, OPEN, CLOSE, READ, WRITE), remoteFilename, fileSize, outcome (SUCCESS / FAILED), and duration in milliseconds.

CloudWatch metrics and the TreatMissingData: breaching pattern

Transfer Family emits four metrics in the AWS/Transfer namespace: FilesIn (upload count), FilesOut (download count), BytesIn (upload bytes), BytesOut (download bytes). Metrics have 1-minute granularity and are available by ServerId (aggregate across all users) or by Username (per-user). The critical behavior: if no files are transferred in a period, no datapoint is emitted for that period. CloudWatch treats absent datapoints as "no data" by default.

# FilesIn metric for specific user over the last 24 hours
aws cloudwatch get-metric-statistics \
  --namespace AWS/Transfer \
  --metric-name FilesIn \
  --dimensions \
    Name=ServerId,Value=s-0abc \
    Name=Username,Value=alice \
  --start-time $(date -u -d '24 hours ago' +%Y-%m-%dT%H:%M:%SZ) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%SZ) \
  --period 3600 \
  --statistics Sum

# SLA alarm: alert if alice uploads ZERO files between 08:00-09:00 UTC
# TreatMissingData: breaching = missing datapoint counts as alarm condition
aws cloudwatch put-metric-alarm \
  --alarm-name transfer-alice-daily-upload-missing \
  --alarm-description "Alice daily file upload SLA breach — no upload detected" \
  --namespace AWS/Transfer \
  --metric-name FilesIn \
  --dimensions \
    Name=ServerId,Value=s-0abc \
    Name=Username,Value=alice \
  --statistic Sum \
  --period 3600 \
  --evaluation-periods 1 \
  --threshold 1 \
  --comparison-operator LessThanThreshold \
  --treat-missing-data breaching \
  --alarm-actions arn:aws:sns:us-east-1:123456789012:sftp-ops-alerts

# Exfiltration alarm: alert on unusually high download volume
# TreatMissingData: notBreaching = no download activity is normal and fine
aws cloudwatch put-metric-alarm \
  --alarm-name transfer-alice-high-bytesout \
  --namespace AWS/Transfer \
  --metric-name BytesOut \
  --dimensions \
    Name=ServerId,Value=s-0abc \
    Name=Username,Value=alice \
  --statistic Sum \
  --period 3600 \
  --evaluation-periods 1 \
  --threshold 104857600 \
  --comparison-operator GreaterThanThreshold \
  --treat-missing-data notBreaching \
  --alarm-actions arn:aws:sns:us-east-1:123456789012:sftp-security-alerts

The TreatMissingData choice encodes the semantics of the metric. For SLA alarms — "partner should have uploaded by 09:00 UTC" — use breaching: the absence of uploads is itself the failure condition you're monitoring. For anomaly detection alarms — "alert on unusually high download volume" — use notBreaching: no download activity is normal (Alice doesn't download anything most days). Never use the default missing behavior for SLA monitoring: it holds the alarm in its previous state, which means if the alarm was previously OK (Alice uploaded yesterday) and today's upload is missing, the alarm stays OK — silent failure.

EventBridge for downstream MCP processing triggers

Transfer Family publishes three event types to the default EventBridge bus automatically: File Transfer Complete, File Transfer Failed, and Session Terminated. No server configuration is required to enable these events — they fire for every transfer on every Transfer Family server in the account. The File Transfer Complete event includes the S3 bucket, key, version ID, transferred bytes, username, and session ID — everything a downstream MCP processing Lambda needs to read and process the uploaded file without parsing the S3 key path for context.

# EventBridge rule — trigger MCP processing Lambda on each completed upload
aws events put-rule \
  --name transfer-file-complete \
  --event-pattern '{
    "source": ["aws.transfer"],
    "detail-type": ["File Transfer Complete"],
    "detail": {
      "ServerId": ["s-0abc"],
      "StatusCode": ["200"]
    }
  }'

aws events put-targets \
  --rule transfer-file-complete \
  --targets '[{
    "Id": "mcp-processor",
    "Arn": "arn:aws:lambda:us-east-1:123456789012:function:mcp-process-upload"
  }]'

# EventBridge event structure (reference):
# {
#   "source": "aws.transfer",
#   "detail-type": "File Transfer Complete",
#   "detail": {
#     "ServerId": "s-0abc",
#     "Username": "alice",
#     "SessionId": "sftp-sess-uuid",
#     "FileLocation": {
#       "S3Bucket": "mcp-file-intake",
#       "S3Key": "alice/reports/q1.csv",
#       "S3VersionId": "v123"
#     },
#     "BytesTransferred": 45678,
#     "StatusCode": "200"
#   }
# }

The key advantage of EventBridge over S3 event notifications for Transfer Family is user context: the Username and SessionId fields arrive with the event, so your processing Lambda can route based on user identity without parsing the S3 key. Use S3 event notifications when you need the lowest possible trigger latency (milliseconds) and the key path structure is sufficient context. Use EventBridge when user context matters or when you want to filter events by ServerId or Username at the event bus level before any Lambda invocation.

Logs Insights for authentication failure forensics

Transfer Family session logs in CloudWatch Logs support Logs Insights queries for debugging authentication failures, auditing upload history, and identifying slow transfers. Log group name: /aws/transfer/<server-id>. Key fields: username, source_ip, action (AUTH, OPEN, CLOSE, READ, WRITE), outcome (SUCCESS / FAILED), duration (milliseconds), remoteFilename, fileSize.

# Authentication failures in the last 24 hours, grouped by user and source IP
fields @timestamp, username, source_ip, outcome
| filter @logGroup = '/aws/transfer/s-0abc'
| filter action = 'AUTH'
| filter outcome = 'FAILED'
| stats count() as failure_count by username, source_ip
| sort failure_count desc
| limit 20

# All uploads by alice in the last 7 days
fields @timestamp, username, remoteFilename, fileSize
| filter @logGroup = '/aws/transfer/s-0abc'
| filter username = 'alice'
| filter action = 'CLOSE'
| filter outcome = 'SUCCESS'
| sort @timestamp desc

# Slow transfers — uploads taking longer than 60 seconds
fields @timestamp, username, remoteFilename, fileSize, duration
| filter @logGroup = '/aws/transfer/s-0abc'
| filter action = 'CLOSE'
| filter duration > 60000
| stats avg(duration) as avg_ms, max(duration) as max_ms,
        avg(fileSize) as avg_bytes by username
| sort avg_ms desc

For ongoing auth failure rate monitoring, create a CloudWatch Logs metric filter on the log group that counts outcome = 'FAILED' entries and publishes a custom metric. Then alarm on that metric — this gives you a CloudWatch alarm on authentication failure spikes without manual Logs Insights queries. The metric filter runs automatically as new log data arrives; you don't need to schedule queries.

One gotcha with BytesIn: the CloudWatch metric counts bytes written during an upload attempt, including bytes from failed partial uploads that were rolled back. If a partner's upload fails halfway through a large file, BytesIn counts those bytes even though the S3 object was never created. Use the EventBridge File Transfer Failed events to track failed transfers separately and subtract from BytesIn for accurate ingestion accounting.

Consolidated failure modes

Production checklists

Server creation checklist

• Chose endpoint type (PUBLIC or VPC) — this is immutable after creation
• Chose identity provider type (SERVICE_MANAGED or AWS_LAMBDA) — immutable after creation
• Logging role attached at server creation time (not as an afterthought)
• Security group (VPC endpoint) allows inbound port 22 from partner CIDRs; port 21 + 49152–65535 if FTPS
• For VPC with Elastic IPs: AddressAllocationIds count matches SubnetIds count exactly

S3 integration checklist

• S3 bucket ObjectOwnership set to BucketOwnerEnforced
• All public access blocked on the bucket
• No bucket policy Allow * for transfer.amazonaws.com principal
• Session policy attached at create-user time for every user — not just the IAM role
• HomeDirectoryType LOGICAL with HomeDirectoryMappings hiding bucket name and parent prefix
• S3 lifecycle policy configured (STANDARD → IA at 30d → Glacier IR at 90d → expire at 365d)
• NoncurrentVersionExpiration rule in lifecycle to prevent unbounded version accumulation

Custom identity provider checklist

• Lambda resource policy grants lambda:InvokeFunction to transfer.amazonaws.com with server ARN as source-arn
• HomeDirectoryDetails returned as json.dumps(list) string — not a Python list
• Policy returned as json.dumps(dict) string — not a Python dict
• Auth denial returns {} empty dict — not None, not HTTP 4xx
• SSH key auth returns PublicKeys in OpenSSH format (ssh-rsa AAAA... or ssh-ed25519 AAAA...)
• Verified with aws transfer test-identity-provider before SFTP client testing

Observability checklist

• SLA alarms use TreatMissingData: breaching (zero uploads = missing datapoint = alarm)
• Anomaly alarms use TreatMissingData: notBreaching (no unusual activity is normal)
• EventBridge File Transfer Complete rule targets processing Lambda (default bus, not custom bus)
• Log group /aws/transfer/<server-id> exists and contains session streams after first transfer
• Logs Insights auth failure query validated against known good/bad authentication attempts