MCP Server SFTP-to-S3 Integration — Bucket Policy, Session Policy, and Directory Isolation

S3 bucket setup for Transfer Family

The S3 bucket that backs your Transfer Family server needs two configuration decisions: public access blocking and object ownership. Public access must be fully blocked — Transfer Family accesses S3 through IAM, not presigned URLs, and a bucket with any public access is a security risk for file-intake workloads. Object ownership should be set to BucketOwnerEnforced, which disables ACLs and ensures the bucket owner account always owns every object — critical for preventing ACL-based data exfiltration from uploaded files.

# Create and configure the S3 bucket
aws s3api create-bucket \
  --bucket mcp-file-intake \
  --region us-east-1

# Block all public access
aws s3api put-public-access-block \
  --bucket mcp-file-intake \
  --public-access-block-configuration \
    BlockPublicAcls=true,\
    IgnorePublicAcls=true,\
    BlockPublicPolicy=true,\
    RestrictPublicBuckets=true

# Set BucketOwnerEnforced — disables ACLs, bucket owner owns all objects
aws s3api put-bucket-ownership-controls \
  --bucket mcp-file-intake \
  --ownership-controls '{"Rules": [{"ObjectOwnership": "BucketOwnerEnforced"}]}'

# Enable versioning (recommended for intake buckets — allows recovery of overwritten files)
aws s3api put-bucket-versioning \
  --bucket mcp-file-intake \
  --versioning-configuration Status=Enabled

# Enable server-side encryption (SSE-S3 default, or use SSE-KMS)
aws s3api put-bucket-encryption \
  --bucket mcp-file-intake \
  --server-side-encryption-configuration '{
    "Rules": [{
      "ApplyServerSideEncryptionByDefault": {
        "SSEAlgorithm": "AES256"
      },
      "BucketKeyEnabled": true
    }]
  }'

No bucket policy is needed for Transfer Family access — the IAM role attached to each user provides the access grant. If you do add a bucket policy (for example, to enforce TLS or deny non-Transfer-Family access), be careful not to add an Allow * statement for transfer.amazonaws.com as the principal — that would grant every Transfer Family server in your account access to the bucket.

IAM role and session policy for tenant isolation

The Transfer Family user role grants broad S3 access to the bucket. The session policy attached at user-creation time is the per-user restriction that limits the role's effective permissions to a single prefix. AWS STS intersects the role policy and the session policy — the effective permission is the set of actions that BOTH policies allow. A user with a session policy scoped to bucket/alice/* cannot read bucket/bob/* even though the role grants access to all prefixes.

# IAM role permission policy — broad access to the bucket
# (session policy will restrict this per user)
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListBucket",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetBucketLocation"
      ],
      "Resource": "arn:aws:s3:::mcp-file-intake",
      "Condition": {
        "StringLike": {
          "s3:prefix": ["${transfer:UserName}/*", "${transfer:UserName}"]
        }
      }
    },
    {
      "Sid": "ObjectAccess",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:GetObjectVersion"
      ],
      "Resource": "arn:aws:s3:::mcp-file-intake/${transfer:UserName}/*"
    }
  ]
}

# Per-user session policy (passed as --policy at create-user time)
# Replace ${transfer:UserName} with the actual username (alice, bob, etc.)
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowUserPrefix",
      "Effect": "Allow",
      "Action": ["s3:*"],
      "Resource": [
        "arn:aws:s3:::mcp-file-intake",
        "arn:aws:s3:::mcp-file-intake/alice",
        "arn:aws:s3:::mcp-file-intake/alice/*"
      ]
    }
  ]
}

The ${transfer:UserName} policy variable in the role policy is a Transfer Family-specific IAM condition variable that resolves to the authenticated username at runtime. Using it in the role policy means you can create one role for all users — the role automatically restricts s3:prefix listing to the requesting user's prefix without per-user role creation. However, the session policy passed at user-creation time must still use the literal username (policy variables are not supported in session policies).

LOGICAL vs ABSOLUTE home directory types

The HomeDirectoryType controls what the SFTP user sees as their root directory and how paths map to S3 keys. ABSOLUTE mode sets the home directory to a specific S3 path like /mcp-file-intake/alice — the user sees raw S3 key names relative to that path, and the SFTP root is that prefix. LOGICAL mode lets you define explicit Entry→Target mappings — the user's / can map to /mcp-file-intake/alice, and you can add additional mount points like /shared → /mcp-file-intake/shared-read.

# ABSOLUTE home directory — simple but inflexible
aws transfer create-user \
  --server-id s-0123456789abcdef0 \
  --user-name alice \
  --role arn:aws:iam::123456789012:role/TransferUserRole \
  --home-directory /mcp-file-intake/alice \
  --home-directory-type ABSOLUTE

# With ABSOLUTE, user sees:
# /                    → s3://mcp-file-intake/alice/
# /reports/q1.csv     → s3://mcp-file-intake/alice/reports/q1.csv
# Cannot mount additional paths

# LOGICAL home directory — flexible with explicit mappings
aws transfer create-user \
  --server-id s-0123456789abcdef0 \
  --user-name alice \
  --role arn:aws:iam::123456789012:role/TransferUserRole \
  --home-directory-type LOGICAL \
  --home-directory-mappings '[
    {
      "Entry": "/",
      "Target": "/mcp-file-intake/alice"
    },
    {
      "Entry": "/shared",
      "Target": "/mcp-file-intake/shared-read"
    },
    {
      "Entry": "/outbound",
      "Target": "/mcp-processed-output/alice"
    }
  ]' \
  --policy '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["s3:*"],"Resource":["arn:aws:s3:::mcp-file-intake","arn:aws:s3:::mcp-file-intake/alice/*","arn:aws:s3:::mcp-file-intake/shared-read/*","arn:aws:s3:::mcp-processed-output/alice/*"]}]}'

LOGICAL mode is required when you need to mount multiple S3 paths or map to directories in a second bucket. With LOGICAL mappings, the SFTP client cannot traverse above / — the server rejects any cd .. attempts at the root. Unmapped paths return "No such file" errors. ABSOLUTE mode is simpler and works well when users only need access to a single prefix — no mapping configuration needed, but you lose the ability to add virtual mount points later without recreating the user.

Lifecycle policies and storage management

SFTP intake buckets accumulate files indefinitely unless you add lifecycle policies. For MCP processing pipelines, a common pattern is: keep uploaded files in standard S3 for 7 days (processing window), transition to S3-IA after 30 days (low-access archive), transition to Glacier Instant Retrieval after 90 days, and expire after 1 year. Apply this lifecycle to the user prefix paths, not the entire bucket, to avoid accidentally archiving processed outputs in other prefixes.

# S3 lifecycle policy for SFTP intake prefixes
aws s3api put-bucket-lifecycle-configuration \
  --bucket mcp-file-intake \
  --lifecycle-configuration '{
    "Rules": [
      {
        "ID": "sftp-intake-lifecycle",
        "Status": "Enabled",
        "Filter": { "Prefix": "" },
        "Transitions": [
          {
            "Days": 30,
            "StorageClass": "STANDARD_IA"
          },
          {
            "Days": 90,
            "StorageClass": "GLACIER_IR"
          }
        ],
        "Expiration": { "Days": 365 },
        "NoncurrentVersionExpiration": {
          "NoncurrentDays": 30,
          "NewerNoncurrentVersions": 5
        }
      }
    ]
  }'

# List current objects in a user's prefix (for monitoring)
aws s3 ls s3://mcp-file-intake/alice/ --recursive --human-readable | \
  awk '{ sum += $3 } END { print "Total:", sum, "files" }'

Versioning and lifecycle interact: the NoncurrentVersionExpiration rule removes old versions after 30 days and keeps only the 5 most recent versions. Without this rule, a user who repeatedly uploads the same filename accumulates an unbounded number of object versions. For SFTP intake patterns where partners upload daily batch files with fixed filenames (like orders-2026-10-02.csv), versioning provides a 30-day recovery window without permanent storage growth.

Processing uploaded files with Lambda

The standard pattern for MCP tools that process SFTP uploads is: file lands in S3 → S3 event notification triggers Lambda → Lambda processes the file and writes output. The Lambda function receives the S3 bucket name and object key in the event. For Transfer Family uploads, the key includes the username prefix (alice/reports/q1.csv), which lets the Lambda route processing based on user context.

# Lambda function handler — process SFTP upload
import boto3
import json

s3 = boto3.client('s3')

def handler(event, context):
    for record in event['Records']:
        bucket = record['s3']['bucket']['name']
        key = record['s3']['object']['key']  # e.g., "alice/reports/q1.csv"

        # Extract username from the key path
        parts = key.split('/')
        username = parts[0]  # "alice"
        relative_path = '/'.join(parts[1:])  # "reports/q1.csv"

        # Download and process
        response = s3.get_object(Bucket=bucket, Key=key)
        content = response['Body'].read().decode('utf-8')

        # Process content via your MCP tool logic...
        result = process_file(content, username=username, path=relative_path)

        # Write processed output to a different prefix
        s3.put_object(
            Bucket='mcp-processed-output',
            Key=f'{username}/{relative_path}',
            Body=json.dumps(result).encode('utf-8'),
            ContentType='application/json'
        )

        print(f"Processed {key} for user {username}: {len(content)} bytes")

def process_file(content, username, path):
    # Your MCP tool processing logic here
    return {"username": username, "path": path, "lines": len(content.splitlines())}

The S3 event notification fires within milliseconds of the file being written. For large files, the Lambda may time out if processing takes longer than the function's configured timeout (max 15 minutes). For long-running processing, use the S3 event to enqueue an SQS message and process it in a separate worker or Step Functions state machine. See Step Functions Distributed Map for fan-out processing of large file batches.

Failure modes reference