MCP Server Lake Formation Governed Tables — StartTransactionCommand, UpdateTableObjects

What governed tables are

A governed table is a Glue table with IsRegisteredWithLakeFormation: true (set during table creation via Glue). Lake Formation tracks the S3 object manifest for governed tables — which .parquet / .orc / .avro files are part of the table, what their ETag is, and what partition they belong to. This manifest is the source of truth instead of an S3 prefix listing.

Governed tables work with Iceberg-compatible query engines (Athena v3, Glue ETL, Spark on EMR). The Lake Formation API is the manifest layer — the actual data is still stored in S3; Lake Formation just tracks which files are part of the table.

Starting transactions

StartTransactionCommand creates a new transaction and returns a TransactionId. Use READ_AND_WRITE for write transactions (the only type that can call UpdateTableObjectsCommand). Use READ_ONLY for read transactions — this provides a stable snapshot ID for consistent multi-table reads in query engines, but does not allow writing.

import {
  LakeFormationClient,
  StartTransactionCommand,
  CommitTransactionCommand,
  CancelTransactionCommand,
  ExtendTransactionCommand,
  GetTransactionCommand,
  ListTransactionsCommand,
} from '@aws-sdk/client-lakeformation';

const lakeFormation = new LakeFormationClient({ region: process.env.AWS_REGION ?? 'us-east-1' });

export async function startWriteTransaction(): Promise<string> {
  const response = await lakeFormation.send(new StartTransactionCommand({
    TransactionType: 'READ_AND_WRITE',
  }));
  return response.TransactionId!;
}

export async function startReadTransaction(): Promise<string> {
  const response = await lakeFormation.send(new StartTransactionCommand({
    TransactionType: 'READ_ONLY',
  }));
  return response.TransactionId!;
}

Transactions have a default TTL of 10 hours. If you have a long-running write (e.g., an MCP tool that processes data in batches), call ExtendTransactionCommand before the timeout. Each extension resets the TTL to 10 hours from the extension time.

export async function extendTransaction(transactionId: string): Promise<void> {
  await lakeFormation.send(new ExtendTransactionCommand({ TransactionId: transactionId }));
}

Adding S3 objects in a transaction

UpdateTableObjectsCommand registers new S3 objects into a governed table (or removes existing ones) within an open transaction. The Uri must be an S3 URI matching the table's registered location. The ETag and Size fields are required for AddObject — Lake Formation uses them to verify the file exists and was not modified between the write and the commit.

import {
  UpdateTableObjectsCommand,
  type WriteOperation,
} from '@aws-sdk/client-lakeformation';

export async function addTableObjects(
  transactionId: string,
  databaseName: string,
  tableName: string,
  objects: Array<{
    s3Uri: string;    // e.g. 's3://my-lake/events/dt=2026-11-06/part-00000.parquet'
    etag: string;     // S3 ETag from the PutObject response
    size: number;     // file size in bytes from S3 HeadObject or PutObject response
    partitionValues?: string[];  // e.g. ['2026-11-06'] for a date partition
  }>,
): Promise<void> {
  const writeOperations: WriteOperation[] = objects.map(obj => ({
    AddObject: {
      Uri: obj.s3Uri,
      ETag: obj.etag,
      Size: obj.size,
      // PartitionValues: ['2026-11-06'] — required if the table is partitioned
      // Order must match the table's partition key order in the Glue schema
      PartitionValues: obj.partitionValues,
    },
  }));

  await lakeFormation.send(new UpdateTableObjectsCommand({
    TransactionId: transactionId,
    DatabaseName: databaseName,
    TableName: tableName,
    WriteOperations: writeOperations,
  }));
}

The ETag must exactly match the S3 object's ETag. For multipart uploads, the ETag format is "hex-etag-N" where N is the number of parts — the format differs from single-part PutObject. Lake Formation will reject the commit with ResourceNotReadyException if the ETag does not match the actual S3 object at commit time.

A complete write workflow — upload the file, register it, commit:

import { S3Client, PutObjectCommand, HeadObjectCommand } from '@aws-sdk/client-s3';

const s3 = new S3Client({ region: process.env.AWS_REGION ?? 'us-east-1' });

export async function appendParquetToTable(
  databaseName: string,
  tableName: string,
  bucket: string,
  key: string,
  parquetBuffer: Buffer,
  partitionValues?: string[],
): Promise<void> {
  // Step 1: upload the file to S3
  const putResponse = await s3.send(new PutObjectCommand({
    Bucket: bucket,
    Key: key,
    Body: parquetBuffer,
    ContentType: 'application/octet-stream',
  }));
  const etag = putResponse.ETag!.replace(/"/g, '');  // strip surrounding quotes
  const size = parquetBuffer.length;

  // Step 2: start a Lake Formation transaction
  const txId = await startWriteTransaction();

  try {
    // Step 3: register the file in the transaction
    await addTableObjects(txId, databaseName, tableName, [{
      s3Uri: `s3://${bucket}/${key}`,
      etag,
      size,
      partitionValues,
    }]);

    // Step 4: commit — file becomes visible to readers only after this call
    await lakeFormation.send(new CommitTransactionCommand({ TransactionId: txId }));

  } catch (error) {
    // If anything fails, cancel the transaction — the S3 file is still there
    // but Lake Formation won't expose it to readers (orphaned file)
    await lakeFormation.send(new CancelTransactionCommand({ TransactionId: txId }));
    throw error;
  }
}

Canceling a transaction does not delete the S3 files you uploaded — it only tells Lake Formation not to include them in the table manifest. You are responsible for cleaning up orphaned S3 objects, typically via an S3 lifecycle rule on the table prefix or a separate cleanup job.

Removing objects in a transaction

Use DeleteObject in WriteOperations to remove files from the table manifest. This marks the objects as deleted in the Lake Formation manifest but does not delete the S3 objects themselves — delete the S3 objects separately after the transaction commits if you no longer need them.

import { UpdateTableObjectsCommand } from '@aws-sdk/client-lakeformation';

export async function removeTableObjects(
  transactionId: string,
  databaseName: string,
  tableName: string,
  objects: Array<{ s3Uri: string; etag: string; partitionValues?: string[] }>,
): Promise<void> {
  await lakeFormation.send(new UpdateTableObjectsCommand({
    TransactionId: transactionId,
    DatabaseName: databaseName,
    TableName: tableName,
    WriteOperations: objects.map(obj => ({
      DeleteObject: {
        Uri: obj.s3Uri,
        ETag: obj.etag,  // required for delete too — prevents removing wrong version
        PartitionValues: obj.partitionValues,
      },
    })),
  }));
}

Reading the table object manifest

GetTableObjectsCommand returns the list of S3 objects that constitute the table at a given transaction or point in time. Use it to build a file list for a custom query engine, to verify what files were committed, or to implement time-travel reads.

import {
  GetTableObjectsCommand,
  type GetTableObjectsCommandInput,
} from '@aws-sdk/client-lakeformation';

// Get current objects (latest committed state)
export async function getTableObjects(
  databaseName: string,
  tableName: string,
  partitionPredicate?: string,  // e.g. 'dt = \'2026-11-06\''
) {
  const objects = [];
  let nextToken: string | undefined;

  do {
    const input: GetTableObjectsCommandInput = {
      DatabaseName: databaseName,
      TableName: tableName,
      PartitionPredicate: partitionPredicate,  // optional SQL predicate for partition pruning
      NextToken: nextToken,
    };

    const response = await lakeFormation.send(new GetTableObjectsCommand(input));

    for (const partitionObj of response.Objects ?? []) {
      objects.push(...(partitionObj.Objects ?? []).map(obj => ({
        uri: obj.Uri,
        etag: obj.ETag,
        size: obj.Size,
        partitionValues: partitionObj.PartitionValues,
      })));
    }

    nextToken = response.NextToken;
  } while (nextToken);

  return objects;
}

// Time-travel read: get objects as of a specific point in time
export async function getTableObjectsAtTime(
  databaseName: string,
  tableName: string,
  asOf: Date,
) {
  const objects = [];
  let nextToken: string | undefined;

  do {
    const response = await lakeFormation.send(new GetTableObjectsCommand({
      DatabaseName: databaseName,
      TableName: tableName,
      QueryAsOfTime: asOf,  // returns object manifest as of this timestamp
      NextToken: nextToken,
    }));

    for (const partitionObj of response.Objects ?? []) {
      objects.push(...(partitionObj.Objects ?? []));
    }

    nextToken = response.NextToken;
  } while (nextToken);

  return objects;
}

The QueryAsOfTime parameter enables point-in-time snapshot reads. This is useful for auditing (what was in the table at the time of a report run?), re-processing (re-run a job against yesterday's snapshot), or debugging (compare object lists before and after a problematic write). Lake Formation retains transaction history for 7 days by default — queries against timestamps older than the retention window return an error.

Committing and canceling transactions

import {
  CommitTransactionCommand,
  CancelTransactionCommand,
  GetTransactionCommand,
  ListTransactionsCommand,
} from '@aws-sdk/client-lakeformation';

export async function commitTransaction(transactionId: string): Promise<void> {
  const response = await lakeFormation.send(
    new CommitTransactionCommand({ TransactionId: transactionId })
  );
  // response.TransactionStatus will be 'committed' on success
  if (response.TransactionStatus !== 'committed') {
    throw new Error(`Unexpected commit status: ${response.TransactionStatus}`);
  }
}

export async function cancelTransaction(transactionId: string): Promise<void> {
  await lakeFormation.send(
    new CancelTransactionCommand({ TransactionId: transactionId })
  );
}

export async function getTransactionStatus(transactionId: string) {
  const response = await lakeFormation.send(
    new GetTransactionCommand({ TransactionId: transactionId })
  );
  const tx = response.Transaction;
  return {
    id: tx?.TransactionId,
    status: tx?.TransactionStatus,
    startedAt: tx?.TransactionStartTime,
    endedAt: tx?.TransactionEndTime,
  };
}

// List active transactions (useful for detecting stuck transactions)
export async function listActiveTransactions() {
  const transactions = [];
  let nextToken: string | undefined;

  do {
    const response = await lakeFormation.send(new ListTransactionsCommand({
      StatusFilter: 'ACTIVE',  // 'ACTIVE' | 'COMMITTED' | 'ABORTED' | 'ALL'
      NextToken: nextToken,
    }));
    transactions.push(...(response.Transactions ?? []));
    nextToken = response.NextToken;
  } while (nextToken);

  return transactions;
}

Transaction state machine

Lake Formation uses optimistic concurrency control during commit. If two transactions both modified the same table objects, the first to commit wins and the second receives TransactionCanceledException with the message "Write conflict". Your MCP tool should catch this, cancel the failed transaction, and retry the full write sequence (re-upload if needed, start new transaction, re-register objects, commit again).

IAM reference for governed table operations

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "LakeFormationTransactions",
      "Effect": "Allow",
      "Action": [
        "lakeformation:StartTransaction",
        "lakeformation:CommitTransaction",
        "lakeformation:CancelTransaction",
        "lakeformation:ExtendTransaction",
        "lakeformation:GetTransaction",
        "lakeformation:ListTransactions"
      ],
      "Resource": "*"
    },
    {
      "Sid": "LakeFormationTableObjects",
      "Effect": "Allow",
      "Action": [
        "lakeformation:UpdateTableObjects",
        "lakeformation:GetTableObjects"
      ],
      "Resource": "*"
    },
    {
      "Sid": "S3DataLakeAccess",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::my-lake-bucket",
        "arn:aws:s3:::my-lake-bucket/*"
      ]
    }
  ]
}

The MCP server role also needs Lake Formation data-plane permissions on the specific table: either a SELECT + INSERT grant via GrantPermissionsCommand, or the role must be a data lake admin. The IAM policy above is insufficient alone — both IAM and Lake Formation permissions are required.

Error handling reference

Related pages

• Lake Formation overview — RegisterResourceCommand, data lake settings — registering S3 resources and configuring data lake admins
• Lake Formation permissions — GrantPermissionsCommand, column security — granting SELECT/INSERT on governed tables
• Lake Formation LF-tags — TBAC, CreateLFTagCommand, tag expressions — tag-based access control
• AWS Glue Data Catalog for MCP servers
• Athena Iceberg tables for MCP servers