MCP Server EMR Studio — CreateStudio, CreateEditor, SparkMagic notebooks

Studio and workspace concepts

EMR Studio has three layers: the Studio (organizational boundary — one per team or environment), the Editor/Workspace (a JupyterLab server instance that persists notebooks to S3), and the Cluster or Serverless Application (the actual compute that runs Spark code when you execute a notebook cell).

Workspaces are lightweight JupyterLab servers (ml.t3.medium by default). The workspace handles notebook storage and kernel management, but all Spark execution happens on the attached cluster or Serverless application. A workspace can be detached from one cluster and reattached to another without losing notebook state.

Creating a studio with CreateStudioCommand

A studio requires two security groups (workspace SG for the JupyterLab ENI, engine SG for the cluster ENI) and an S3 bucket for notebook persistence. The AuthMode determines how users authenticate: SSO for IAM Identity Center (recommended for teams), IAM for direct IAM authentication (useful for automated MCP server use).

import {
  EMRClient,
  CreateStudioCommand,
  ListStudiosCommand,
  GetStudioCommand,
} from '@aws-sdk/client-emr';

const emr = new EMRClient({ region: process.env.AWS_REGION ?? 'us-east-1' });

export async function createStudio(params: {
  name: string;
  vpcId: string;
  subnetIds: string[];          // subnets where workspace ENIs will be placed
  workspaceSecurityGroupId: string;
  engineSecurityGroupId: string;
  defaultS3Location: string;   // s3://bucket/emr-studio/ — notebook storage
  serviceRoleArn: string;       // IAM role for Studio to access S3 and EMR
  userRoleArn?: string;         // per-user role for SSO auth
  idcInstanceArn?: string;      // IAM Identity Center instance ARN for SSO
}): Promise<string> {
  const response = await emr.send(new CreateStudioCommand({
    Name: params.name,
    AuthMode: params.idcInstanceArn ? 'SSO' : 'IAM',
    VpcId: params.vpcId,
    SubnetIds: params.subnetIds,
    WorkspaceSecurityGroupId: params.workspaceSecurityGroupId,
    EngineSecurityGroupId: params.engineSecurityGroupId,
    DefaultS3Location: params.defaultS3Location,
    ServiceRole: params.serviceRoleArn,
    UserRole: params.userRoleArn,
    IdcInstanceArn: params.idcInstanceArn,
    Tags: [{ Key: 'ManagedBy', Value: 'alivemcp' }],
  }));

  return response.StudioId!;
}

export async function listStudios(): Promise<Array<{ id: string; name: string; url: string }>> {
  const studios = [];
  let marker: string | undefined;

  do {
    const r = await emr.send(new ListStudiosCommand({ Marker: marker }));
    for (const s of r.Studios ?? []) {
      studios.push({ id: s.StudioId!, name: s.Name!, url: s.Url! });
    }
    marker = r.Marker;
  } while (marker);

  return studios;
}

The workspace security group must allow HTTPS outbound to the engine security group on port 18888 (SparkMagic uses this for the Livy REST API). The engine security group must allow inbound TCP on port 18888 from the workspace security group. EMR creates the actual ENIs when a workspace is started and an editor is attached — these security groups govern the network path between the JupyterLab server and the Spark cluster.

Creating and starting a workspace editor

CreateEditorCommand provisions a workspace (JupyterLab server). The workspace starts in STARTING state and transitions to READY once the JupyterLab server is up. Use GetEditorCommand to poll state.

import {
  CreateEditorCommand,
  StartEditorCommand,
  GetEditorCommand,
  ListEditorsCommand,
  DeleteEditorCommand,
  type EditorState,
} from '@aws-sdk/client-emr';

export async function createWorkspace(params: {
  studioId: string;
  name: string;
  subnetId: string;
}): Promise<string> {
  const response = await emr.send(new CreateEditorCommand({
    Name: params.name,
    StudioId: params.studioId,
    SubnetId: params.subnetId,
  }));

  return response.EditorId!;
}

export async function startWorkspace(
  editorId: string,
  clusterId?: string,          // optional: attach to a cluster on start
  applicationId?: string,      // optional: attach to Serverless app on start
): Promise<void> {
  await emr.send(new StartEditorCommand({
    EditorId: editorId,
    ClusterId: clusterId,
    ApplicationId: applicationId,
  }));

  // Wait for READY
  const deadline = Date.now() + 5 * 60 * 1000;
  while (Date.now() < deadline) {
    const r = await emr.send(new GetEditorCommand({ EditorId: editorId }));
    const state = r.Editor?.Status?.State as EditorState;
    if (state === 'READY') return;
    if (state === 'FAILED') {
      throw new Error(`Workspace ${editorId} failed to start: ${r.Editor?.Status?.StateChangeReason?.Message}`);
    }
    await new Promise(res => setTimeout(res, 10_000));
  }
  throw new Error(`Workspace ${editorId} did not reach READY within 5 minutes`);
}

export async function listWorkspaces(studioId: string) {
  const editors = [];
  let marker: string | undefined;

  do {
    const r = await emr.send(new ListEditorsCommand({ StudioId: studioId, Marker: marker }));
    editors.push(...(r.Editors ?? []));
    marker = r.Marker;
  } while (marker);

  return editors.map(e => ({
    id: e.EditorId!,
    name: e.Name!,
    state: e.Status?.State,
    url: e.EditorArn,
    lastModified: e.LastModifiedTime,
  }));
}

Attaching and detaching from clusters

An editor can be attached to an EMR cluster (EC2-backed) or an EMR Serverless application for Spark execution. Use AttachEditorCommand to link the workspace to compute, and DetachEditorCommand to unlink. A workspace can only be attached to one compute target at a time.

import {
  AttachEditorCommand,
  DetachEditorCommand,
} from '@aws-sdk/client-emr';

// Attach to an existing EMR cluster
export async function attachToCluster(
  editorId: string,
  clusterId: string,
): Promise<void> {
  await emr.send(new AttachEditorCommand({
    EditorId: editorId,
    ClusterId: clusterId,
  }));
}

// Attach to an EMR Serverless application
export async function attachToServerless(
  editorId: string,
  applicationId: string,
): Promise<void> {
  await emr.send(new AttachEditorCommand({
    EditorId: editorId,
    ApplicationId: applicationId,
  }));
}

export async function detachFromCompute(editorId: string): Promise<void> {
  await emr.send(new DetachEditorCommand({ EditorId: editorId }));
}

When a workspace is attached to a cluster, the SparkMagic kernel inside JupyterLab connects to the cluster's Apache Livy server on port 18888. Notebook cells using the %%spark, %%pyspark, or %%sparkr magic commands execute on the cluster via Livy REST API calls. The workspace itself (JupyterLab server) is a lightweight proxy — it does not run Spark code directly.

SparkMagic kernel selection

EMR Studio uses SparkMagic, a set of Jupyter kernels that communicate with Spark clusters via Apache Livy. Each kernel variant runs code in a different language on the attached cluster.

Switching kernels within the same workspace starts a new Livy session. The old session's data (cached DataFrames, defined variables) is not transferred — the new kernel starts fresh. For MCP tools that programmatically create notebooks, default to PySpark unless the user specifies otherwise.

Git repository integration

EMR Studio supports linking a Git repository so notebooks are version-controlled. Linking is done at the workspace level and uses a Secrets Manager secret for credentials.

import {
  CreateRepositoryCommand,
  LinkRepositoryCommand,
  ListRepositoriesCommand,
} from '@aws-sdk/client-emr';

// First: create a repository record (references the Git remote)
export async function createRepository(params: {
  name: string;
  gitUrl: string;        // https://github.com/org/repo
  gitBranch: string;
  credentialSecretArn: string;  // Secrets Manager secret with username/password or token
}): Promise<string> {
  const response = await emr.send(new CreateRepositoryCommand({
    Name: params.name,
    GitRepository: {
      Url: params.gitUrl,
      Branch: params.gitBranch,
      RepositoryAuthConfiguration: {
        SecretArn: params.credentialSecretArn,
      },
    },
  }));

  return response.Repository?.Id!;
}

// Second: link the repository to a workspace editor
export async function linkRepositoryToWorkspace(
  editorId: string,
  repositoryId: string,
): Promise<void> {
  await emr.send(new LinkRepositoryCommand({
    EditorId: editorId,
    RepositoryId: repositoryId,
  }));
}

export async function listRepositories() {
  const repos = [];
  let marker: string | undefined;

  do {
    const r = await emr.send(new ListRepositoriesCommand({ Marker: marker }));
    repos.push(...(r.Repositories ?? []));
    marker = r.Marker;
  } while (marker);

  return repos.map(r => ({
    id: r.Id!,
    name: r.Name!,
    url: r.GitRepository?.Url,
    branch: r.GitRepository?.Branch,
    status: r.RepositoryStatus,
  }));
}

The Secrets Manager secret must contain a JSON object with username and password (or a personal access token as the password). For GitHub, use a fine-grained personal access token with repo:read and repo:write scopes, stored as {"username": "git", "password": "github_pat_xxx"}. The secret must be in the same AWS region as the EMR Studio.

Workspace cost management

Workspace instances are billed per hour from creation to deletion, regardless of whether they are attached to a cluster. For MCP tools that provision workspaces on-demand, delete workspaces when they are no longer needed.

import { DeleteEditorCommand, StopEditorCommand } from '@aws-sdk/client-emr';

// Stop a workspace (suspends billing, preserves notebooks in S3)
export async function stopWorkspace(editorId: string): Promise<void> {
  await emr.send(new StopEditorCommand({ EditorId: editorId }));
}

// Delete a workspace permanently
export async function deleteWorkspace(editorId: string): Promise<void> {
  await emr.send(new DeleteEditorCommand({ EditorId: editorId }));
  // Note: notebooks persisted to S3 are NOT deleted — only the workspace instance
}

StopEditorCommand stops the JupyterLab server (ending workspace-hour billing) while preserving all notebooks in the configured S3 bucket. StartEditorCommand restores it, loading notebooks from S3. DeleteEditorCommand removes the workspace record from EMR Studio but does not delete the S3 objects — you must clean those up manually if needed.

Related pages

• EMR cluster management — RunJobFlow, AddJobFlowSteps, termination protection — EC2-backed EMR clusters for batch jobs
• EMR Serverless — CreateApplication, StartJobRun, pre-initialized capacity — serverless Spark without cluster management
• EMR Spark patterns — spark-submit, S3A, Glue metastore, dynamic allocation — Spark configuration for EMR
• SageMaker endpoints for MCP servers
• AWS Glue Data Catalog for MCP servers