Getting started with the Workflow CLI Core commands Approval gates Status symbols Migration scenarios Assessment and breaking changes

Use the solution

This section describes how to run a migration using the Migration Assistant for Amazon OpenSearch Service solution after you have deployed it on Amazon EKS. The day-to-day operator interface is the Workflow CLI, which runs in the Migration Console pod (migration-console-0) on Amazon EKS. The supporting console CLI provides component-level inspection and ad-hoc operations during validation and troubleshooting.

Getting started with the Workflow CLI

This sequence is the shortest safe path to your first migration to Amazon OpenSearch Service or Amazon OpenSearch Serverless: load the right schema for your version, prove connectivity, run a small pilot, and only then run the full workflow.

Before you start

Make sure all of the following are true:

Migration Assistant is deployed on Amazon EKS. See Deploy the solution.
The source cluster and the Amazon OpenSearch Service domain or Amazon OpenSearch Serverless collection are reachable from the Amazon EKS cluster.
Snapshot storage is ready in Amazon S3 if you plan to run backfill.
Any basic-auth Kubernetes secrets you need can be created in the ma namespace.

Step 1: Access the Migration Console


kubectl exec -it migration-console-0 -n ma -- /bin/bash

If you are accessing Amazon EKS from a new shell, refresh your kubeconfig first:


aws eks update-kubeconfig --region <REGION> --name migration-eks-cluster-<STAGE>-<REGION>

Step 2: Confirm the installed version


console --version

This matters because the workflow schema can change by release.

Step 3: Load the version-matched sample


workflow configure sample --load

This gives you the safest starting point for your installed release.

Step 4: Edit the workflow configuration


workflow configure edit

Fill in the fields that describe your migration:

Source endpoint, version, and authentication.
Target endpoint and authentication. For Amazon OpenSearch Serverless, set service: aoss in the SigV4 authConfig. For Amazon OpenSearch Service, set service: es.
Snapshot repository details if you are running backfill.
The migration pattern: backfill only, capture and replay only, or both.

Note

Do not start by editing every possible field. Start with the minimum required fields for your path.

Target configuration for Amazon OpenSearch Serverless

When the target is an Amazon OpenSearch Serverless collection, set the target cluster like this:


{
  "targetClusters": {
    "target": {
      "endpoint": "https://<collection-id>.<region>.aoss.amazonaws.com",
      "authConfig": {
        "sigv4": {
          "region": "<region>",
          "service": "aoss"
        }
      }
    }
  }
}

The migration IAM role created by the Amazon EKS deployment must also be added as a principal in your collection’s data access policy. The IAM role is named <eks-cluster-name>-migrations-role. Add it to the collection’s data access policy with both collection-level and index-level permissions before running the workflow.

Target configuration for Amazon OpenSearch Service

When the target is an Amazon OpenSearch Service domain:


{
  "targetClusters": {
    "target": {
      "endpoint": "https://<domain-endpoint>",
      "authConfig": {
        "sigv4": {
          "region": "<region>",
          "service": "es"
        }
      }
    }
  }
}

If your domain has fine-grained access control (FGAC) enabled, map the migration IAM role to a security role on the domain (typically all_access during migration, then scoped down). See Troubleshooting.

Step 5: Create Kubernetes secrets if you use basic authentication


kubectl create secret generic source-credentials \
  --from-literal=username=<SOURCE_USER> \
  --from-literal=password=<SOURCE_PASSWORD> \
  -n ma

kubectl create secret generic target-credentials \
  --from-literal=username=<TARGET_USER> \
  --from-literal=password=<TARGET_PASSWORD> \
  -n ma

Reference those secret names in authConfig.basic.secretName in your workflow configuration.

Step 6: Verify connectivity before submitting a workflow


console clusters connection-check

The check runs against both source and target by default. To narrow it to one side:


console clusters connection-check --cluster source
console clusters connection-check --cluster target

For a direct API check:


console clusters curl source /
console clusters curl target /

If these checks fail, stop and fix connectivity or authentication first. Do not start a workflow yet.

Step 7: Verify AWS identity if you use SigV4

If your source or target uses Amazon OpenSearch Service or Amazon OpenSearch Serverless, verify pod identity is working from the Migration Console pod:


aws sts get-caller-identity

If console clusters connection-check works in the Migration Console but the workflow later fails with HTTP 401 or 403, verify that the Argo workflow executor pods are using the argo-workflow-executor service account with its EKS Pod Identity association. On Amazon EKS, both the Migration Console pod and the workflow executor pods get Pod Identity-backed AWS credentials automatically through the bootstrap script.

Step 8: Run a pilot migration first

Use a small allowlist or a representative subset before you attempt the full migration. This is the easiest way to catch mapping issues, authentication issues, and throughput problems early.


workflow submit
workflow manage

Use workflow manage to watch the run and approve any gated steps.

Step 9: Validate the pilot

Check counts and basic behavior on the Amazon OpenSearch Service domain or Amazon OpenSearch Serverless collection before you expand scope:


console clusters cat-indices
console clusters curl target /<index>/_count
console clusters curl target /<index>/_search?size=5&pretty

If you are migrating applications with live traffic, also validate representative queries against the target.

Step 10: Run the real migration

After the pilot succeeds, widen the configuration to the full index set and submit again:


workflow configure edit
workflow submit
workflow manage

Step 11: Use logs if anything fails


workflow status
workflow log all
workflow log all -f

workflow submit automatically stops and replaces an existing workflow with the same name, so you do not need to manually clean up between runs. If a previous run left orphaned migration custom resources, use workflow reset instead of deleting Argo workflows directly:


workflow reset                # interactive — lists CRDs and prompts before delete
workflow reset migration-foo  # delete a specific resource by name
workflow reset --all          # delete everything (capture proxies are protected)
workflow reset --all --include-proxies --delete-storage  # also remove capture proxies and Apache Kafka PVCs

Core commands

The workflow CLI orchestrates a full migration; the console CLI inspects or manually drives a single component during validation and troubleshooting.

Workflow commands

Command	Why you use it
`workflow configure sample`	Shows the sample schema for your installed version
`workflow configure sample --load`	Loads that sample as your starting point
`workflow configure edit`	Opens the workflow config in your editor (`$EDITOR`, defaults to `vi`). Use `--stdin` to pipe configuration from stdin instead of launching an editor
`workflow configure view`	Prints the current config
`workflow configure clear`	Clears the current config and lets you start over. Use `--confirm` to skip the confirmation prompt
`workflow submit`	Starts the migration workflow (auto-stops and replaces an existing one with the same name)
`workflow submit --wait --timeout 300`	Submits and blocks until the workflow completes or the timeout is reached
`workflow manage`	Primary day-to-day interface for monitoring, approvals, and logs (interactive TUI)
`workflow status`	Shows the current workflow tree in a non-interactive form
`workflow status --all`	Shows running and completed workflows
`workflow log all`	Shows logs across workflow pods
`workflow log all -f`	Streams logs live
`workflow approve step <PATTERN>`	Approves pending step gates that match exact names or globs
`workflow reset`	Lists migration custom resources and lets you delete them safely

Workflow command flags

The flags below cover all options available on each workflow subcommand. Global flags (-v/--verbose) can be placed before any subcommand to increase logging.

workflow submit

Flag	Purpose
`--wait`	Block until the workflow completes or times out (default: return immediately)
`--timeout <seconds>`	Timeout when using `--wait` (default: 120)
`--wait-interval <seconds>`	Polling interval between status checks when using `--wait` (default: 2)
`--session <name>`	Configuration session name to load parameters from (default: `default`)
`--workflow-name <name>`	Name of the workflow to replace if one already exists (default: `migration-workflow`)
`--namespace <ns>`	Kubernetes namespace (default: `ma`)

workflow status

Flag	Purpose
`--all`	Show all workflows including completed ones (default: only running)
`--all-workflows`	Show status for all workflows regardless of name (mutually exclusive with `--workflow-name`)
`--workflow-name <name>`	Show status for a specific workflow (default: `migration-workflow`)
`--live-status`	Run a current status check for each snapshot and backfill still in progress
`--argo-server <url>`	Argo Server URL (default: auto-detected from environment)
`--namespace <ns>`	Kubernetes namespace (default: `ma`)
`--insecure`	Skip TLS certificate verification (default: true)
`--token <token>`	Bearer token for Argo Server authentication

workflow log

Flag	Purpose
`--follow` / `-f`	Stream logs live using `stern` (default: show historical logs)
`--timestamps`	Include RFC 3339 timestamps in log output
`--all-workflows`	Show output for all workflows (mutually exclusive with `--workflow-name`)
`--workflow-name <name>`	Filter logs to a specific workflow (default: `migration-workflow`)
`--source` / `--target` / `--snapshot` / `--task` / `--from-snapshot-migration <value>`	Filter pods by the corresponding migration label (available on `workflow log filter`)
`--label <key=value>`	Additional raw label selector to filter pods, repeatable (e.g. `--label source=a --label target=b`); available on `workflow log filter`
`--namespace <ns>`	Kubernetes namespace (default: `ma`)
`--argo-server <url>`	Argo Server URL
`--insecure`	Skip TLS certificate verification (default: true)
`--token <token>`	Bearer token for authentication

workflow manage

Flag	Purpose
`--workflow-name <name>`	Workflow to manage (default: `migration-workflow`)
`--argo-server <url>`	Argo Server URL (default: auto-detected from environment)
`--namespace <ns>`	Kubernetes namespace (default: `ma`)
`--insecure`	Skip TLS certificate verification (default: true)
`--token <token>`	Bearer token for authentication

workflow reset

Flag	Purpose
`<path>`	Name or glob pattern of a specific resource to delete (e.g. `migration-foo`, `kafka-*`)
`--all`	Delete all migration custom resources
`--cascade`	Also delete resources that depend on the targeted resource
`--include-proxies`	Include capture proxies in deletion (they are protected by default)
`--delete-storage`	Delete Kafka PVCs and orphaned PVs during reset
`--namespace <ns>`	Kubernetes namespace (default: `ma`)

workflow approve

Flag	Purpose
`<NAMES>` (one or more, supplied to a subcommand such as `step`, `change`, or `retry`)	Exact names or glob patterns of pending approval gates to approve (e.g. `workflow approve step 'evaluatemetadata.*'`)
`--workflow-name <name>`	Workflow containing the gates (default: `migration-workflow`)
`--argo-server <url>`	Argo Server URL (default: auto-detected from environment)
`--namespace <ns>`	Kubernetes namespace (default: `ma`)
`--insecure`	Skip TLS certificate verification (default: true)
`--token <token>`	Bearer token for authentication

Console commands

The console CLI groups operations by component:

Command	Why you use it
`console --version`	Confirms which schema and behavior your Migration Console is running
`console clusters connection-check`	Verifies the Migration Console can reach and authenticate to source and target
`console clusters cat-indices [--cluster source\|target\|proxy]`	Lists indexes on one or both clusters
`console clusters curl source /_cat/indices?v`	Issues a direct API request against the named cluster
`console clusters clear-indices --cluster target --acknowledge-risk`	Destructive — deletes all indexes on the named cluster
`console snapshot {create\|status\|delete\|unregister-repo}`	Manage snapshots in Amazon S3
`console metadata {migrate\|evaluate}`	Run or preview metadata migration outside the workflow
`console backfill {describe\|start\|pause\|stop\|scale\|status}`	Inspect or drive RFS backfill
`console replay {describe\|start\|stop\|scale\|status}`	Inspect or drive Traffic Replayer
`console metrics {list\|get-data}`	Inspect Migration Assistant metrics
`console kafka {create-topic\|list-topics\|delete-topic\|…}`	Inspect Strimzi-managed Apache Kafka used by capture and replay
`console tuples`	Inspect captured request/response tuples for replay validation

Note

The workflow path drives metadata, backfill, and replay automatically. Reach for the equivalent console command only when you want to inspect state or work around a specific failure (for example, to call console snapshot status while a long-running snapshot is in progress).

Approval gates

Not every migration step should run without human review. Approval gates let the workflow stop at meaningful checkpoints — typically transitions after metadata work, backfill milestones, and cutover-sensitive steps — so you can validate before continuing.


workflow manage
workflow approve step <STEP_NAME>

Status symbols

Symbol	Meaning
`✓`	Succeeded
`▶`	Running
`○`	Pending
`✗`	Failed
`⟳`	Waiting for approval

Migration scenarios

Migration Assistant supports three migration patterns. Pick the one that matches your downtime tolerance.

Scenario 1: Backfill only

Best when you can tolerate a brief write freeze, or when writes can be paused and replayed from an external queue.


Snapshot source → Migrate metadata → Backfill documents → Verify → Switch traffic

Scenario 2: Capture and Replay only

Best when the data is small enough that live replay alone can synchronize the target on Amazon OpenSearch Service or Amazon OpenSearch Serverless, or when you want to replay traffic against multiple targets to compare results.


Reroute traffic to capture proxy → Migrate metadata → Replay traffic → Verify → Switch traffic to target

Scenario 3: Backfill + Capture and Replay (zero-downtime)

The most comprehensive approach. Capture begins first so no writes are lost, then backfill brings over historical data, then replay catches the target up to real-time.


Reroute traffic to capture proxy → Snapshot source → Migrate metadata → Backfill documents → Replay captured traffic → Verify → Switch traffic to target

Assessment and breaking changes

Before running a migration, review breaking changes between your source and target versions. Breaking changes may require modifications to your client applications after the migration completes.

Understanding breaking changes

Between major versions of Elasticsearch and OpenSearch, features may be deprecated or removed. Common breaking changes include:

Removal of mapping types (Elasticsearch 6.x to 7.x and later)
Changes to field types (for example, string replaced by text and keyword)
Query DSL syntax changes
REST API endpoint changes
Plugin compatibility differences

For a complete list of breaking changes for your migration path, see the Migration Assistant documentation on GitHub.

Impact of data transformations

Any time you apply a transformation to your data — such as changing index names, modifying field mappings, or splitting indexes with type mappings — these changes may need to be reflected in your client configurations. Run production-like queries against the Amazon OpenSearch Service domain or Amazon OpenSearch Serverless collection before switching over actual production traffic. This verifies that clients can communicate with the target, locate the necessary indexes and fields, and retrieve expected results.

For complex migrations involving multiple transformations, perform a trial migration with representative non-production data to fully test client compatibility.

Warning Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

Uninstall the solution

Reroute traffic to the capture proxy