Skip to content

MapToList processor #3935

@oeyh

Description

@oeyh

Is your feature request related to a problem? Please describe.
A processor that converts a map of key-value pairs:

{
  "my-map": {
    "key1": "value1",
    "key2": "value2",
    "key3": "value3"
  }
}

to a list of object with key and value under separate fields:

{
  "my-list": [
    {
      "key": "key1",
      "value": "value1"
    },
    {
      "key": "key2",
      "value": "value2"
    },
    {
      "key": "key3",
      "value": "value3"
    }
  ]
}

This can be used when my-map contains a large number of fields to avoid mapping explosion in OpenSearch while keeping the key-value pairs searchable.

Describe the solution you'd like
This can be a reversed operation of what ListToMap processor does. The configuration options would be like this:

- map_to_list:
    source: "my-map"
    target: "my-list"
    key_name: "key"
    value_name: "value"

source(required): the source map to perform the operation
target(required): the target list
key_name: the key name of the field to hold the original key, default is "key"
value_name: the key name of the field to hold the original value, default is "value"

Describe alternatives you've considered (Optional)
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Metadata

Metadata

Assignees

Labels

plugin - processorA plugin to manipulate data in the data prepper pipeline.

Type

No type
No fields configured for issues without a type.

Projects

Status
Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions