Streaming

rpk transform deploy

Deploy a transform.

When run in the same directory as a transform.yaml, this reads the configuration file, then looks for a .wasm file with the same name as your project. If the input and output topics are specified in the configuration file, those are used. Otherwise, the topics can be specified on the command line using the --input-topic and --output-topic flags.

To deploy Wasm files directly without a transform.yaml file:

rpk transform deploy --file transform.wasm --name myTransform \
--input-topic my-topic-1 \
--output-topic my-topic-2 \
--output-topic my-topic-3

Environment variables can be specified for the transform using the --var flag, these are separated by an equals for example: --var=KEY=VALUE.

The --var flag can be repeated to specify multiple variables like so:

rpk transform deploy --var FOO=BAR --var FIZZ=BUZZ

The --from-offset flag can be used to specify where on the input topic the transform should begin processing. Expressed as:

  • @T - Begin reading records with committed timestamp >= T (UNIX time, ms from epoch)

  • +N - Begin reading N records from the start of each input partition

  • -N - Begin reading N records prior to the end of each input partition

Note that the broker will only respect from-offset on the first deploy for a given transform. Re-deploying the transform will cause processing to pick up at the last committed offset. Recall that this state is maintained until the transform is deleted.

Usage

rpk transform deploy [flags]

Examples

This section provides examples of how to use rpk transform deploy.

Deploy Wasm files directly without a transform.yaml file.

rpk transform deploy --file transform.wasm --name myTransform \
--input-topic my-topic-1 \
--output-topic my-topic-2
--output-topic my-topic-3

Deploy a transformation with multiple environment variables.

rpk transform deploy --var FOO=BAR --var FIZZ=BUZZ

Configure compression for batches output by data transforms. The default setting is none but you can choose from none, gzip, snappy, lz4, or zstd.

rpk transform deploy --compression <compression_type>

Flags

Value Type Description

--compression

string

Output batch compression type. Valid values: none, gzip, snappy, lz4, zstd. Defaults to none.

--file

string

The WebAssembly module to deploy.

--from-offset

string

Starting offset for input topic partitions. Use @T for a timestamp (Unix milliseconds), +N for N records from the start, or -N for N records from the end. Only respected on first deploy.

-i, --input-topic

string

The input topic to apply the transform to.

--name

string

The name of the transform.

-o, --output-topic

stringSlice

The output topic to write the transform results to (repeatable).

--var

environmentVariable

Specify an environment variable in the form of KEY=VALUE.

Global flags

Value Type Description

--config

string

Redpanda or rpk config file; default search paths are ~/.config/rpk/rpk.yaml, $PWD/redpanda.yaml, and /etc/redpanda/redpanda.yaml.

-X, --config-opt

stringArray

Override rpk configuration settings; -X help for detail or -X list for terser detail.

--ignore-profile

bool

Ignore rpk.yaml and redpanda.yaml; use default settings.

--profile

string

rpk profile to use.

-v, --verbose

bool

Enable verbose logging.

Enabling compression may increase computation costs and could impact latency at the output topic.

For more details, see Sizing for Production.