Skip to content

Feature request: decode key/val lazily to allow error handling #4067

@dreamorosi

Description

@dreamorosi

Use case

When working with the recently merged kafkaConsumer, as a customer I want to be able to handle decoding errors and potentially take additional steps to log, persist, or redirect messages that failed to decode.

The current implementation eagerly decodes keys and values before the customer handler runs, which makes it impossible for customers to handle errors, short of wrapping it in a global try/catch.

Solution/User Experience

Customers should be able to do this:

const handler = kafkaConsumer(
  async (event) => {
    for (const record of event.records) {
      try {
        const { value, key } = record; // decoding & parsing now happens here when accessing
        // ... your business logic
      } catch (error) {
        // handle error, maybe send to a DLQ
      }
    }
  },
  {
    value: {
      type: SchemaType.JSON,
      parserSchema: valueZodSchema,
    },
    key: {
      type: SchemaType.JSON,
      parserSchema: keyZodSchema,
    },
  }
);

Alternative solutions

Acknowledgment

Future readers

Please react with 👍 and your use case to help us understand customer demand.

Metadata

Metadata

Assignees

Labels

completedThis item is complete and has been merged/shippedfeature-requestThis item refers to a feature request for an existing or new utility

Type

No type

Projects

Status

Shipped

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

    close