Context

Logstash allows for putting dynamic fields in the event data, such as the current time in your specified format.

This format is specified in the sprintf +FORMAT, and applied to the @timestamp field.

Issue

The locale of the sprintf differs from the locale of the date filter. Thi

If i have an input yaml connecting to jdbc source like:

inputs:
  somedb:
    jdbc:
      connectionUrl: jdbc:jtds:redact
      user: someuser
      password: somepass

the log has

2020-12-08 17:01:28,076 [main] INFO  com.yotpo.metorikku.Job - these are the config inputs: Some(Map(.....somepass

ideally this should be printed as ****** in the log

Pivot missing categories breaks FeatureSet/AggregatedFeatureSet

Summary

When defining a feature set, it's expected that pivot will have all categories and, as a consequence, the resulting Source dataframe will be suitable to be transformed. When a different behavior happens, FeatureSet and AggregatedFeatureSet breaks.

Feature related:

Age: legacy

Enhance KgtkReader to read MD files written by KgtkWriter.