DataOps

When there are many connectors, it is not convenient for users to locate the failed task because it cannot be filtered by the task status.

We need to support writing to and reading from TFRecord format.

Reference doc: https://www.tensorflow.org/tutorials/load_data/tfrecord

More on TypeTransfomers can be found here.

Related PR that adds PyTorch tensor and module as Flyte types: flyteorg/flytekit#1032

The default RubrixLogHTTPMiddleware record mapper for token classification expect a structured including a text field for inputs. This could make prediction model inputs a bit cumbersome. Default mapper could accepts also flat strings as inputs:

def token_classification_mapper(inputs, outputs):
    i

Bigquery external table API supports auto schema detection for external table source by a boolean property - ref.

If the target field names are the same as the source fields in the Google Spreadsheet, it should be better not to supply any schema rather than sp

Support copy into queries

Currently, both Kafka and Influx sink logs only the data(Row) that is being sent.

Add support for logging column names as well along with data points similar to the implementation in log sink.

This will enable users to correlate the data points with column names.

Is your feature request related to a problem? Please describe.
I was trying to store extra group information with below details

[PUT] /v1beta1/groups/{id}
{
  "name": "My Group",
  "slug": "my-group",
  "orgId": "{ORG_ID}",
  "metadata": {
    "description": "my-group-description"
    "is_active": true
  }
}

But shield responded with

Status: 400 Bad Request
{
   

What is the feature request? What problem does it solve?

As employees leave the organization/company or users change mails , eventually the notification list configured for the job would start containing a lot of invalid mails. This causes issues with SMTP relay (e.g postfix) which could be buffering all invalid requests until the queu is full, which cause all mails coming for all jobs to b

In golang client, consumers get dynamic message instance after parsing. Add an example in the docs on how to use dynamic message instance to get values of different types in consumer code.

List of protobuf types to cover

• timestamp
• duration
• bytes
• message type
• struct
• map

Deliverables

• add unit tests
• add extractor
• add README.md in plugins/extractors/neo4j, defining output
• register your extractor plugins/extractors/populate.go
• add extractor the extractor list in docs/reference/extractor.md

Output must contain a Table

Explore the Table Data Model and add as many features as possible.

Table

| Fi