[AWS] Remove quotes from VPC flow log message and move dot_expander to top

[kaiyan-sheng]

What does this PR do?

Turned out the incoming VPC flow log messages look like {"message":"2 428961148399 eni-0e0bf7be352692297 - - - - - - - 1671029698 1671029728 - NODATA"} instead of {\"message\":\"2 428961148399 eni-0e0bf7be352692297 - - - - - - - 1671029698 1671029728 - NODATA\"}.

Also moving the dot_expander processor to the top of the ingest pipeline so all fields with dot are expanded into an object field before further processing.

For testing the dot_expander processor change, here is what I did using the dev tools:

POST _ingest/pipeline/logs-aws.vpcflow-1.28.3/_simulate
{
  "docs": [
    {
      "_source": {
        "message": "5 64111117617 eni-069xxxxxb7a490 89.160.20.156 10.200.0.0 50041 33004 17 52 1 164000066 1640000297 REJECT OK vpc-09676f97xxxxxb8a7 subnet-02d645xxxxxxxdbc0 i-0axxxxxx1ad77 1 IPv4 89.160.20.156 10.200.0.80 us-east-1 use1-az5 - - AMAZON CLOUDFRONT ingress 1",
        "@timestamp": "2022-03-31T22:33:50Z",
        "event.id": "36768711127022897352489210216814512733652448397667467264",
        "cloud.provider": "aws",
        "cloud.account.id": "428152502467",
        "cloud.region": "us-east-1",
        "aws.firehose.arn": "arn:aws:firehose:us-east-1:428152502467:deliverystream/vpc-flow-log-stream-http-endpoint",
        "aws.firehose.request_id": "a70eae51-0629-4c78-b0dc-799a19642585",
        "aws.kinesis.type": "deliverystream",
        "aws.kinesis.name": "vpc-flow-log-stream-http-endpoint"
      }
    }
    ]
}

Note: you have to use a message field that contains the region.
Without the change in this PR, I got:

    "cloud": {
            "availability_zone": "use1-az5",
            "instance": {
              "id": "i-0axxxxxx1ad77"
            },
            "provider": "aws",
            "region": [
              "us-east-1",
              "us-east-1"
            ],
            "account": {
              "id": "64111117617"
            }
          }

With the change in this PR, I got:

    "cloud": {
            "availability_zone": "use1-az5",
            "instance": {
              "id": "i-0axxxxxx1ad77"
            },
            "provider": "aws",
            "region": "us-east-1",
            "account": {
              "id": "64111117617"
            }
          }

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-12-16T17:20:28.873+0000

• Duration: 43 min 31 sec

Test stats 🧪

Test	Results
Failed	0
Passed	178
Skipped	3
Total	181

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (14/14)	💚
Files	93.333% (14/15)	👎 -4.416
Classes	93.333% (14/15)	👎 -4.416
Methods	85.214% (219/257)	👎 -6.285
Lines	96.022% (5914/6159)	👍 4.363
Conditionals	100.0% (0/0)	💚

[zmoog]

LGTM

[elasticmachine]

Package aws - 1.28.3 containing this change is available at https://epr.elastic.co/search?package=aws