Creating Real-Time Dashboards Using AWS OpenSearch, EventBridge, and WebSockets

If you've attempted to build a dashboard, then you're familiar with the hassle of polling. You hit your API every couple of seconds, grab updates, and pray your data doesn't feel stale. However, if we're being honest, polling is inefficient, wasteful, and antiquated. In the modern era, users expect supplies to be dynamic and flowing. We, as developers, should meet that expectation without melting our servers.

In this post, I will walk you through a serverless, event-driven architecture that I've leveraged to build real-time dashboards using AWS. This architecture will tie together EventBridge, OpenSearch, and API Gateway WebSockets with a hint of Lambda and DynamoDB. By the end, you'll have some understanding of how all the pieces are tied together to create a live dashboard data pipeline that can scale, can be cost-friendly, and actually feels fast for the end-user.

Let’s get started!

Why Not Just Poll?

Traditional dashboards depend on queries every few seconds on some database. While this is fairly simple, it has a major disadvantage:

• Latency: The data feels outdated, always just behind.
• Cost: There are excessive, unnecessary queries being created for every poll that is hitting your backend.
• User experience: Users see stale charts, and they get frustrated staring at a chart that doesn't feel dynamic.

Instead of forcing the UI to constantly check back in, "Are we there yet?", we change that—events will push to the dashboard as they happen.

Now we consider the AWS trio:

• Amazon EventBridge – the backbone of the architecture, as it captures domain events.
• Amazon OpenSearch Service – provides fast indexing and returning based on event queries.
• Amazon API Gateway (WebSocket) + Lambda + DynamoDB – provides a live communication layer that pushes updates to each client in real-time.

Sounds great thus far? Let’s review how the architecture all works together.

Architecture Overview

Below shows an end-to-end flow of how events are pushed through the architecture:

1. A downstream service emits an event → EventBridge captures the event.
2. EventBridge routes the event to an Indexing Lambda, which normalizes and stores the event to OpenSearch.
3. The event succeeds indexing, and the Lambda creates a 'delta' event back into EventBridge.
4. That event in EventBridge triggers the Broadcast Lambda, which looks up active WebSocket connections in DynamoDB.
5. The Broadcast Lambda pushes updates to clients over API Gateway WebSockets.
6. Clients render changes immediately — no refreshing, no polling.

An illustration (very ASCII) of the flow:

Service → EventBridge → Indexing Lambda → OpenSearch
↓
EventBridge (delta)
↓
Broadcast Lambda
↓
API Gateway (WebSocket) → Clients (UI)

Step 1: Indexing Data to OpenSearch

Before pushing the event to dashboards, it is always best to have an indexing strategy. Here is one example to use for an index template:

{
  "index_patterns": ["metrics-*"],
  "template": {
    "settings": {
      "number_of_shards": 1,
      "number_of_replicas": 1
    },
    "mappings": {
      "dynamic": "false",
      "properties": {
        "@timestamp": { "type": "date" },
        "service": { "type": "keyword" },
        "eventType": { "type": "keyword" },
        "latencyMs": { "type": "long" },
        "message": {
          "type": "text",
          "fields": {
            "raw": { "type": "keyword" }
          }
        }
      }
    }
  }
}

A few best practices I learned the hard way:

• Use stable IDs (eventId) for documents to remove duplicates.
• Use Index State Management (ISM) to rotate indices in a daily time frame.
• Use ISM to auto-expire outdated data by a time of 14 days (or however long you want).

Why? Because dashboards are not data lakes. You should be able to query the data you want and not have to page through potentially large indexes.

Step 2: Infrastructure for WebSocket

Firstly, you need a serverless WebSocket API. We will use API Gateway to create the WebSocket API. There are three important routes you care about:

• $connect – This will save the connectionId in DynamoDB.
• $disconnect – Removing it upon client termination.
• $default – This only needs to return optional messages from the client (e.g., when a user subscribes to a channel).

Here is an example of the DynamoDB table set up (via SAM/CDK):

Resources:
  ConnTable:
    Type: AWS::DynamoDB::Table
    Properties:
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: connectionId
          AttributeType: S
      KeySchema:
        - AttributeName: connectionId
          KeyType: HASH

For each active client, we put a row in this table. That is how the Broadcast Lambda knows who is online.

Step 3: Broadcasting Events

Once the new event is indexed, we are off to the races. We send it out to all the active clients without hesitation.

Here is a sample Lambda function:

import { DynamoDBClient, ScanCommand } from "@aws-sdk/client-dynamodb";
import { ApiGatewayManagementApiClient, PostToConnectionCommand } from "@aws-sdk/client-apigatewaymanagementapi";

const ddb = new DynamoDBClient();
const api = new ApiGatewayManagementApiClient({ endpoint: process.env.WS_ENDPOINT });
const TABLE = process.env.TABLE;

export const broadcast = async (event) => {
  const payload = event.detail || event;
  
  const conns = await ddb.send(new ScanCommand({ TableName: TABLE }));
  const targets = conns.Items.map(i => i.connectionId.S);

  const msg = JSON.stringify({
    type: "hits",
    hits: payload.hits || [payload.doc]
  });

  await Promise.allSettled(
    targets.map(id =>
      api.send(
        new PostToConnectionCommand({ ConnectionId: id, Data: Buffer.from(msg) })
      )
    )
  );

  return { sent: targets.length };
};

As you can see:

• Scan DynamoDB to get the active connections.
• Serialize the payloads into the JSON message.
• Utilize multiple promises with Promise.allSettled so that if one connection breaks, the rest of the batch goes through.

Step 4: Client Side

Let’s keep things unbelievably simple here:

<script>
  const ws = new WebSocket('wss://<api-id>.execute-api.<region>.amazonaws.com/prod');

  ws.onmessage = (evt) => {
    const msg = JSON.parse(evt.data);

    if (msg.type === 'hits') {
      console.log("Live update:", msg.hits);
      // Update your chart or table here
    }
  };
</script>

That's all the magic it takes to bring the data to life, no cron jobs, no reloads, just instant feedback.

Lessons Learned (The Hard Way)

Here are some keys points I wish someone would have told me sooner.

• Idempotency – Always, always, always use steady id's for your documents to avoid duplicate assignments.
• Do not spam everyone – Utilize channel attributes in DynamoDB, which will allow you to push events to just the right clients.
• There are size limits – Each WebSocket message is limited to a set size of 32 KB; if you go over, batch or sample your payload.
• Do not forget about costs – Do filtering in OpenSearch query, and only push deltas through WebSockets.

Security First

Attach a Lambda Authorizer to the $connect to validate JWTs or API keys.

Have you thought about it; do you really want every single user to see every event? Or would they rather filter events by, team, service or location?

Frequently Asked Questions

Q: Can this work without OpenSearch?

Yes, you could just push raw events directly to clients. But dashboard interfaces generally require querying, filtering and analytics. OpenSearch can do that.

Q: How many WebSocket clients can API Gateway handle?

Tens of thousands. For very high scale scenarios, shard your connections and distribute across multiple WebSocket API end points.

Q: What about retries to a failed push?

There are DLQs (Dead Letter Queues) or retries in your Broadcast Lambda. You should get any failed connections and remove them from DynamoDB quickly.

Q: Is this overkill for small apps?

Honestly. Yes. If you are building a hobby project, polling every five seconds is fine. Once you want to start providing real-time metrics to users or users on dashboards this will be worth it.

Conclusion

With EventBridge, OpenSearch, and WebSockets we can get rid of polling and build dashboards that feel alive in real-time. On top of that, this stack is completely serverless and can scale with your traffic without you having to babysit anything.

The next time someone asks you to stream metrics, logs or KPIs into a dashboard interface consider using this pipeline as opposed to cron jobs.

So what do you think, would you replace your polling inside your dashboards with this option? Or do you feel there could be challenges that I did not mention?

Regardless, I would like to hear your thoughts on how you might change this experience into your own projects.