DEV Community

Cover image for Building a Smart Knowledge Platform with MindsDB
K Om Senapati
K Om Senapati

Posted on

Building a Smart Knowledge Platform with MindsDB

Working with structured data like CSVs or JSON often means writing queries, managing schemas, and building pipelines just to answer simple questions. If you want to make that data interactive like asking natural language questions or building an AI agent on top of it, things get complicated fast. You usually need to write code, understand vector stores, manage embeddings, and stitch tools together.

That’s the problem I faced.

I wanted a simple way to:

  • Upload structured data
  • Turn it into something queryable with natural language
  • Build an AI agent that could talk about that data

The Solution: MindsDB

Mindsdb Logo

That’s where MindsDB came in.

MindsDB recently added support for Knowledge Bases (KBs). You can now ingest structured data into a KB, use an embedding model (cloud or local), and build multi-step workflows or AI agents on top of it without writing code.

This release immediately clicked with me. I had already used MindsDB’s AI Table feature to build a quiz app. It lets you interact LLMs so I was familiar with the platform and trusted its approach.

With this KBs + Agents integration, I no longer needed to code the whole pipeline. I just wrote a couple of SQL queries in the MindsDB GUI, and voilà — I had an AI agent ready to go.

MindsDB Philosophy

What I especially liked is how MindsDB thinks about data and intelligence. Their philosophy is:

  • Connect: Connect data from hundreds of data sources
  • Unify: Unify data from multiple (structured and unstructured) data sources within MindsDB, enabling federated queries as if all data resides in a single database
  • Respond: Use agents to give intelligent, context-aware responses from that unified data

Dive into MindsDB 👉 MindsDB Docs

As a Python, and SQL guy, this felt like the right mix of practicality and power.


About Agent Hub

Agent Hub works with structured data: CSV and JSON, Knowledge Bases, and AI Agents.

Features:

  1. User can upload structured data.
  2. User can create a KB and ingest the data into it using any OpenAI-compatible API embedding model (cloud/local) or Ollama models.
  3. User can query the KB and also generate a summary from it.
  4. User can create AI Agents and chat with them.

For structured data, I needed a database. I found out that MindsDB has a files database, which is perfect for storing user-uploaded data files.

Then I created a mini MindsDB package to encompass all the MindsDB logic using the MindsDB Python SDK and SQL queries to do all the operations:

  • Create a project for each user
  • Prefix files with the username
  • Create KBs
  • Ingest data into KBs
  • Delete KBs
  • Query and summarize the KB using AI Table
  • Create and use AI Agents

Then I created a simple Flask app which used a SQLite database and SQLAlchemy as the ORM. I used HTML Jinja2 templates. Yeah, no frontend framework.
Python is 💖.

GitHub logo k0msenapati / agent-hub

Agent Hub is an AI collaboration platform designed to transform your data into intelligent AI agents.

🤖 Agent Hub

Note

Agent Hub is an AI collaboration platform designed to transform your data into intelligent AI agents. With Agent Hub, you can unlock powerful insights and automate tasks effortlessly by creating tailored AI agents, building comprehensive knowledge bases, and managing your data files seamlessly.

🎬 Project Showcase

Demo Video Blog Post
YouTube Blog

🌟 Features

Agent Hub features intro:

  • AI Agents – Create and manage AI agents tailored to your specific needs for intelligent responses.
  • Knowledge Bases – Build comprehensive knowledge bases from your data for efficient querying and insights.
  • File Management – Upload and organize your files to integrate seamlessly with AI processes.

💻 Installation

Follow these steps to set up and run Agent Hub:

  1. Setup MindsDB and Ollama with Docker (docker-compose.yml is provided):

    docker-compose up -d
    Enter fullscreen mode Exit fullscreen mode
  2. Install nomic-embed-text in Ollama container:

    docker exec ollama ollama pull nomic-embed-text
    Enter fullscreen mode Exit fullscreen mode
  3. Create KB Summarizer Model and Connect Pgvector in MindsDB


Challenges

  1. I ran into an issue with MindsDB when performing operations on a KB using an embedding model from Ollama. It would throw an "Event Loop Closed" error on the first attempt. Strangely, repeating the same operation a second time worked just fine. So, I handled that in the app using a simple try-catch with a retry.

  2. For the frontend, I kind of vibe-coded the Jinja2 templates. But hey, the idea is mine 😜

Building this was fairly easy as all the heavy lifting is done by MindsDB.
I just had to trigger the right queries from the Flask backend.


Use Cases

  • Upload structured data like logs, reports, or exports
  • Query or summarize data with natural language
  • Create intelligent agents trained on your data

Final Thoughts

This is just an MVP, but it opens up a lot of possibilities.

Right now, it works with structured data like CSV and JSON but it can easily be extended to handle other formats like web pages, PDFs, or Word docs. MindsDB’s Connect layer also makes it easy to bring in data from external databases and APIs, so the potential to scale and integrate is built-in.

What I’ve built is a starting point: a way to turn raw, structured data into something intelligent. It’s simple, but powerful. And with MindsDB handling the heavy lifting, it’s also surprisingly easy to build.

If you found this article useful, share it with your peers and community to spread the word about this.

Follow me for more content like this!

Twitter | GitHub | YouTube

bye bye gif

Top comments (16)

Collapse
 
rohan_sharma profile image
Rohan Sharma

Liked the project!!

Collapse
 
k0msenapati profile image
K Om Senapati

Thanku

Collapse
 
sharmachaitanya945 profile image
Chaitanya Sharma

Nice Project yes had similar issue with using ollama used the retry method as well for the same

Collapse
 
chiragagg5k profile image
Chirag Aggarwal

amazing one 🔥

Collapse
 
k0msenapati profile image
K Om Senapati

Waiting for yours

Collapse
 
anmolbaranwal profile image
Anmol Baranwal • Edited

This is nice 🔥

Collapse
 
k0msenapati profile image
K Om Senapati

DMs are for a reason 😒

Collapse
 
prasant_f0c8a07abb232c0f0 profile image
Prasant

Awesome project 🫡

Wish I had docker support

Collapse
 
jyoti_ranjanjena_a3178e7 profile image
Jyoti Ranjan Jena

This is really good idea and implementation 🔥🔥

Collapse
 
yashksaini profile image
Yash Kumar Saini

Actual SaaS platform for MindsDB. A real Gem

Collapse
 
pheonixcoder_56 profile image
Pheonix Coder 🐦‍🔥

Cool use case of MindsDB

I am also thinking of some project

Collapse
 
k0msenapati profile image
K Om Senapati

Try

Collapse
 
dinesh_mohanty_5ff1ba67cd profile image
Dinesh Mohanty

Awesome 👍

Collapse
 
debasis_panda_31e768f5e44 profile image
Debasis Panda

Nice

Some comments may only be visible to logged-in visitors. Sign in to view all comments.