MongoDB Duplicate Key Error

Question

So right now, I'm working on a service to allow multiple events to store data on MongoDB. They store event data by creating new collections on MongoDB every time a new event comes on. If the same event needs to store a different set of data, a new document in MongoDB should be created.

The code below is the service I created to handle this.

import WhiteBoardEvent from '../model/event.model';
import IEventStore from '../interface/eventStore.interface';
import * as MongoClient from 'mongodb';

export class EventStore implements IEventStore {
    private mongoDBEndpoint = "mongodb://192.168.10.10:27017";


    public insert(event: WhiteBoardEvent, callback: (err: any) => void): void {
        MongoClient.connect(this.mongoDBEndpoint, { connectTimeoutMS: 1000 }, (connErr, db) => {
            if (connErr) { db.close(); callback(connErr); return; }
            this.getNextSequence(db, event, (err, sequence) => {
                if (err) { db.close(); callback(err); return; }
                event.sequence = sequence;
                db.collection(event.roomId).insert(event, (err) => {
                    db.close();
                    callback(err);
                });
            });
        });
    }

    private createCounterCollection(db: MongoClient.Db, event: WhiteBoardEvent, callback: (err: any) => void): void {
        db.collection("counters").insert({
            roomId: event.roomId,
            sequence: 0
        }, callback);
    }

    private getNextSequence(db: MongoClient.Db, event: WhiteBoardEvent, callback: (err: any, sequence: number) => void): void {
        var collection = db.collection("counters");
        collection.findOneAndUpdate(
            { roomID: event.roomId },
            {
                $inc: { sequence: 1 },
                // new: true
            },
            {
                upsert: true,
                returnOriginal: false
            },
            (err, r) => {
                if (err) {
                    this.createCounterCollection(db, event, (err) => {
                        if (err) { callback(err, -1); return; }
                        callback(null, 0);
                    });
                    return;
                }
                callback(null, r.value.sequence);
                console.log("counter : " + r.value.sequence);
            }
        );
    }
}

The following code is a test file I created so that I can see the changes in MongoDB.

import * as timers from 'timers';
import WhiteBoardEvent from './data/model/event.model';
import { EventStore } from './data/service/eventStore.service';

var model = new WhiteBoardEvent();
model.name = "t2";
model.roomId = "testRoom";
model.timestamp = new Date();
model.userId = "";


var model2 = new WhiteBoardEvent();
model2.name = "t1";
model2.roomId = "testRoom2";
model2.timestamp = new Date();
model2.userId = "";

var eventStore = new EventStore();

var timer1 = timers.setInterval(()=>{
    eventStore.insert(model, (err)=>{
        if(err){
            console.log(err);
        }else{
            console.log("Test Completed!");

        }
    });
}, 1000);

var timer2 = timers.setInterval(()=>{
    eventStore.insert(model2, (err)=>{
        if(err){
            console.log(err);
        }else{
            console.log("Test Completed!");

        }
    });
}, 1000);

This is a snippet of the output I get. Here, "Test Completed" is shown for the first instances, after that, I'm getting the duplicate errors.

counter : 1
counter : 1
Test Completed!
Test Completed!
counter : 2
{ MongoError: E11000 duplicate key error collection: admin.testRoom index: 
_id_ dup key: { : ObjectId('59d5da14cedd6f28a5db8c93') }

Can anyone help me with this? Thank you in advance!

Kirk Larkin · Accepted Answer · 2017-10-05 07:40:05Z

You are creating two instances of WhiteBoardEvent without explicitly setting an ID (this is fine, but relevant). Have a look at this excerpt from your code above:

db.collection(event.roomId).insert(event, (err) => {
    db.close();
    callback(err);
});

After handing event over to MongoDB's insert, it is checked to see if it has an ID - it does not. Because of this, the MongoDB code generates an ID for you (see here). This is all great - it's what you want.

However, what happens the next time your setInterval callback is invoked? Well, model and model2 now have an ID set - it was set according to the rules I just described. In this case, now that there's an ID set on the model going into insert, you are trying to reuse the same ID as the MongoDB code leaves it alone.

In your test code, you could simply clear out the ID in your eventStore.insert callback to ensure that a new ID is generated every time. e.g.:

eventStore.insert(model, (err)=>{
    model._id = null;

    if(err){
        console.log(err);
    }else{
        console.log("Test Completed!");
    }
});

I found an alternative where I used casting. it was like: delete (<any>model)._id;

Sven · Accepted Answer · 2017-10-05 07:37:31Z

It is likely that in your scheme you have you have a key set on unique: true. Adding another object with the same key or a key not filled in will result in a duplicate key error. Because, if a field is not filled in it will be filled in with null. So 2 times null is a duplicate key error. To make sure this will not happen.

Use sparse: true instead of unique: true. Also note that a field with unique: true is never able to have two of the same keys. Sparse is only able to have multiple nulls(undefined) inside and works the same as unique: true further.

In your case you have to times the userid on "", this will probably cause the error if its set on unique.model.userId = "";

Hope this will solve your answer. Else please show us your model.

Sven

Collectives™ on Stack Overflow

MongoDB Duplicate Key Error

2 Answers 2

1 Comment

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Related