[ACCEPTED]-Inserting a big array of object in mongodb from nodejs-bigdata
Accepted answer
You can use bulk inserts.
There are two types 2 of bulk operations:
- Ordered bulk operations. These operations execute all the operation in order and error out on the first write error.
- Unordered bulk operations. These operations execute all the operations in parallel and aggregates up all the errors. Unordered bulk operations do not guarantee order of execution.
So you can do something 1 like this:
var MongoClient = require('mongodb').MongoClient;
MongoClient.connect("mongodb://myserver:27017/test", function(err, db) {
// Get the collection
var col = db.collection('myColl');
// Initialize the Ordered Batch
// You can use initializeUnorderedBulkOp to initialize Unordered Batch
var batch = col.initializeOrderedBulkOp();
for (var i = 0; i < sizeOfResult; ++i) {
var newKey = {
field_1: result[i][1],
field_2: result[i][2],
field_3: result[i][3]
};
batch.insert(newKey);
}
// Execute the operations
batch.execute(function(err, result) {
console.dir(err);
console.dir(result);
db.close();
});
});
As For Version > 3.2, insertMany has been introduced 5 which uses bulkWrite under the hood for bulk insert 4 purposes only.
insertMany supports ordered and unordered inserts. Unordered being 3 faster as mongo decides the ordering. Likewise 2 your implementation for best through-put 1 should be ::
var sizeOfArray = arrayOfObjects.length; for(var i = 0; i < sizeOfResult; ++i) { newKey = { field_1: result[i][1], field_2: result[i][2], field_3: result[i][3] }; } collection.insertMany(newKey, { ordered: false }).then((res) => { console.log("Number of records inserted: " + res.insertedCount); })
Source:
stackoverflow.com
More Related questions
Cookie Warning
We use cookies to improve the performance of the site. By staying on our site, you agree to the terms of use of cookies.