Using db.collection.bulkWrite() method in MongoDB (with examples)

Introduction

The db.collection.bulkWrite() method in MongoDB is a powerful tool that allows the execution of multiple write operations with a single request to the database. Bulk operations can significantly enhance performance when dealing with large datasets by reducing the number of roundtrips between the application and the database server. In this tutorial, we will explore how to effectively utilize the bulkWrite() method with practical code examples.

Understanding bulkWrite Operations

A bulkWrite() operation in MongoDB supports a mix of insert, update, delete, and replace operations executed in order. The method takes an array of write operation objects and an optional options object. Each write operation object represents a single database operation and is of one of the following types: insertOne, updateOne, updateMany, deleteOne, deleteMany, or replaceOne.

Basic Example

db.collection.bulkWrite([
    { insertOne: { "document": { _id: 1, name: 'John Doe', age: 28 } } },
    { insertOne: { "document": { _id: 2, name: 'Jane Doe', age: 25 } } }
]);

This script inserts two new documents into the collection. The result of the operation typically includes the number of inserted, matched, modified, and deleted documents.

Combining Different Operations

db.collection.bulkWrite([
    { insertOne: { "document": { name: 'Alice', age: 30 } } },
    { updateOne: { "filter": { age: { $gt: 28 } }, "update": { $set: { location: 'Unknown' } } } },
    { deleteOne: { "filter": { age: { $lt: 25 } } } }
]);

Here, we are performing an insert, an update, and a delete operation in a single bulkWrite() call. The updateOne operation targets documents where the age is greater than 28 and assigns a new field location. The deleteOne operation removes documents where the age is less than 25.

Ordered vs. Unordered Bulk Operations

By default, BulkWrite operations are ordered, meaning they will execute sequentially and stop at the first error encountered. You can perform unordered operations by setting the ordered option to false.

db.collection.bulkWrite(
    [
        // ... (your operations here)
    ],
    { ordered: false }  // Continue after error
);

When ordered is set to false, MongoDB will continue processing the remaining operations even if one fails.

Handling Errors

When an error occurs in an ordered bulkWrite, subsequent operations are not processed. The error contains information about the operation that caused it and the index of that operation in the array.

For unordered bulk operations, you can capture individual errors for each failed operation, and other operations will continue to be processed.

Working with Write Concern and Journalling

MongoDB’s write concern allows you to specify the level of acknowledgment requested from MongoDB for write operations. Additionally, you can specify journalling options. This is crucial for critical applications that require a guarantee of write durability.

db.collection.bulkWrite(
    [
        // ... (your operations here)
    ],
    { writeConcern: { w: 'majority', j: true } }
);

The w: 'majority' option ensures that the majority of the replica set members have applied the write. The j: true option requires that the mongod instance write the operation to the journal.

Advanced Bulk Examples

The real power of bulkWrite becomes apparent when performing large-scale updates or inserting vast amounts of data. Here is an example of a complex bulk operation that involves conditional updates.

db.collection.bulkWrite([
    { updateMany: {
        "filter": { "item": { $regex: /^p/ } },
        "update": { $set: { "category": 'paper' } }
    } }
]);

The above operation will update all documents where the ‘item’ field’s value begins with a ‘p’ and set their ‘category’ field to ‘paper’.

Transaction Support in BulkWrite

With MongoDB 4.0 and later, bulkWrite can be used within multi-document transactions. This allows for a batch of operations to be applied atomically.

const session = db.getMongo().startSession();
session.startTransaction();
try {
    db.collection.bulkWrite(
        [
            // ... (your operations here)
        ],
        { session: session }
    );
    session.commitTransaction();
} catch (e) {
    session.abortTransaction();
    throw e;
} finally {
    session.endSession();
}

In this example, if the bulkWrite fails, the transaction is aborted and no changes are made to the database.

Optimization Strategies

When using bulkWrite, you should be mindful of the maximum size of a bulk write operation. Large batches may exceed this limit and cause an error. It’s also important to be aware of the performance impact on the database, especially when using bulk operations that modify large portions of the data.

For best performance, batch your write operations in sensible chunks and monitor the performance impact on your application and the database during execution.

Conclusion

By leveraging the bulkWrite() method in MongoDB, developers can perform efficient bulk operations that save time and resources. Throughout this tutorial, we’ve covered basic to advanced usage of bulkWrite, strategies for handling errors, the differences between ordered and unordered operations, and tips for optimizing bulk write performance.