MongoDB: Using Vermongo to track document history (with examples)

Introduction

MongoDB, the popular NoSQL database, is known for its flexibility and performance when dealing with large data sets and complex data structures. However, tracking changes to documents historically has always been a developer’s predicament. Unlike traditional relational databases that offer built-in solutions for versioning, MongoDB requires additional mechanisms. Amongst various approaches, Vermongo has emerged as a compelling solution to track document history effectively and efficiently. In this tutorial, we delve into the world of Vermongo and how you can leverage it to maintain version information for your MongoDB documents.

What is Vermongo?

Vermongo is a pattern for MongoDB that allows for the implementation of versioning on documents within a collection. In essence, it creates a parallel collection which shadows a source collection to store historical versions of documents. Each time an update or delete operation is performed, a copy of the original document is saved in this versioned collection before the operation proceeds.

Implementing this approach manually would require a lot of boilerplate code and careful consideration to avoid race conditions and ensure data integrity. In this guide, we explore how to implement Vermongo using the ‘vermongo’ plugin for popular MongoDB object modeling tool Mongoose. This approach simplifies the versioning of documents in a MongoDB database.

Setting Up the Project

First and foremost, ensure you have Node.js and MongoDB installed on your system. Then, set up a new Node project:

mkdir vermongo-tutorial
cd vermongo-tutorial
npm init -y
npm install mongoose vermongo

Coding the Schema with Vermongo

With the project folder prepared and our necessary packages installed, the next step is to set up a Mongoose schema with Vermongo:

const mongoose = require('mongoose');
const { vermongoOptions, vermongoPlugin } = require('vermongo');

const schema = new mongoose.Schema({
  itemName: String,
}, vermongoOptions);

schema.plugin(vermongoPlugin);

const Item = mongoose.model('Item', schema);

This simple Item schema now has versioning capabilities. Note the usage of vermongoPlugin and the optional vermongoOptions as part of the schema definition.

Connecting to MongoDB

Next, you need to establish a connection to your MongoDB database:

mongoose.connect('mongodb://localhost:27017/vermongoTutorial', {
  useNewUrlParser: true,
  useUnifiedTopology: true
})
.then(() => {
  console.log('Connected to database!');
})
.catch((error) => {
  console.error('Database connection failed:', error);
});

Creating and Tracking Changes

With our schema set and connection established, it’s time to create and track changes to a document:

const newItem = new Item({ itemName: 'Sample Item' });

newItem.save()
  .then((item) => {
    console.log('Item created:', item);

    Item.findByIdAndUpdate(item._id, { itemName: 'Updated Item' }, { new: true })
      .exec()
      .then((updatedItem) => {
        console.log('Item updated:', updatedItem);

        // Now let's find the history
        Item.historyModel()
          .find({ refId: item._id })
          .exec()
          .then((history) => {
            console.log('Document history:', history);
          })
          .catch(console.error);
      })
      .catch(console.error);
  })
  .catch(console.error);

The item creation will result in one document in the ‘items’ collection. When we update it, another document with the previous state is stored in ‘items_versions’.

Deleting Documents and Tracking History

If we delete an item, its history remains:

Item.findByIdAndDelete(item._id)
  .exec()
  .then(() => {
    Item.historyModel()
      .find({})
      .exec()
      .then((history) => {
        console.log('Remaining history after deletion:', history);
      })
      .catch(console.error);
  })
  .catch(console.error);

Note how we access the historical data using historyModel(), a method added to our Mongoose model by Vermongo.

Advanced: Handling Complex Schemas and Relations

While our examples have been straightforward, let’s look at how you might handle more complex data structures:

Imagine you have a User model that can have multiple Item children:

const userSchema = new mongoose.Schema({
  username: String,
  // Other user fields...
  items: [{ type: mongoose.Schema.Types.ObjectId, ref: 'Item' }],
});

To track the versions of nested documents, you must also employ versioning at the nested levels. In scenarios where this would be impractical or if the relation could potentially reference a large number of documents, a different architecture might be more suitable, such as event sourcing or a dedicated change-tracking system. Vermongo is best suited for more straightforward use-cases where the complete snapshot of the document is required.

Error Handling and Recovery

Should something go wrong during an update or delete operation, you will have the original state of the document stored safely in the versions collection, from where you can recover data if needed:

// Example recovery function
const recoverItem = async (itemId) => {
  try {
    const history = await Item.historyModel()
      .findOne({ refId: itemId })
      .sort({ 'versionAt': -1 }); // Latest version

    if (history) {
      // Update the main collection with the last known good state
      await Item.findByIdAndUpdate(history.refId, history, { new: true }).exec();
      console.log('Item recovered successfully.');
    }
  } catch (error) {
    console.error('Recovery failed:', error);
  }
};

This recovery feature makes it a handy tool for unforeseen accidents or data corruption scenarios.

Performance Considerations

Vermongo creates additional overhead because it copies the entire document before each update or delete operation. Moreover, as your versions collection grows, the storage required for your database could potentially double. While this may not be a concern for smaller projects, larger applications with frequent updates may need a pruning strategy for historical data.

Conclusion

Throughout this tutorial, we have learned how to utilize Vermongo for versioning documents in MongoDB. It provides a systematic approach to maintaining document history with minimal disruption to standard coding practices. As we manage the data lifecycle, tools like Vermongo can be crucial for scenarios that demand auditability and historical data integrity. However, careful planning and an understanding of the trade-offs is necessary to ensure that this solution fits within your application requirements.