MongoDB: Setting an expiration time for a document (TTL index)

Introduction

MongoDB provides a powerful feature known as TTL (Time To Live) indexes that helps you manage the retention of documents within a collection. A TTL index is particularly useful when storing information that only needs to persist for a certain duration, such as cache data, session information, or logs. In this tutorial, we will explore how to set an expiration time for a document using TTL indexes, walking through basic to advanced examples.

Understanding TTL Indexes

Before diving into code examples, it’s vital to understand what TTL indexes are and how they work. A TTL index is created on a field that holds a BSON date type, and it dictates how long data should remain in the collection. After the specified expiration time has elapsed, MongoDB’s background task will automatically delete the document. It is an elegant solution for datasets with potentially self-removing documents without the need for custom scripts or manual intervention.

Creating a Basic TTL Index

To create a TTL index, you will first need a collection with a field that includes date information. Let’s create a simple example:

// Connect to the MongoDB database
const MongoClient = require('mongodb').MongoClient;
const url = 'mongodb://localhost:27017';
const client = new MongoClient(url);

async function createTTLIndex() {
  await client.connect();
  console.log('Connected to database!');

  const db = client.db('sampleDB');
  const collection = db.collection('events');

  // Create a TTL index
  await collection.createIndex({ 'expireAt': 1 }, { expireAfterSeconds: 3600 });

  console.log('TTL index created successfully');

  client.close();
}

createTTLIndex();

In the code above, we create an index on the ‘expireAt’ field of documents within the ‘events’ collection. The ‘expireAfterSeconds’ option dictates that each document should expire an hour (3600 seconds) after the time specified in the ‘expireAt’ field.

Inspecting a TTL Index

After creating a TTL index, you can confirm its creation and parameters by inspecting the indexes of a collection:

// Connect to the MongoDB database
async function checkTTLIndex() {
  await client.connect();
  const db = client.db('sampleDB');
  const collection = db.collection('events');

  const indexes = await collection.indexes();
  console.log(indexes);

  client.close();
}

checkTTLIndex();

This will print out a list of all indexes, including the TTL index you have created, to the console.

Inserting Expire-able Data

With the TTL index in place, you can now insert documents with an ‘expireAt’ field that uses MongoDB’s Date objects to define expiration times:

// Connect to the MongoDB database
async function insertDocumentWithExpiration() {
  await client.connect();
  const db = client.db('sampleDB');
  const collection = db.collection('events');

  await collection.insertOne({
    event: 'session_end',
    expireAt: new Date(Date.now() + 3600000), // The document will expire in one hour
  });

  client.close();
}

insertDocumentWithExpiration();

This insertion will include an ‘expireAt’ field that specifies exactly when the document should be removed from the collection.

Advanced TTL Index Usage

While TTL indexes are typically used with a single expiration value, you can also implement more complex scenarios. For example, you can set individual expiration times for each document at the time of insertion.

Here’s how:

// Connect to the MongoDB database
async function insertDocumentWithCustomExpiration() {
  await client.connect();
  const db = client.db('sampleDB');
  const collection = db.collection('events');

  await collection.insertMany([
    {
      event: 'cache_entry',
      expireAt: new Date(Date.now() + 300000), // Expires in 5 minutes
    },
    {
      event: 'user_session',
      expireAt: new Date(Date.now() + 86400000), // Expires in 1 day
    }
  ]);

  client.close();
}

insertDocumentWithCustomExpiration();

Each document has a distinct `expireAt` value setting varying expiration periods.

Modifying TTL Settings

After creating a TTL index, you may need to adjust its settings. To update the `expireAfterSeconds` value of an existing TTL index, use the `collMod` command with the `index` document:

// Connect to the MongoDB database
async function updateTTLExpiration() {
  await client.connect();
  const db = client.db('sampleDB');
  const collection = db.collection('events');

  await db.command({
    collMod: 'events',
    index: {
      keyPattern: { expireAt: 1 },
      expireAfterSeconds: 7200 // Update to expire the documents after 2 hours
    }
  });

  client.close();
}

updateTTLExpiration();

This command updates the expiration setting to 2 hours (7200 seconds) for all documents in the ‘events’ collection.

Handling Expired Documents

Expired documents are removed by a background thread that runs at a regular interval, not exactly at the expiration time. The precise timing of the deletion may vary, and the process is designed to avoid impacting the performance of the database.

Should you require immediate removal, a workaround is to explicitly delete documents past their expiration time:

// Connect to the MongoDB database
async function removeExpiredDocuments() {
  await client.connect();
  const db = client.db('sampleDB');
  const collection = db.collection('events');

  await collection.deleteMany({ expireAt: { $lte: new Date() } });

  client.close();
}

removeExpiredDocuments();

This command will delete documents whose ‘expireAt’ date is less than or equal to the current date and time.

Monitoring TTL Index Performance

When maintaining large databases with TTL indexes, monitoring their performance can help identify any lifecycle bottlenecks of data. MongoDB provides tools such as the MongoDB Atlas platform to assist in tracking and visualizing these metrics.

Conclusion

TTL indexes in MongoDB present an efficient method for handling temporary data. By automating data purging based on time, they simplify management and improve the overall performance of applications that deal with ephemeral or time-sensitive datasets.