Audit logs for Firestore documents

Audit logs for Firestore documents

How to add changelogs aka audit logs to any Cloud Firestore document.

If you ever built applications with multiple user roles and permissions, where the major part of your application was about settings and configurations, chances are that you might have already implemented or worked on a feature request to implement a changelog of all the changes that are happening in your app, answering the most important bits, what changed, when the changes took place and who made those changes. And even otherwise, for any application that stores and modifies data over time (which, by the way is almost every application with user interaction), it's important to store the logs about what changes are happening in your data. These logs could be used in future for all sorts of audits and bookkeeping purposes and even for data rollbacks, log dashboards, etc.

Storing a change snapshot for all atomic changes to the database records, is important for all applications.

Though, implementing these logs in traditional SQL databases are not that straightforward, if you happen to use any NoSQL databases, chances are that you can store data snapshots for your before and after changes to your data, without breaking a sweat, and thus implementing the very rudimentary changelog of your database. As mentioned earlier, there are 3 important pieces of information, what, when and who, that we are interested in whenever a change occurs. Let's dig a bit deeper into how we are going to proceed in order to make these information available.

  • What – To be able to provide information about what changes occurred in our data, we need to store snapshots of before and after values with every change that occur for a record aka document (for most common cloud NoSQL solutions, we call a data record as document) in our table aka collection.
  • When – This information is relatively easier to get, as all we need is the timestamp when the change occurred.
  • Who – Now comes the tricky part, as, this information is something that's not completely identifiable by the atomic database operations. We need the application to pass in this information whenever an update operation is performed. As a good practice, we would always include the updatedBy field that would contain the userId of the user who requested to perform the update operation to the document.

With above information in place, let's find out how we would implement an audit log for our Cloud Firestore documents.


Some background to facilitate the setup

  • Firebase lets you invoke a Cloud Function depending upon the defined Cloud Firestore triggers.
  • When the function is invoked due to an update to the document, it contains a snapshot of before and after changes to the document.

Steps

  1. Create a generic createDocument function to create a document in cloud firestore. Note: this step is optional and you can use the default way of creating new documents, however I prefer this cleaner abstraction.
const admin = require('firebase-admin')
const db = admin.firestore()

/**
 * Creates a new document in the Firebase collection.
 * Additionally appends `uid` and `createdAt` fields
 * to the data. Once the document has been created,
 * it returns the final document object that has been stored
 * in the firestore collection.
 *
 * @param {String} collection Firebase collection name
 * @param {Object} data The document object data
 * @returns {Object} The document object with its uid field
 */
module.exports = async function createDocument(collection, data) {
  if (!collection || !data) {
    console.log('skip document creation, no collection or data provided')
    return null
  }

  const newDocumentRef = db.collection(collection).doc()
  const uid = newDocumentRef.id
  const createdAt = Date.now()
  const finalData = {...data, uid, createdAt}

  console.log('start writing data at path', `${collection}/${uid}`)
  await newDocumentRef.set(finalData)
  console.log('end writing data at path', `${collection}/${uid}`)

  return finalData
}
createDocument.js to be used for the firebase cloud functions

2.   Make sure to always include the updatedBy field with the value of the userId of the user requesting the change in your application. This information needs to be added in application level code and would differ depending upon where you are updating the document from. If you use the firebase web sdk, you can write a wrapper updateDocument function to always add this information when updating your document.

// general firebase initialization code for the web
// this may differ depending upon your web project setup
import firebase from 'firebase/app'
import 'firebase/firestore'
import 'firebase/auth'
const app = firebase.initializeApp(ENV.FIREBASE)
const db = firebase.firestore(app)
const auth = firebase.auth(app)

/**
 * Updates the document with the provided data.
 * Additionally appends the `updatedAt` and `updatedBy`
 * fields to the document.
 *
 * @param {String} collection Firebase collection name
 * @param {String} id Firebase document id
 * @param {Object} data Updates that needs to be applied
 * @returns {Object} Applied updates to the document
 */
export default async function updateDocument(collection, id, data) {
  if (!collection || !id || !data) return null

  const documentRef = db.doc(`${collection}/${id}`)
  const updatedAt = Date.now()
  const updatedBy = auth.currentUser ? auth.currentUser.uid : null

  const updates = {...data, updatedAt, updatedBy}

  await documentRef.update(updates)

  return updates
}
updateDocument function to be used for the web client

3.   Create a generic function to add changelog to any document. This will make use of the fact that in cloud firestore you can create a subcollection for each of your cloud firestore document.

// use the wrapper method that we created earlier
// you can replace this with raw firebase calls if you like
const createDocument = require('./createDocument')

/**
 * The method allows enabling changelog for any document
 * in firestore collection. The ideal usage is adding this
 * to the cloud function trigger for the document
 * where the changelog needs to be added.
 *
 * @example
 *  await addChangelogToDocument({
 *    documentPath: 'orders/xxxx',
 *    before: {...}, // data before
 *    after: {...}, // data after
 *  })
 *
 * // when using cloud functions
 * exports.onOrderUpdate = functions.firestore
 *   .document('orders/{orderId}')
 *   .onUpdate(async (change, context) => {
 *     const {orderId} = context.params
 *     const documentPath = `orders/${orderId}`
 *     const before = change.before.data()
 *     const after = change.after.data()
 *
 *     try {
 *       await addChangelogToDocument({documentPath, before, after})
 *     } catch (err) {
 *        // log errors
 *     }
 *  }
 *
 * @param {String} documentPath The root path of the document for tracking changelog
 * @param {Object} before The data before the changes have been written
 * @param {Object} after The data after the changes have been written
 * @param {String} collectionName The sub collection name for writing the changelog to, defaults to `changelog`
 */
module.exports = async function addChangelogToDocument({
  documentPath,
  before,
  after,
  collectionName = 'changelog'
}) {
  if (!documentPath || !collectionName) {
    console.log('skip writing, no document path provided')
    return null
  }

  const collectionPath = `${documentPath}/${collectionName}`
  const updatedByUserId = after.updatedBy
  let user = null

  // if available, also write the user information for the log
  if (updatedByUserId) {
    // this assumes that you have a method `getUserById`
    // to get any user details by the provided userId
    // the actual implementation would differ based on
    // your app setup and the data model used for storing user
    try {
      console.log('start fetching user details', updatedByUserId)
      const userDetails = await getUserById(updatedByUserId)
      console.log('end fetching user details', updatedByUserId)

      if (userDetails) {
        user = {
          name: userDetails.name || null,
          email: userDetails.email || null,
          uid: userDetails.uid || null,
        }
      }
    } catch (err) {
      console.log('error setting user detail', err)
    }
  }

  const data = {before, after, user}

  console.log('start writing changelog at path', collectionPath)
  await createDocument(collectionPath, data)
  console.log('end writing changelog at path', collectionPath)
}
addChangelogToDocument.js file to be used for the firebase cloud functions

4.   Create the cloud functions trigger for the document path where you want to add this changelog.

const functions = require('firebase-functions')
const addChangelogToDocument = require('./addChangelogToDocument')

// for example the following code is used
// to add changelog to all orders
// that are stored inside of `orders` collection
exports.onOrderUpdate = functions.firestore
  .document('orders/{orderId}')
  .onUpdate(async (change, context) => {
    const {orderId} = context.params
    const documentPath = `orders/${orderId}`
    const before = change.before.data()
    const after = change.after.data()

    try {
      await addChangelogToDocument({documentPath, before, after})
    } catch (err) {
      console.error(
        'error adding changelog to document',
        documentPath,
        err
      )
    }
  })
index.js file inside the functions folder

5.   And finally, deploy the cloud functions by running firebase deploy --only functions from the command-line.

Congratulations! You have successfully added a changelog to your document which you can verify from the firebase database console.

Now whenever you make an update to your specified document, you'd see that a timed changelog is added to the document within the changelog subcollection. This subcollection could then be used for any sort of bookkeeping or feature implementation in the future.


If you happen to run into issues or spot any mistakes with this article, please feel free to comment and I'd try my best to help you out / correct the mistakes.

Happy Coding!