Week 10
Consistent Error Handling

Assignment Due

Assignment 3 - Authentication - is due before 10:30 am on Monday, March 22nd.
It is worth 10% of your final grade.

# Agenda

AMA (5 mins)
Quiz 7 - Authentication (20 mins)
Break (5 mins)
Enhanced Validation (40 mins)
Break (10 mins)
Error Handling Middleware (50 mins)
Break (10 mins)
Review Final Project (15 mins)
Lab Time

# Enhanced Validation

# Unique properties

Last time we looked at how to manually check if a property value, like email, has already been stored in the target collection. We used the Mongoose .countDocuments() method in the route handler. While it is important to understand how and why that works, there is a better way to handle this.

In a larger application, we may well have several properties with a unique: true constraint set in the schema, so a common middleware function would be a good idea. We also talked about the OOP Expert Principle and this kind of validation check really belongs in the Model class.

As it happens there is a Mongoose schema plugin made just for this use case. It is called mongoose-unique-validator (opens new window). Let's install it in our project.

npm install mongoose-unique-validator

Then we can implement it in the /models/User module. Import it at the top and then register the plugin right near the bottom, after all of the other schema definition code, but before creating and exporting the Model.

import uniqueValidator from 'mongoose-unique-validator'

// ... the rest of the code we already had

schema.plugin(uniqueValidator)

// register the plugin before these last two lines
const Model = mongoose.model('User', schema)
export default Model

The schema.plugin() method takes two arguments: the plugin module and an optional configuration object. Among other things, the configuration object let's us set a custom error message.

The message property can be set to either a string or a function that returns a string. Create a function that returns a specific message for email properties and a generic message for any other property with a unique: true schema setting.

schema.plugin(uniqueValidator, {
  message: props =>
    props.path === 'email'
      ? `The email address '${props.value}' is already registered.`
      : `The ${props.path} must be unique. '${props.value}' is already in use.`
})

Alternate Syntax

The code block above is combining arrow function syntax and the ternary operator. It could also be written out in more verbose syntax like this ...

schema.plugin(uniqueValidator, {
  message: function(props) {
    if (props.path === 'email') {
      return `The email address '${props.value}' is already registered.`
    } else {
      return `The ${props.path} must be unique. '${props.value}' is already in use.`
    }
  }
})

# Email address format

While we are looking at the email property, it would be nice if there was a simple way to validate that the given email address is in fact a correctly formatted email. We could go read the RFC spec and figure out a complicated RegEx comparison, but ... you guessed it. Somebody has already done the hard work.

There is a widely used NPM library called validator (opens new window) that among many other options, has a reliable method for checking email addresses. Go ahead and add it as a dependency for your project.

npm install validator

Don't forget to import validator at the top of your User model module.

Now you can use it in a custom validation method on your schema. According to the documentation (opens new window), Mongoose allows us to set a function as the value for the validate property, or if we want to set a custom error message, we can set an object with both a validator function that returns a boolean and a message function that returns a string. This is the approach that we will take.

const schema = new mongoose.Schema({
  // ...other props
  email: {
    type: String,
    trim: true,
    required: true,
    unique: true,
    validate: {
      validator: value => validator.isEmail(value),
      message: props => `${props.value} is not a valid email address.`
    }
  }
})

# Timestamps

Another common schema requirement is to have the application automatically set two timestamp properties on the document object: createdAt and updatedAt.

This is easily accomplished with Mongoose by setting timestamps: true in the optional second argument to the Schema constructor function. This options object (opens new window) has nearly 2 dozen settings to modify the default behaviour of our document schema.

Turn on the timestamps option (opens new window) for the User model.

const schema = new mongoose.Schema(
  {
    // all of the property definitions and validators
  },
  {
    timestamps: true
  }
)

# Setter Functions

It is a very common software design pattern to intercept values as they are assigned to an object's property, and do some extra processing on it before actually storing the final value. You can read more about using Setter Functions (opens new window) in plain JavaScript objects at MDN Docs.

The Mongoose schema definition makes it simple to define setter methods (opens new window) that can transform the input data before it is stored in MongoDB, or before using it in a query. This is better illustrated with a couple of examples.

# Normalize Email

When storing a user's email address, particularly if using it as their username for authentication, it is a good practice to always convert it lower case. e.g. "MICKEY.Mouse@Disney.com" should be stored and compared as "mickey.mouse@disney.com".

We can add a setter function to do this.

const schema = new mongoose.Schema({
  email: {
    type: String,
    trim: true,
    maxlength: 512,
    required: true,
    unique: true,
    set: value => value.toLowerCase(),
    validate: {
      validator: value => validator.isEmail(value),
      message: props => `${props.value} is not a valid email address.`
    }
  }
})

# Force Integer

The Number type in Mongoose, like JavaScript, can hold integers or floating point values. There are times when we need to be sure that the value stored is in fact an integer -- for example, when storing monetary values for a credit card payment system like Stripe.

We can add a setter function to always round up to the next integer so that "100.01" cents would be stored as "101" cents.

const schema = new mongoose.Schema({
  price: {
    type: Number,
    min: 100,
    required: true,
    set: value => Math.ceil(value)
  }
})

Break

Take a walk. Refill your coffee. We'll continue in 10 minutes.

# Express Error Handler Middleware

OK. So now you have better input validation in your Mongoose Models, but most of the code examples that we have seen so far are not giving the client helpful error messages. We have been masking them in generic "500 Server Error" messages.

It is time to fix that!

Express let's you define special error handler middleware to help clean up the route handlers and ensure consistent response formatting. When some other middleware function calls next(error) passing an error object as a argument, Express will short-circuit the request pipeline and jump to the first registered error handler middleware function.

Function Signature

Express error handler middleware functions look just like ordinary middleware, but with one extra argument – the err argument.

function handleError (err, req, res, next) { ... }

Great, but how does that help with our route handler functions? Well, remember when I told you that almost everything in Express is a middleware function? That includes route handlers. We just have to add the next argument to the function signature.

// this
router.post('/auth/users', async (req, res) => { ... })
// becomes
router.post('/auth/users', async (req, res, next) => { ... })

OK. OK. OK. Let's build the thing!

Create a new file called errorHandler.js in the /middleware folder and add this starter code that will implement a default "500 Server Error".

const formatServerError = function (err) {
  return [
    {
      status: '500',
      title: 'Server error',
      description: err.message || 'Please check the logs',
    },
  ]
}

export default function handleError (err, req, res, next) {
  const code = 500
  const payload = formatServerError(err)

  res.status(code).send({ errors: payload })
}

# Validation error

Now let's add in some logic to check for Mongoose validation errors (opens new window) and format them according to the JSON:API specification (opens new window).

Mongoose errors returned after failed validation contain an errors object whose values are ValidatorError objects. Each ValidatorError has kind, path, value, and message properties.

// An example Mongoose errors object might look like this

error = {
  // ... some other properties
  errors: {
    password: {
      message: `'password' is required`,
      kind: 'Invalid password',
      path: 'password',
      value: ''
    },
    email: {
      message: "The email address 'm.mouse@disney.com' is already registered.",
      kind: 'Invalid email',
      path: 'email',
      value: 'm.mouse@disney.com'
    }
  }
}

We need to iterate over those ValidatorError objects and transform them into the JSON:API format like that would look like this.

errors = [
  {
    status: '400',
    title: 'Validation Error',
    detail: "'password' is required",
    source: {pointer: '/data/attributes/password', value: ''}
  },
  {
    status: '400',
    title: 'Validation Error',
    detail: "The email address 'm.mouse@disney.com' is already registered.",
    source: {pointer: '/data/attributes/email', value: 'm.mouse@disney.com'}
  }
]

We can use JavaScript's Object.values() (opens new window) method to extract the various ValidatorError objects into an array and then use the Array.map() (opens new window) method to loop over them and return a new array of properly formatted error objects.

At the top of the module, create a helper function to do the formatting.

const formatValidationError = function(errors) {
  return Object.values(errors).map(e => ({
    status: '400',
    title: 'Validation Error',
    detail: e.message,
    source: {pointer: `/data/attributes/${e.path}`, value: e.value}
  }))
}

Now in your main error handler function you need to check if the error that was passed-in is a Mongoose validation error and if so, call the formatter function that you just created to set the payload variable. Otherwise, treat it as a generic 500 - Server Error.

export default function handleError (err, req, res, next) {
  const isValidationError = err?.name === 'ValidationError'
  const code = isValidationError ? 400 : 500
  
  let payload
  if (code === 400) {
    payload = formatValidationError(err.errors)
  } else {
    payload = formatServerError(err)
  }

  res.status(code).send({errors: payload})
}

Optional Chaining Syntax

The code above uses the ?. optional chaining operator (opens new window) that was added to JavaScript with ES2020. This significantly simplifies checking values of nested object properties, where the intermediate properties may not be set.

Without optional chaining you have have to explicitly check if the err object had a name property before checking its value. e.g.

const isValidationError = err.hasOwnProperty('name') && err.name === 'ValidationError'

Almost done. What about other kinds of errors — e.g. a resource not found error?

Let's add an option to the code assignment that checks if the err object has a code property. If it does use that instead of the default '500', and assume that it is a plain JavaScript error object which should be wrapped in an array.

const code = isValidationError ? 400 : err.code || 500

export default function handleError (err, req, res, next) {
  const isValidationError = err?.name === 'ValidationError'
  const code = isValidationError ? 400 : err.code || 500

  let payload = [err]
  if (code === 400) payload = formatValidationErrors(err.errors)
  if (code === 500) payload = formatServerError(err)

  res.status(code).send({ errors: payload })
}

# Refactor Router Modules

That is looking good, but we still need to update the route handlers to take advantage of this new capability. Refactor the "register user" route handler from last week.

# Before

// Register a new user
router.post('/users', sanitizeBody, async (req, res) => {
  try {
    let newUser = new User(req.sanitizedBody)
    const itExists = Boolean(await User.countDocuments({email: newUser.email}))
    if (itExists) {
      return res.status(400).send({
        errors: [
          {
            status: '400',
            title: 'Validation Error',
            detail: `Email address '${newUser.email}' is already registered.`,
            source: {pointer: '/data/attributes/email'}
          }
        ]
      })
    }
    await newUser.save()
    res.status(201).send({ data: newUser })
  } catch (err) {
    debug('Error saving new user: ', err.message)
    res.status(500).send({
      errors: [
        {
          status: '500',
          title: 'Problem saving document to the database.'
          description: err.message
        }
      ]
    })
  }
})

# Changes to this code

Update the function arguments to accept the next function being passed in.
Now that you are using the mongoose-unique-validator plugin, drop the unique validation check in the route handler.
Simplify the catch block by calling next(err) instead of defining and sending the error response here.

# After

Try it on your own before looking at the solution

// Register a new user
router.post('/users', sanitizeBody, async (req, res, next) => {
  try {
    let newUser = new User(req.sanitizedBody)
    await newUser.save()
    res.status(201).send({data: newUser})
  } catch (err) {
    debug('Error saving new user: ', err.message)
    next(err)
  }
})

Much simpler, but we're not quite done. There are two more steps.

# Error Logging

There is another repetitive part of error handling that we can eliminate. Notice that debug statement in the catch block. Instead of writing that out every time we are going to call next(err), we can create another error handler function to do this automatically.

This makes it much easier to change how you log these errors in the future. You may want to use a logger middleware like Winston (opens new window). Or you may want to use a third party service like bugsnag (opens new window), LogRocket (opens new window) or Rollbar (opens new window). It is much easier to change if the logging is being managed from a middleware function.

A simple implementation would be to use debug to write the error to the console and then call next(err) to pass control to the next error handler function. Create a new module called logErrors.js in the /middleware folder.

import createDebug from 'debug'
const debug = createDebug('errorLog')

export default function logError (err, req, res, next) {
  debug(err)
  next(err)
}

Now you can simplify your route handler even more ...

// Register a new user
router.post('/users', sanitizeBody, (req, res, next) => {
  new User(req.sanitizedBody)
    .save()
    .then(newUser => res.status(201).send({ data: newUser }))
    .catch(next)
})

We are down to six lines of code from the 31 that we started with!

That is more than an 80% reduction and a real win for readability and maintainability.

middleware for the win!

# Register the error handlers

Before you try to test this out, you need to put the two new middleware functions into action.

In the main application module, app.js, add the two new error handler middleware functions to the end of the request handling pipeline. You want to apply them globally, so there is no route path argument.

Important

Make sure that the error handlers middleware functions are registered last after all of the other middleware and route handlers.

import morgan from 'morgan'
import express from 'express'
import sanitizeMongo from 'express-mongo-sanitize'
import logError from './middleware/logError.js'
import handleError from './middleware/handleError.js'
import authRouter from './routes/auth/index.js'
import carsRouter from './routes/cars.js'
import peopleRouter from './routes/people.js'

import connectDatabase from './startup/connectDatabase.js'
connectDatabase()

const app = express()

// Middleware
app.use(morgan('tiny'))
app.use(express.json())
app.use(sanitizeMongo())

// Route handlers
app.use('/auth', authRouter)
app.use('/api/cars', carsRouter)
app.use('/api/people', peopleRouter)

// Error handlers
app.use(logError)
app.use(handleError)

export default app

TIP

The multiple imports in this module could be streamlined by setting-up re-exporters in the routes and middleware folders.

# Custom Exception Classes

For common errors that you might want to throw from several places in the application, you can extend the standard JavaScript Error class (opens new window).

# Resource not found

For example, you could standardize the error that is passed to your new error handler middleware for resource not found exceptions.

Create a new top level folder in your project called exceptions. Create a new module file in that folder called ResourceNotFound.js.

class ResourceNotFoundException extends Error {
  constructor(...args) {
    super(...args)
    Error.captureStackTrace(this, ResourceNotFoundException)
    this.code = 404 // this will trigger correct handling by our middleware
    this.status = '404'
    this.title = 'Resource does not exist'
    this.description = this.message
  }
}

export default ResourceNotFoundException

Stack trace API

This is available only in the V8 runtime engine used in Node.js and the Chrome browser. You can read more detail about the Stack trace API (opens new window) in the V8 Dev Docs.

With that new custom exception defined, you can refactor the implementation of your route handler code from this ...

router.get('/:id', async (req, res) => {
  try {
    const car = await Car.findById(req.params.id).populate('owner')
    if (!car) {
      throw new Error('Resource not found')
    }
    res.send({ data: car })
  } catch (err) {
    sendResourceNotFound(req, res)
  }
})

to this ...

import ResourceNotFoundError from '../exceptions/ResourceNotFound'

router.get('/:id', async (req, res, next) => {
  try {
    const car = await Car.findById(req.params.id).populate('owner')
    if (!car) {
      throw new ResourceNotFoundError(`We could not find a car with id: ${req.params.id}`)
    }
    res.send({ data: car })
  } catch (err) {
    next(err)
  }
})

You no longer need the sendResourceNotFoundError() helper function at the bottom of your route handler modules and you will no longer accidentally mask other errors by allowing the error handler middleware to correctly format and return the errors.