Creating custom tokens in morgan: the NPM Logger

Morgan is one of npm’s popular module for logging HTTP requests, if you have worked on express logging you must have come across Morgan.

Honestly, it’s not a hard module to use, you can just begin using this in minutes by following this documentation and also it doesn’t come with too many customisations just with the basic stuff that we need to get started.

Using morgan is very easy and does its job well, you can check the below code for setting up the basic morgan

					

const express = require('express'),
morgan = require('morgan');

const app = express()
app.use(morgan('combined'))
app.get('/', (req, res) => {
  res.send('hello, world!')
})

app.listen(3002)

well, you can see how easy is to use morgan, hardly 10 lines (that includes code formatting too 🙂 ) and it will generate the below logs with all the basic request fields like Date, Http method, version, status code, content length, Browser details etc.

					

::1 - - [28/Jun/2020:17:55:30 +0000] "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:77.0) Gecko/20100101 Firefox/78.0"

But at times, customizing it is a lot difficult in ideal logging scenarios, the above fields are not sufficient and not in the right format too.  What if we need more dynamic fields like request headers, unique IDs associated with requests, the response time of APIs, different date format or possibly changing the format of logs itself in JSON for processing them in future.

To solve the above problems, we need to customize the default behaviour of Morgan, and this is what our blog is all about, so let’s get started with it.

Formatting the log output in morgan

Morgan gives you two possible ways to customize the log format where you can play around the exact details of logs you need.

Creating custom tokens

For creating custom tokens all you have to do is call morgan.token(name, callback)

The first argument is the name of the token for reference while logging.

Also, the second argument is the callback function with req, res objects which gets called all the time morgan logs anything, and we just have to return the value from this callback that we want to write to the log. Let’s see a simple example that will log the number of headers coming with every request.

					

morgan.token('req-headers-length', function(req, res, param) {
  return Object.keys(req.headers).length
});

To use the above token in morgan, just add the name of token in morgan object.

					

app.use(morgan(':req-headers-length'))

It will log the number of headers in request every time we get an API call.

Adding your custom token with morgan’s predefined tokens

					

app.use(morgan('(:method) :url => :status  header count => :req-headers-length'))

It will print the below output in the terminal every time we receive an API call in our node service.

					

(GET) / => 304  header count => 9

And what if we want this to be in JSON format?

Since morgan middleware expects a string of token, we can create a function and returns a JSON by stringifying it.

					

const getCustomMorganFormat = () =>
  JSON.stringify({
    method: ":method",
    url: ":url",
    http_version: ":http-version",
    response_time: ":response-time",
    status: ":status",
    content_length: ":res[content-length]",
    timestamp: ":date[iso]",
    headers_count: "req-headers-length",
});

And now we can call this function as morgan parameter.

					

app.use(morgan(getCustomMorganFormat()));

And we are done, it will start generating the logs in JSON format, like the example below.

					

{"method":"GET","url":"/","http_version":"1.1","response_time":"0.239","status":"304","content_length":"-","timestamp":"2020-06-28T14:24:57.696Z","headers_count":"9"}

Before winding up, one critical section of code, all the earlier snippets will write the logs in standard output(commonly your terminal window), we haven’t considered anything on customizing output location. As distributed systems are pretty much everywhere, logging to standard output is not at all suitable for a production instance it is as similar as console.log, then what’s the solution?

Morgan gives this option too, you can customize the path of logs using the streams, and to use that we need to require two module path for setting up file location and fs for writing to file.

					

const fs = require("fs"),
  path = require("path");

Now, create a new stream with the file name as access.log in append mode that will keep on adding data in the file

					

const httpLogStream = fs.createWriteStream(path.join(__dirname, "access.log"), {
  flags: "a",
});

And use the stream with our existing morgan object like this

					

app.use(morgan(getCustomMorganFormat(), { stream: httpLogStream }));

And we are done, every time our APIs get a hit it will add the logs in access.log, you can change the path of your file based on your requirements, for simplicity we have added in project directory itself.

And we are done 🙂
The code repo can be found here

Hope you enjoy reading and learned something out of it, feel free to add your questions or thoughts in the comments.