Using redis cache in Node application featured image
UI Development

Using Redis cache in Node applications

This article will explain about the cache and the situation where cache can be used and how we will be using Redis as cache in our Node applications.

What is a caching and when we need to use it ?

Caching is a process of storing a copy of data in cache or temporary storage location, so that it can be accessed more quickly when we need it. This will improve the application performance a lot and reduce the burden on the database and therefor the APIs.

Imagine you have a server and you have a database or API in another server from which you will pull out the data. When the user request the server for data. The server will pull our data from the database or API. If we’ve caching implemented then server will cache the data in cache server and wont request database again and again.

There are two main scenarios where we’d want to use a cache.

  • To save network calls to database or API. Once we are querying for a few commonly used data and often modified data, then this data should be cached.
  • To avoid repeated computation in database. When we need data from the database which is  generated after complex computations. This will be very helpful to decrease the load on the database or API.

 

Cache is extremely useful in application which uses distributed server setup. The infrastructure which we used for cache database are going to be very expensive when put next to normal database. So we must always use cache carefully, we should not cram everything into the cache. If we save all the information in cache it’ll degrade the performance of the application.

Redis

Redis is an open source BSD licensed database. It is in-memory data structure store used as a database, cache and message broker. Out of the box, a Redis instance supports 16 logical databases. These databases are effectively siloed far from each other, and once you run a command in one database it doesn’t affect any of the data stored in other databases in your Redis instance. By default 0 database is chosen once you first hook up with Redis. It supports most used data structures like strings, hashes, lists, sets etc… Redis has the most useful caching’s features in-built, some of them are Lua script, LRU eviction, transactions and different levels of on-disk persistence and provides high availability.

Redis has bunch of features which are able to leverage those features using commands. We can find all the list of commands here with clear description. Redis has good support of client libraries for several programming languages. We can see the whole list of clients here. There are good set of various libraries for Node application as well, among them the foremost popular and recommend libraries are node_redis and ioredis. Here we are going to use node_redis.

Installation

Installation of Redis in Mac and Linux based OS are simple when compared to installation in Windows.

In Mac

In Mac we can install Redis using Homebrew, If you dont have Homebrew installed on your Mac, you can run the following command on your terminal to install it.

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

After the installation you can install Redis by using below command.

brew install redis

In Windows

For Windows users you can use this short guide from redislabs.com.

After the installation of Redis in Mac or Windows you can run the Redis server using below command. If you install it as a service then you don’t need it as it will be running by default.

redis-server

Make sure the Redis server is running, then you can run Redis CLI using below command.

redis-cli

Node application

As mentioned already, we will create a simple Node application that will retrieve the list of post offices information using it’s pin-code.

Using below bash command just create a simple node application.

npm init -y
npm install --save express node-fetch redis
npm install --save-dev nodemon

Node developers may already be familiar with expressnode-fetch and nodemon packages. When it comes to redis package, this will install node-redis which we already mentioned previously. You can find more details on node-redis here.

After you ran above bash command, you can create root index.js file and update below start command in the package.js file.

"scripts": {
    "start": "nodemon index"
}

Below is the code snippet to setup the basic skeleton of node application running with express and simple HTML page for testing the API which we build.

const express = require('express');
const path = require('path');
const fetch = require('node-fetch');
const redis = require('redis');
 
const app = express();
const router = express.Router();
 
router.get('/', function (req, res) {
    res.sendFile(path.join(__dirname + '/views/home.html'));
    //__dirname : It will resolve to your project folder.
});
 
//add the router
app.use('/', router);
app.listen(PORT);
 
console.log(`Running at Port ${PORT}`);

You can run and verify the application with the start command using “npm start” in bash.

Now we can just create a request handler for fetching the post office details and adding the handler to express route.

async function getPincodeInfo (req, res, next) {
    try {
        console.log('Fetching Postal Office details...');
        const { pincode } = req.params;
        const response = await fetch(`https://api.postalpincode.in/pincode/${pincode}`);
        const data = await response.json();
        const postalInfo = data[0];
        res.send(postalInfo);
    } catch (err) {
        console.error(err);
        res.status(500);
    }
}
 
app.get('/post-office/:pincode', getPincodeInfo);

We can run and verify, if everything working fine. I just tested with pin-code 516002. Below is the JSON result.

{Message: "Number of pincode(s) found:14", Status: "Success", PostOffice: [,…]}
Message: "Number of pincode(s) found:14"
Status: "Success"
PostOffice: [,…]
0: {Name: "Buddayapalli", Description: null, BranchType: "Branch Post Office", DeliveryStatus: "Delivery",…}
1: {Name: "Chinnachowk", Description: null, BranchType: "Branch Post Office", DeliveryStatus: "Delivery",…}
2: {Name: "Degalavandlapalli", Description: null, BranchType: "Branch Post Office",…}
3: {Name: "Jayanagar (Cuddapah)", Description: null, BranchType: "Sub Post Office",…}
4: {Name: "Lingampalli", Description: null, BranchType: "Branch Post Office", DeliveryStatus: "Delivery",…}
5: {Name: "Machupalli", Description: null, BranchType: "Branch Post Office", DeliveryStatus: "Delivery",…}
6: {Name: "Patha Cuddapah", Description: null, BranchType: "Branch Post Office",…}
7: {Name: "Ramanjaneyapuram", Description: null, BranchType: "Branch Post Office",…}
8: {Name: "Ramarajupalli", Description: null, BranchType: "Branch Post Office",…}
9: {Name: "RIMS", Description: null, BranchType: "Sub Post Office", DeliveryStatus: "Non-Delivery",…}
10: {Name: "RTC Work Shop", Description: null, BranchType: "Branch Post Office",…}
11: {Name: "Shankarapuram (Cuddapah)", Description: null, BranchType: "Sub Post Office",…}
12: {Name: "Takkolu", Description: null, BranchType: "Branch Post Office", DeliveryStatus: "Delivery",…}
13: {Name: "Ukkayapalle", Description: null, BranchType: "Branch Post Office", DeliveryStatus: "Delivery",…}

Now we implement cache for the above sample. Create redis client instance to connect to database. In creating the instance we need to set the port, host and database.

const PORT = process.env.PORT || 5000;
const REDIS_PORT = process.env.REDIS_PORT || 6379; // Default port of Redis, you can change this
const REDIS_HOST = process.env.REDIS_HOST || '127.0.0.1'; // Host where Redis was installed, currently its in local
const REDIS_DB_NUMBER = process.env.REDIS_DB_NUMBER || 0; // 0 is default database
 
const cacheClient = redis.createClient({ port: REDIS_PORT, host: REDIS_HOST });
cacheClient.send_command('SELECT', [REDIS_DB_NUMBER]); // Redis has 0-15 databases, selecting db number

In the above code snippet, we select database 0 which is default database. We can switch from one database to another database with SELECT command. To run the SELECT command through node-redis client, you need to use .send_command(‘COMMAND NAME’, [parms]) method. Now we will implement cache as middleware and the code looks like this.

// Cache middleware
function cache (req, res, next) {
    const { pincode } = req.params;
 
    cacheClient.get('PINCODE-' + pincode, (err, data) => {
        if (err) throw err;
 
        if (data !== null) {
            res.send(JSON.parse(data));
        } else {
            next();
        }
    });
}

We need add this middleware to express route and add the save cache logic in previous getPincodeInfo handler. Pulling all logic together, the entire code is as shown below.

const express = require('express');
const path = require('path');
const fetch = require('node-fetch');
const redis = require('redis');
 
const app = express();
const router = express.Router();
const PORT = process.env.PORT || 5000;
const REDIS_PORT = process.env.REDIS_PORT || 6379; // Default port of Redis, you can change this
const REDIS_HOST = process.env.REDIS_HOST || '127.0.0.1'; // Host where Redis was installed, currently its in local
const REDIS_DB_NUMBER = process.env.REDIS_DB_NUMBER || 0; // 0 is default database
 
const cacheClient = redis.createClient({ port: REDIS_PORT, host: REDIS_HOST });
cacheClient.send_command('SELECT', [REDIS_DB_NUMBER]); // Redis has 0-15 databases, selecting db number
 
async function getPincodeInfo (req, res, next) {
    try {
        console.log('Fetching Postal Office details...');
        const { pincode } = req.params;
        const response = await fetch(`https://api.postalpincode.in/pincode/${pincode}`);
        const data = await response.json();
        const postalInfo = data[0];
 
        // Set data to Redis
        cacheClient.setex('PINCODE-' + pincode, 3600, JSON.stringify(postalInfo));
 
        res.send(postalInfo);
    } catch (err) {
        console.error(err);
        res.status(500);
    }
}
 
// Cache middleware
function cache (req, res, next) {
    const { pincode } = req.params;
 
    cacheClient.get('PINCODE-' + pincode, (err, data) => {
        if (err) throw err;
 
        if (data !== null) {
            res.send(JSON.parse(data));
        } else {
            next();
        }
    });
}
 
app.get('/post-office/:pincode', cache, getPincodeInfo);
 
router.get('/', function (req, res) {
    res.sendFile(path.join(__dirname + '/views/home.html'));
    //__dirname : It will resolve to your project folder.
});
 
//add the router
app.use('/', router);
app.listen(PORT);
 
console.log(`Running at Port ${PORT}`);

In the above code snippet we used .SETEX() method to save the data into Redis. This will take there parameters, first param is the key, second param is expiry time in seconds and the third one is the value in string format. So we stringify the JSON object before saving. Then we use .GET() method to fetch the data from the cache.

Now, let’s run the application and see the total time of the both requests with and without cache, you will be able to see lot of difference.

There are some more useful commands which helps to verify the data in Redis using redis-cli.

# To switch from one database to another database, below we have selected database 1
>> select 1
>> OK
 
# Save the data with key
>> set MyKey "Hello !!!"
>> OK
 
# To get all the saved keys
>> keys *
>> 1) MyKey
 
# Get the data with key
>> get "MyKey"
>> "Hello !!!"
 
# Delete the data with key, returns number 1 confirming the key value deleted, else will return 0
>> del "MyKey"
>> (integer) 1
 
# SetEx to save data with timeout
>> setex MyKey 3600 "Hello !!!"
>> OK
 
# After running SETEX command you can verify how much time left for the key to expire. This will return the duration in seconds
>> TTL MyKey
>> (integer) 3598
 
# To clear all the data in the current database
>> flushdb
>> OK
 
# To clear all the data from all the databases
>> flushall
>> OK

Conclusion

Using cache will boost the appliance performance but we should always be conscious in choosing the information we cache. The infrastructure which we use for cache server will be costlier and if we fill everything in cache, then the search time will increase so that leads to degradation of performance which makes lesser sense to use cache. When we are using cache with should think about two things. When should we make an entry into the cache and when do we evict the data from cache. The process of entry and evicting data in cache is called cache policy. Cache performance is completely dependent on cache policy. There are multiple policies in cache and LRU is the most popular cache policy. LRU means Least Recently Used, what it says is when ever you make an entry put that on the top and when ever you evict data then do it from the bottom. Redis has LRU feature in-build that is one of the reason we are use it for caching.

Happy coding !!!

 

About The Author