Eflintdev

Page Content Section

HomeContentNodeJS REST API using MongoDB, AWS Lambda, and Cognito

NodeJS REST API using MongoDB, AWS Lambda, and Cognito

Using Serverless Node with MongoDB
Node REST API feature

Hydrating the FrontEnd Using a Serverless Stack!

Read more about developing serverless APIs for use in web applications, using Node JS, NoSQL, and other AWS Services.

The Details

One of the greatest benefits of using cloud infrastructure is that it offers several options for providing data to applications. For instance, with Amazon Web Services it's possible to utilize Amazon Elastic Compute Cloud (EC2) for setting up a host of services or REST APIs. However, in some cases, simply being able to provide data through an event-driven function-based microservice could suffice for many reasons.

This is where Node JS becomes quite handy because it's a powerful JavaScript runtime environment that can be used to develop REST APIs through a variety of techniques.

The Data Flow

With the case of eflintdev.com (this site), I've provided it data using a Node API that is provisioned through serverless microservices. The serverless API runs on Node JS v. 18+ and the data is sourced from a MongoDB database. Below we have a diagram which illustrates all the major components of this setup:

AWS Cognito Is In There

You probably noticed the inclusion of AWS Cognito in the previous data flow diagram, and this is especially vital for protecting the routes that either write, update, or delete. Such is achieved using JWT based tokens provided through a confidential client auth flow for authentication. Also, though it's integral to the permissions scheme, AWS IAM (Identity Access Management) was excluded from the diagram for the sake of simplicity. But IAM permeates the cloud infrastructure and works to secure or provide access control for the components in this setup.

// Snippet from serverless.yml provider.apiGateway section ... apiGateway: restApiId: ${env:AWS_GTWAY_ID} restApiRootResourceId: ${env:AWS_GTWAY_RESOURCE_ID} ...

AWS API Gateway REST API - Good-to-go!

It's possible to provide access to an externally or preconfigured API Gateway REST API using the provider.apiGateway data in the serverless.yml file, as I've done.

// .env file # AWS PROFILE VARS PROFILE=myxaws-serverless-profile # MONGODB VARS MONGO_URI=mongodb+srv://siteproject:RkUbdLfomFMrX8jq@atlascluster.w58uxfg.mongodb.net/?retryWrites=true&w=majority&appName=PlutoCluster MONGO_PORT=8080 # MongoDB Collections ARTICLES_COLLECTION=Article ARTICLES_TEST_COLLECTION=ArticlesTest # MongoDB TimeOut MONGO_SERVER_SEL_TIMEOUT=3000 # SERVERLESS VARS AWS_REGION=us-east-1 SERVERLESS_ROLE=arn:aws:iam::45623340075:role/actse-sentinel-role DEPLOY_BUCKET=flint-serverless-bin AWS_GTWAY_NAME=RXGateway AWS_GTWAY_ID=2305fgks56 AWS_GTWAY_RESOURCE_ID=rwoptylsjg AWS_GTWAY_ID=powldgefisl AWS_ACCESS_KEY_ID=YPSFGAETIDSVBW AWS_SECRET_ACCESS_KEY=999999334ekfghgjjss000eer/dffe # COGNITO VARS COGNITO_AUTH_URL=https://cognito-idp.us-east-1.amazonaws.com/us-east-1_mmcsjktylwr COGNITO_URL=https://cognito-idp.us-east-1.amazonaws.com/us-east-1_mmcsjktylwr COGNITO_CLIENTID=3dfkssldffldgglesdkfe COGNITO_CLIENTNAME=ApplicationPool COGNITO_UPOOLNAME=ApplicationPool

Environment Variables from .env

This is a pseudo version of the .env file that I used for this project, and the real version is secured within Ansible vault. When deploying with Ansible the environment variables can be sourced from Ansible vault, and the same is true for using the Ansible agent when deploying with Jenkins.
# Serverless YAML Excerpt (microservice functions for http requests) ... plugins: - serverless-dotenv-plugin - serverless-api-gateway-caching - serverless-plugin-typescript ... functions: getArticles: handler: getArticles.handler events: - http: path: /articles method: get authorizer: arn: arn:aws:cognito-idp:${env:AWS_REGION}:${env:COGNITO_USERPOOL} caching: enabled: true cacheKeyParameters: - name: request.querystring.id createArticle: handler: createArticle.handler events: - http: path: /articles method: post authorizer: arn: arn:aws:cognito-idp:${env:AWS_REGION}:${env:COGNITO_USERPOOL} deleteArticle: handler: deleteArticle.handler events: - http: path: /articles method: delete authorizer: arn: arn:aws:cognito-idp:${env:AWS_REGION}:${env:COGNITO_USERPOOL} updateArticle: handler: updateArticle.handler events: - http: path: /articles method: patch authorizer: arn: arn:aws:cognito-idp:${env:AWS_REGION}:${env:COGNITO_USERPOOL} ...

Serverless YAML Configuration

After having sourced the environment variables I defined the serverless functions in the serverless.yml file. Among the many configurations there is the provisioning of authorizers via an AWS Cognito user pool (confidential client) setup. Also notice that caching is enabled on the GET route, for achieving better load times in the frontend. Later on, I set the throttling (rate) to 120 and the bursts were reduced to 480 for the GET request.

AWS Gateway in AWS Management Console

Once deployed the stages and methods fall into place and can be tweaked inside of the AWS Management Console if desired. Otherwise, a developer can continue to stack on the settings in the serverless.yml.

Ensuring Data Structure(s) with Mongoose

The data for this project has a specific data structure that is modeled using Mongoose Schema. This made it possible to also configure some type validation measures, so that data is not added to the MongoDB database without meeting some specific requirements.

// Article schema using Mongoose const articleSchema = new mongoose.Schema({ active: { type: Boolean, required: true }, title: { type: String, required: true, unique: true }, name: { type: String, required: false }, slug: { type: String, required: false }, description: { type: String, required: false }, content: [contentSchema], imageGrid: [imageGridSchema] });
// Article.ImageGrid sub-schema using Mongoose const imageGridSchema = new mongoose.Schema({ title: { type: String, required: false }, href: { type: String, required: false }, content: { type: String, required: false }, imgAltText: { type: String, required: false }, backgroundImage: { type: String, required: false }, hardCodedImage: { type: String, required: false } });

Mongoose is also featured with the means to allow developers to create custom validators. This utility and middleware affords a few more helpful utilities for customizing how data is accessed, or delivered from a database/ collection:

Indexes

  • Indexes
  • Counters
  • Custom Field Selection
  • Custom Field Exclusion

GET, POST, PATCH, and DELETE

The four necessary requests were developed using Mongoose for CRUD operations. I've provided some code snippets for the GET and POST requests.

GET

These are the GET request examples in both JavaScript and TypeScript.
// GET Request in JavaScript /** * Helper function for retrieving all articles. * @param {Connection} conn - The mongoose connection instance. * @param {APIGatewayProxyCallback} callback - The API Gateway Proxy Callback. * @returns Promise<unknown> */ exports.getArticles = async (conn, callback) => { const response = await conn.model(process.env.ARTICLES_COLLECTION, articleSchema).find({}) .then(result => result) .then(res => { return callback(null, { statusCode: 200, headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ data: res }) }); }) .catch(error => error ? callback(null, { statusCode: 500, headers: {}, body: JSON.stringify({ data: `Error while retrieving data!: ${error}` }) }) : null) .finally(async () => { conn = null; return await mongoose.connection.close(); }); return response; }
// GET Request in TypeScript /** * Gets articles. * @param conn - The mongoose connection instance. * @param callback - The API Gateway Proxy Callback * @returns Promise<unknown> */ export const getArticles = async (conn: Connection, callback: APIGatewayProxyCallback): Promise<unknown> => { const response = await conn.model((process.env.ARTICLES_COLLECTION as string), articleSchema).find({}) .then((res: mongoose.Document<unknown, object, Article>[]) => { return callback(null, { statusCode: 200, headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ data: res }) }); }) .catch((error: Error) => error ? callback(null, { statusCode: 500, headers: {}, body: JSON.stringify({ data: `Error while retrieving data!: ${error}` }) }) : null) .finally(async () => { (conn as unknown) = null; return await mongoose.connection.close(); }); return response; }

POST

These are the POST request examples in both JavaScript and TypeScript.
// POST Request in JavaScript /** * Creates an article. * @param {APIGatewayEvent} event - The mongoose connection. * @param {Context} context - AWS Lambda Context. * @param {APIGatewayProxyCallback} callback - The API Gateway Proxy Callback * @returns Promise<unknown> */ exports.handler = async (event, context, callback) => { let conn = null; let newArticle = null; if (conn === null) { conn = await mongoose.createConnection(process.env.MONGO_URI, { serverSelectionTimeoutMS: process.env.MONGO_SERVER_SEL_TIMEOUT }) .asPromise(); } const articleSetup = conn.model(process.env.ARTICLES_COLLECTION, articleSchema); newArticle = new articleSetup(event.body); const insertArticle = Promise.resolve(newArticle); const response = await insertArticle.then((result) => { return result.save(); }) .then(res => callback(null, { statusCode: 201, headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ data: res }) })) .catch((error) => { callback(null, { statusCode: 200, headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ data: `There was an error while creating the Article: ${error}` }) }); }) .finally(async () => { await mongoose.connection.close(true); conn = null; }); return response; };
// POST Request in TypeScript /** * Creates an article. * @param event - API Gateway Event. * @param context - AWS Lambda Context. * @param callback - The API Gateway Proxy Callback * @returns Promise<unknown> */ export const handler = async (event: APIGatewayEvent, context: Context, callback: APIGatewayProxyCallback): Promise<unknown> => { let conn!: Connection; let newArticle = null; if (conn === null) { conn = await mongoose.createConnection((process.env.MONGO_URI as string), { serverSelectionTimeoutMS: Number(process.env.MONGO_SERVER_SEL_TIMEOUT) }) .asPromise(); } const articleSetup = conn.model((process.env.ARTICLES_COLLECTION as string), articleSchema); newArticle = new articleSetup(event.body); const insertArticle = Promise.resolve(newArticle); const response: unknown = await insertArticle.then((result) => { return result.save(); }) .then(res => callback(null, { statusCode: 201, headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ data: res }) })) .catch((error: Error) => { callback( null, { statusCode: 200, headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ data: `There was an error while creating the Article: ${error}` }) }); }) .finally(async () => { await mongoose.connection.close(); (conn as unknown) = null; }); return response; };

Unit Tests Examples

Unit testing was implemented for each route using JEST. It has some great support for testing Promises, which is the primary coding pattern for all the http requests in this API.

GET

These are the GET request test examples in both JavaScript and TypeScript.
// GET Request Unit Tests in JavaScript const { proxyCallback } = require('../__mocks__/callback'); const getHandler = require('../getArticles'); const mongoose = require('mongoose'); const dotenv = require('dotenv'); dotenv.config(); describe('Testing GET routes', () => { let connection; let timeout = 6000; beforeAll(async () => { connection = await mongoose .createConnection(process.env.MONGO_URI, { serverSelectionTimeoutMS: process.env.MONGO_SERVER_SEL_TIMEOUT }) .asPromise(); }, timeout); afterAll(async () => { await mongoose.connection.close(); }, timeout); it('should execute getArticles and return status code of 200', async () => { const result = await getHandler.getArticles(connection, proxyCallback); expect(result.data.statusCode).toBe(200); }, timeout); it('should execute getArticleById and return status code of 200', async () => { const result = await getHandler.getArticleById(connection, '1', proxyCallback); expect(result.data.statusCode).toBe(200); }, timeout); });
// GET Request Unit Tests in TypeScript import { getArticleById, getArticles } from "../getArticles"; import { proxyCallback } from "../__mocks__/callback"; import { AwsResponseData } from "../../helpers/custom-types"; describe('Testing GET routes', () => { let connection: Connection; let timeout = 6000; beforeAll(async () => { connection = await mongoose .createConnection((process.env.MONGO_URI as string), { serverSelectionTimeoutMS: Number(process.env.MONGO_SERVER_SEL_TIMEOUT) }) .asPromise(); }, timeout); afterAll(async () => { await mongoose.connection.close(); }, timeout); it('should execute getArticles and return status code of 200', async () => { const result: any = await getArticles(connection, proxyCallback); expect((result as AwsResponseData).data.statusCode).toBe(200); }, timeout); it('should execute getArticleById and return status code of 200', async () => { const result: any = await getArticleById(connection, '1', proxyCallback); expect((result as AwsResponseData).data.statusCode).toBe(200); }, timeout); });

Deployment

It was easy to deploy this serverless project using the Linux terminal. Moreover, it's also convenient to test the serverless functions using the command line as well. However, as mentioned previously, Jenkins can be utilized with Ansible for deploying serverless applications, as well. Ideally, this should be done using an Ansible role, a working Jenkins setup, and Ansible Agent installed in Jenkins.

// deploy command serverless deploy --stage production // local testing command serverless invoke local --function getArticles --data '{"queryStringParameters":{"id":"546ad1rd5fbq40480eg91cd3"}}'
// Snippet from an Ansible role for deploying serverless apps. ... - name: Deploy Serverless (Basic) community.general.serverless: service_path: '{{ aws_serverless_dir }}' state: present - name: Deploy Serverless and receive resource details in Ansible community.general.serverless: stage: '{{ deployment_stage }}' region: '{{ aws_region }}' service_path: '{{ aws_serverless_dir }}' register: sls ...