Restful API with NodeJS, Express, PostgreSQL, Sequelize, Travis, Mocha, Coveralls and Code Climate.
NodeJS is becoming a backend language of choice for many developers.
In this article, I am going to work you through building a Simple Restful API with:
- NodeJS — For writing Javascript server-side applications
- Express — A NodeJS framework
- PostgreSQL — An open source object-relational database
- Sequelize — An ORM(Object Relational Mapping) of PostreSQL
- Travis —A continuous integration service for Testing Applications
- Coveralls — A web service to help you track your code coverage over time.
- Code Climate — Provides automated code review for maintainability and test coverage.
- Babel — To convert ES6 javascript code to ES5
Though overwhelming, but you learn how to use these technologies simultaneously.
Let’s Build This!
This project is going to be built from scratch, without webpack, browserify or Gulp.
Step 1:
Setting Environment and Installing Dependencies:
a. Create a new directory called book-app and switch to that directory
mkdir book-app
cd book-app
b. Initialise npm and follow the instructions
npm init -y
c. Install Express and Body-parser:
npm install --save express body-parser
d. Install babel: Because we need to convert our ES6 code to ES5. Install babel dependencies by copying the code below and paste in the terminal of the working directory:
npm install --save-dev @babel/core @babel/cli @babel/node @babel/plugin-transform-runtime @babel/preset-env @babel/register @babel/runtime babel-loader
Create the
.babelrc file in the path: /book-app/ and populate it with:
{
"presets": ["@babel/preset-env"],
"plugins": [["@babel/transform-runtime"]]
}
e. Install eslint and airbnb style guide. This is Optional!
This will help you format your code in an easy to read state.
npm install --save-dev eslint eslint-config-airbnb-base eslint-plugin-import
Create a file .eslintrc.js in the path: /book-app/
This the way i configured mine:
module.exports = {
"extends": "airbnb-base","rules": {"no-console": 0,"no-param-reassign": [2, {"props": false}],"prefer-destructuring": 0,"treatUndefinedAsUnspecified": true,"arrow-body-style": 0,"comma-dangle": 0,},"env": {"commonjs": true,"node": true,"mocha": true},};
Of course, you can setup yours however you wish.
f. Install nodemon:
In the cause of this tutorial, we will need to run node server over and over again. Nodemon is like a watcher that automatically restarts the server each time changes are made. You can install it globally using:
npm install -g nodemon
Note that this installation will not be in this project package.json file, because it was installed globally.
g. Including nodemon in the package.json file:
Open the package.json file, edit the
scripts object as follows:
"scripts": { "dev": "nodemon --exec babel-node ./api/index.js",
"test": "echo \"Error: no test specified\" && exit 1"
}
Observe the ./api/index.js ? Don’t worry, we will create the file in step 2.
At this point, your package.json file should look like this:
{"name": "book-app","version": "1.0.0","description": "","main": "index.js","scripts": {"dev": "nodemon --exec babel-node ./api/index.js","test": "echo \"Error: no test specified\" && exit 1"},"keywords": [],"author": "","license": "ISC","dependencies": { "body-parser": "^1.18.3", "express": "^4.16.4"},"devDependencies": { "@babel/cli": "^7.4.3", "@babel/core": "^7.4.3", "@babel/node": "^7.2.2", "@babel/plugin-transform-runtime": "^7.4.3", "@babel/preset-env": "^7.4.3", "@babel/register": "^7.4.0", "@babel/runtime": "^7.4.3", "babel-loader": "^8.0.5" }}
h. Structuring the application:
Create a directory called
api . Then create the
index.js file, which is the root file for this application. You should be in the path:
book-app/api/
touch index.js
Change to the
api directory and create another directory called
server. Your directory tree should now look like this:
book-app
├── api
│ ├── server
├── ├── index.js
├── node_modules
├── package.json
└── package-lock.json
Step 2:
Edit the index.js file and start the server
a. Edit the
index.js file:
import express from 'express';
import bodyParser from 'body-parser';const app = express();app.use(bodyParser.json());app.use(bodyParser.urlencoded({ extended: false }));const port = process.env.PORT || 8000;// when a random route is inputed
app.get('*', (req, res) => res.status(200).send({
message: 'Welcome to this API.'
}));app.listen(port, () => {
console.log(`Server is running on PORT ${port}`);
});export default app;
b. Start the server
Since we have defined the script to start the server, we simply run in the terminal from the path:
/book-app/api/:
npm run dev
Hopefully you will see nodemon running in the terminal, with the port number the server is running on:
Step 3
Setup PostreSQL Database
a. Install:
If you don’t already have postreSQL running in your machine, you can download it and setup it.
If you are on a mac OS, you can follow this instruction:
For Windows users:
For Ubuntu users:
b. Setup: You can use the default user called:
postgres or you can create a new user. There is probably hundreds of materials out there that can help you achieve this.
If you have any issues installing and setting up postgreSQL on your local machine, you can use an online database such as ElephantSQL
Step 4
Setup Sequelize
Sequelize is an ORM(Object Relational Mapping) for PostgreSQL. What this means is that, instead of writing raw SQL queries, ORM is used instead, which makes work easier.
a. Install sequelize cli globally:
npm install -g sequelize-cli
Note, if you wish not to install this globally, you’ll need to prefix every call to the
sequelize command with
./node_modules/.bin
b. Create the file: .sequelizerc in the path: /boook-app/. That is, outside of the api folder.
touch .sequelizerc
Write in the file, the code below
const path = require('path')module.exports = {
"config": path.resolve('./api/server/src/config', 'config.js'),
"models-path": path.resolve('./api/server/src/models'),
"seeders-path": path.resolve('./api/server/src/seeders'),
"migrations-path": path.resolve('./api/server/src/migrations')
};
Observe that we required
path. Let’s quickly install it:
npm install --save path
c. Install postgreSQL and sequelize dependencies
npm install --save sequelize pg pg-hstore
Where: pg = postgreSQL and pg-hstore = converts data into the Postgres hstore format.
d. Initialize Sequelize:
This will generate the Sequelize boilerplate which includes,
models,
migrations and
seeders. Run:
sequelize init
The directory tree should now resemble this:
book-app
├── api
│ └── index.js
│ ├── server
│ │ └── src
│ │ └── config
│ │ ├── └── config.js
│ │ └── migrations
│ │ └── models
│ │ ├── └── index.js
│ │ └── seeders│
├── .babelrc
└── .eslintrc.js
└── .sequelizerc
└── package.json
└── package-lock.json
e. Edit the book-app/api/server/models/index.js file.
This is the current state:
'use strict';
var fs = require('fs');
var path = require('path');
var Sequelize = require('sequelize');
var basename = path.basename(module.filename);
var env = process.env.NODE_ENV || 'development';
var config = require(__dirname + '/../config/config.json')[env];
var db = {};
if (config.use_env_variable) {
var sequelize = new Sequelize(process.env[config.use_env_variable]);
} else {
var sequelize = new Sequelize(config.database, config.username, config.password, config);
});
}
});
db.sequelize = sequelize;
db.Sequelize = Sequelize;
module.exports = db;
We will use ES6 syntax.
So, book-app/api/server/models/index.js becomes:
f. Editing the
config.js file:
Let’s quickly install
dotenv package. This will make it easier when using
.env variables
npm install --save dotenv
Now, the
config.js file:
You can replace the
username with the one you used while setting up your postgreSQL. If you don’t remember creating a username, It means you are using the default user, which is
postgres . For the
password if you created a user, use the password, else, leave it the way it is above.
Also, incase you are using an online database, such as ElephantSQL, uncomment this lines:
development: {
use_env_variable: 'DATABASE_URL'
},
Then, you will export the database URL into your environment
export DATABASE_URL=our-database-url
where
our-database-url is the URL we copied from ElephantSQL. Every time you need to run this application, you will need to export the
DATABASE_URL first.
Remember, if you decide to use the online database, and have uncommented the line above, comment the
development setup for the local PostgreSQL database. So as to avoid name duplication.
g. Create a .env file in the path: /book-app/.
DB_NAME=booksDB_USER=stevenDB_PASS=DB_PORT=5432DB_HOST=127.0.0.1SECRET_KEY=any_secret
h. Use Database GUI(Graphical User Interface): Optional.
Instead of doing all database operations in the terminal, which is very painful for some, you can use a GUI such as pgadmin or tableplus
Step 5
Create Databases, Model and Migration
a. Create the Databases:
In the
config.js file above, we referenced a database called
books for development and
book_test for testing. You can either use the GUI or Terminal to create these databases.
Before you run any command, ensure that the PostgreSQL server is running, on mac OS, run:
pg_ctl -D /usr/local/var/postgres start //For mac OS
or:
brew services start postgresql //To always run it on mac OS
pg_ctl -D "C:\Program Files\PostgreSQL\9.6\data" start
//For Windows. Observe the 9.6, yours may be different
sudo service postgresql start //For Linux
Get the list of commands here.
Now, Creating the databases from the terminal:
createdb bookscreatedb book_test
b. Create the Model
We are creating just one model: book.js
sequelize model:create --name Book --attributes title:string, price:string, description:string
Locate book.js in the path: /book-app/api/server/src/models/
Using ES6 syntax and adding to the file, we have the code below:
c. Refactor migration file
A migration file was also created when we ran that command to create the model, located in the path: ..
server/migrations/<date>-create-book.js
Edit the migration file to have:
Save all files and run migration:
sequelize db:migrate
You can check in the database to confirm if migration ran successfully.
Step 6
Create Services, Controllers, Routes and Utilities
Yes, we can put all the code in one file. But that is not best practice. So, is good we separate code into different files. We will create four folders in the path: /book-app/api/server:
book-app
├── api
├── server
├── controllers
├── routes
├── services
├── utils
a. Services:
You might ask, what is a Service? See it as a medium that helps us interface with our book model.
In the Services folder, create a file:
BookService.js
b. Controllers
The Service file created above is used in our controller. Create a file called
BookController.js in the controllers folder
This is the content:
c. Utilities
As seen, in the
BookController.js file, we imported a file called Utils.js. This file contain all the
responses we will assert for while running and testing the api endpoints.
Create the
Utils.js file in the Utils folder.
Its content is:
d. Routes
Create the route file called
BookRoutes.js in the
routes folder.
Its content:
e. Editing the /book-app/api/index.js file to add the
route file defined above:
Step 7
Running Endpoints
Time to see results!
We can test the endpoint using postman app or any API testing tool of your choice.
Start up the server, if you terminated it before:
npm run dev
If you dont know how to use postman, you might need to learn the basics, before continuing.
a. POST a book:
In the postman address bar enter this: //POST
Change the http method to POST, then define the json data to post in the body
As an example:
b. GET all books //GET
c. GET a particular book //GET
d. UPDATE a particular book //PUT
e. DELETE a particular book //DELETE
Step 8
Write Test Cases for Endpoints
I hope, all your endpoint worked as expected.
Now, let’s write test cases for each endpoint.
a. We will need to install testing framework like
mocha and assertion library like
chai and
nyc for test coverage. From the terminal run:
npm install --save-dev mocha chai chai-http nyc
b. We will create a
test folder in the path /book-app/api/. Then create the test file, call it
test.js . Hence, the file is located: /book-app/api/test/test.js
c. Include the test script in package.json file
Replace:
"test": "echo \"Error: no test specified\" && exit 1"
With:
"test": "export NODE_ENV=test && sequelize db:migrate:undo:all && sequelize db:migrate && nyc --require @babel/register mocha ./api/test/test.js --timeout 20000 --exit"
Observe this:
export NODE_ENV=test //Works for mac OS
Using Windows, do:
SET NODE_ENV=test //works for windows
Remember, we have a test database we created earlier:
book_test So, from the test script, we run migrate afresh each time.
Before you run the test, ensure that PostgreSQL server is running.
Save all files and run the test:
npm run test
A Sample output:
All test cases passing!
Step 9
Integrate Travis CI, Coveralls and Code Climate
This i presume, you have been waiting for.
Ensure that all test cases are passing, because Travis will also run those tests. So if any is breaking in your local, it will also break in Travis.
Before we proceed, let’s compile all we have and put them in one folder called
build in this folder all ES6 is converted to ES5 by
babel .
Include the script immediately after the test script in
package.json
"build": "rm -rf ./build && babel -d ./build ./api -s",
We should also include scripts for
coveralls and
code-climate
"
The script should look like:
"scripts": {"dev": "nodemon --exec babel-node ./api/index.js","test": "export NODE_ENV=test && sequelize db:migrate:undo:all && sequelize db:migrate && nyc --require @babel/register mocha ./api/test/test.js --timeout 20000 --exit","build": "rm -rf ./build && babel -d ./build ./api -s","},
For this project, the final look of your
package.json file should be:
Now let’s create the
build folder by running:
npm run build
You will see a folder in the path: /book-app/
book-app
├── api
├── build
a. Sign-Up/Sign-In with Travis
Create an account with Travis if you don’t have one already;
b. Create a
.travis.yml file. This is the file Travis will look for and execute commands. This file should be created in the same path as the
package.json That is: /book-app/
Populate the file with:
Observe that there is a script to create a
database, a
user , run
migration run build
npm run build. Also observe the test command
npm test
c. Push your code to github:
To see the workings of Travis, Coveralls and Code Climate, push the code you have been working on to a new git repository or an existing one created for this project.
After pushing to github, go to your Travis account and turn on that repository:
An example:
Click on the repository, It will take to a page like this:
You may likely see this badge first:
If your
build does not start automatically, look at the top right of the screenshot above, you will see
More options choose the
Trigger build , then a modal pops up, click the
Trigger custom build . This will start the build manually.
A mail will be sent to you when the build is completed. You can check the build status again, you should see the
green passing badge.
d. Use the Travis badge in your github repository
Click on the Travis badge, a modal pops up, Choose
markdown and copy the link and paste in the
README.md file of your repository
e. Create an account with or sign-in to your Coveralls
Connect the same repository we used for Travis by turning it on. Then click on
DETAILS. Scroll down, copy the badge code when you click
EMBED and add it to your
README.md file
f. Create an account with or sign-in to your CodeClimate
Once you are in, Add the same repository you have used in the two cases above.
Once you click on the repository, it will take you to a page similar to this:
Code maintainability can have status A to F
The A status shows that the code can be maintained easily, while a status of F tells the opposite.
To get the badge code, click on the
Repo Settings tab. Click on the
Badges. click the
Markdown, copy the code and add it to your
README.md file
Note:
Let me bring to your notice that Code Climate also have Code Coverage. Since Coveralls is already covering our code, i didn’t see it necessary to add
Code Coverage. If you wish to add it, Click on
Test Coverage and follow the instructions.
So, Your
README.md should look similar to:
[![Build Status]()]()
[![Coverage Status ()]()
[![Maintainability]()]()
With the output:
Bravo!
Step 10:
Final testing and scaling the api
Wow! This article is about 12 minutes read. I admire your patience to see the very end. However, I can’t call this the end because, there are still some stuffs to do, which, you wont have issues executing i guess.
a. You can host the application on
Heroku
b. You can create more endpoints
c. You can write more test cases
d. And lots more.
Conclusion
I hope you didn’t have too much trouble following the instructions in this article. This article is very basic, meaning that you can apply the knowledge gained here to any similar project you are currently working on.
Get the source code here: github
Thank you!