ESP8266 (Web Server): Fast development of HTML + js with Node.js and Pug

Developing a Web server for ESP8266 can quickly become tedious. Indeed, it is necessary to download the files (HTML, js ..) with each modification in order to be able to focus. Fortunately, Node.js will allow us to develop and develop on a computer all the code of the Web Server part before downloading it on the ESP8266. Node.js is so great that we will be able to directly use Pug templates (presentation article) and even retrieve real measurements directly on the ESP8266!

In this tutorial we will see how to quickly realize an HTML interface that will display the measurements of a DHT22 and drive the GPIO of the ESP8266. We can also retrieve the data (and reports) on a computer using a mini web server developed in a few lines of code with Nodejs. The project accounts code is available on GitHub here.

Node.js is a popular cross-platform Open Source project (system independent). Originally designed to develop websites, Nodejs is now used to make applications for smartphones and tablets (Meteorjs, ionicframework, phonegap). It is possible to “transform” the site into application. This is the case, for example, of the Atom text editor developed by GitHub. And if you use Node-RED, this is the engine of this project for IBM IoT. We will stop there on Node.js for this time, we will have the opportunity to use it in many projects in the future.

Equipment used

esp8266 Wemos D1 mini Wemos D1 Mini
chargeur raspbery pi 3 5v 3000ma 5/3A micro-usb power supply

 

wemos d1 mini dht22 shield Shield DHT22 or DHT11 for Wemos D1 Mini

Probe connected on D4 Pin (GPIO2)

led 3mm Led
resistance 220ohms Resistor 220Ω
breadboard Breadboard (optional)
jumper dupont Jumper Dupont (optional)

Install Node.js on your computer

The first thing to do is to install Node.js on your computer. Before you start, it is best to check that Node.js is not already installed on your machine. So definitely the case if you use Raspbian. Open the Terminal or PowerShell on Windows and run this command

$ node -v; npm -v
v6.9.4
3.10.10

If you do not get any results, go to the official Nodejs website to retrieve the version that corresponds to your system. It is preferable to install the stable version (LTS) to avoid edge effects of the version being developed.

nodejs download macos windows linux arm

On macOS, it is preferable to install Node.js using the Homebrew package manager (the official page of the project). Homebrew manages to declare paths. It is also easier to uninstall (cleanly) Node.js and npm (the associated package manager) with the brew uninstall node  command

node-red mongodb brew

Open a Terminal and paste this command

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Answer the questions to complete the installation of Homebrew on your Mac. Once the installation of Homebrew is complete, you can install Node.js by executing the command

brew install node

Node.js for ARM chips (Raspberry PI, Orange Pi …)

Node.js is also available for ARM v6, v7 and v8 platforms. The archive is directly recoverable for download but the simplest remains to go through the Terminal. We start by adding the Nodejs sources to the system’s repository manager. Complete the order according to the desired version.

For version 7.x under development

curl -sL https://deb.nodesource.com/setup_7.x | sudo -E bash -

For the stable version

curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -

Now, a simple apt-get command to install Node.js on your Raspberry Pi for example.

sudo apt install nodejs

Test if everything works

Nodejs has its own shell that is simply launched with the node command in the Terminal. If everything is correctly installed, you will get the prompt (>). Make a small addition. To exit the shell, type .exit

$ node
> 3+1
4
> .exit
$

Prepare a development server for Web Server ESP8266

We start a Web server in a few lines with Node.js. Here, we will use express.js which is a framework simplifying the management of the roads (and accelerates the rendering of the pages, but it’s accessory here). We use npm to install new package.

npm install express --save

To make HTTP requests and retrieve actual data on the ESP8266, we will need the package request

npm install request

Direct use of Pug Templates

It would be a shame to generate the HTML files after each change. To use the Pug files, you have to do 2 things:

  • Create a folder views in the project’s data folder. Then move all your pug files inside. It is possible to keep the tree (but I have not tested)
  • Specify that the view engine will be pug.

Here is the code of the server which will enable us to test in local the HTML interface of our projects ESP8266. It takes only 7 lines of code to run the interface

var express = require('express');
var app = express();

// Créé directement les pages HTML à partir de templates pug 
app.set('views', './views');
app.set('view engine', 'pug');

app.get('/', function(req, res) {
  res.render('index')
});

app.listen(8080);

Save the file as server.js and run the server with the node server.js command from the Terminal (without forgetting to place it in the project directory). Open a browser and type localhost: 8080 in the address bar. And here you can now develop your interfaces and javascript code much faster on your own computer!

nodejs esp8266 webserveur fast development

Provide resources (images …)

If your project contains resources, such as a folder with images, you must make them accessible to the web browser. To do this, you must create a public folder in which you will have to move the resources (images among others). Then we tell express.js to make public resources available to the Web client.

app.use(express.static('public'));
app.use('/static', express.static(__dirname + '/public'));

The modified project tree.

Automatically restart the server after each change

You have noticed that it is necessary to restart the server after each modification (scripts js, templates pug …). We will therefore automate the restart of the Web server after each modification. For this, there is the nodemon plugin that requires very little configuration. This avoids installing and configuring a task manager like Grunt or Gulp, oversized in this case!

npm install -g nodemon

Now, you run the server using the nodemon server.js  command. There are many parameters available that can be put in a small configuration file that should be named nodemon.json and place it on the same level as the server.js file. In the watch section, the directories to be monitored are indicated. The ext key is used to indicate extensions of the files to be monitored.

{	
  "restartable": "rs",
  "ignore": [
    ".git",
    "node_modules/**/node_modules"
  ],
  "verbose": true,
  "execMap": {
    "js": "node --harmony"
  },
  "watch": [
    "views",
    ""
  ],
  "ext": "js json pug"
}

Now we start the server with the command nodemon server.js . We can see here that there are 2 files monitored. Note that the server was automatically restarted after a modification of the index.pug file. It is also possible to restart the web server at any time by entering rs.

Note. Nodemon does not refresh the page on the browser (no Livereload), it must be done manually

$ nodemon server.js
[nodemon] 1.11.0
[nodemon] reading config /Volumes/Macintosh HD/Users/christophe/DHT22WebserverESP8266_SPIFFS_Tuto3/data/nodemon.json
[nodemon] to restart at any time, enter `rs`
[nodemon] ignoring: .git .nyc_output .sass-cache bower_components coverage node_modules .git node_modules/**/node_modules
[nodemon] watching: /Volumes/Macintosh HD/Users/christophe/DHT22WebserverESP8266_SPIFFS_Tuto3/data/views/**/*
[nodemon] watching extensions: js,json,pug
[nodemon] starting `node --harmony server.js`
[nodemon] child pid: 10892
[nodemon] watching 2 files
[nodemon] files triggering change check: views/index.pug
[nodemon] matched rule: **/Volumes/Macintosh HD/Users/christophe/DHT22WebserverESP8266_SPIFFS_Tuto3/data/views/**/*
[nodemon] changes after filters (before/after): 1/1
[nodemon] restarting due to changes...
[nodemon] views/index.pug

[nodemon] starting `node --harmony server.js`
[nodemon] child pid: 10902

Responding to Web Interface Requests:

It is much easier to develop and set up interfaces when a dataset is available. We will therefore answer the calls of the interface and return data. That’s how I developed the integration of Google Charts.

With express.js, you retrieve a call on a route using the function app.use(‘/route’, function (req, res, next) {}). For example, here we intercept a request for data on the route /mesures.json.

app.use('/mesures.json', function (req, res, next) {
});

We will try to retrieve actual measurements on the ESP8266, and if the ESP8266 did not respond within 2 seconds (timeout: 2000),

request({url: baseUrl + '/mesures.json',timeout:2000}, function (error, response, body) {
});

It will suffice to test the response or catch an error to return the data from the ESP8266 or a test set if the ESP8266 did not respond to our request. The response of the ESP8266 is contained in the body

if (!error && response.statusCode == 200) {
  console.log("Mesures receptionnées depuis ESP8666" + body) 
  res.send(body);
} else {
  console.log("ESP8666 muet, envoi du jeu de données test")
  res.send({"t":"21.70","h":"29.50","pa":"984.43"});
}

Which gives once assembled

app.use('/mesures.json', function (req, res, next) {
  request({url: baseUrl + '/mesures.json',timeout:2000}, function (error, response, body) {
    if (!error && response.statusCode == 200) {
      console.log("Mesures receptionnées depuis ESP8666" + body) 
      res.send(body);
    } else {
      console.log("ESP8666 muet, envoi du jeu de données test")
      res.send({"t":"21.70","h":"29.50","pa":"984.43"});
    }
  })
});

For the GPIO, it’s a bit more complicated. Remember, you can easily decode a query in which the data is passed in parameters in the url. For example, http://192.168.1.21/gpio&id=D7&etat=1 . Side Node.js, we’ll have to build the query that goes well. We will use the request function but this time we will not pass to him a series of options that will allow him to construct the URL

var options = {
  url: baseUrl + "/gpio",
  method: 'GET',
  headers: {
    'User-Agent':       'Super Agent/0.0.1',
    'Content-Type':     'application/x-www-form-urlencoded'
  },
  qs: {'id': req.param.('id'), 'etat': req.param('etat')},
  timeout:2000
}

The value of each parameter is retrieved using the function req.param(‘param’). All that remains is to interrogate the ESP8266 and return the answer or a test set if it remains silent after 2 seconds.

request(options, function (error, response, body) {
  if (!error && response.statusCode == 200) {  
    console.log("Retour GPIO ESP8266 : " + body);
    res.send(body);
  } else {
    console.log("ESP8666 muet, envoi du jeu de données test")
    res.send({id:req.param('id'), etat: req.param('etat'), success:'1'});
  }  
};

Full server code

Here is the complete code of the server that you can adapt to your needs. It is also present in the GitHub repositories of the Web Server project ESP8266.

/*
*   Serveur de développement rapide d'interfaces Web pour projets ESP8266
*   Fast development Web Server for ESP8266 projects
*   http://www.projetsdiy.fr - 2017
*/
var express = require('express');
var request = require('request');

// Sert les fichiers publics (css, img) - Serve public files
var app = express();
var baseUrl = "http://192.168.1.22";

// Créé directement les pages HTML à partir de templates pug 
app.set('views', './views');
app.set('view engine', 'pug');

app.use(express.static('public'));
app.use('/static', express.static(__dirname + '/public'));

app.get('/', function(req, res) {
  res.render('index')
});

app.use('/mesures.json', function (req, res, next) {
  request({url: baseUrl + '/mesures.json',timeout:2000}, function (error, response, body) {
    if (!error && response.statusCode == 200) {
      console.log("Mesures receptionnées depuis ESP8666" + body) 
      res.send(body);
    } else {
      console.log("ESP8666 muet, envoi du jeu de données test")
      res.send({"t":"21.70","h":"29.50","pa":"984.43"});
    }
  })
});

app.use('/tabmesures.json', function (req, res, next) {
  request({url: baseUrl + '/tabmesures.json',timeout:2000}, function (error, response, body) {
    if (!error && response.statusCode == 200) {
      console.log("tabmesures receptionnées depuis l'ESP8666" + body) 
      res.send(body);
    } else {
      console.log("ESP8666 muet, envoi du jeu de données test")
      res.send([{"mesure":"Température","valeur":"21.60","unite":"°C","glyph":"glyphicon-indent-left","precedente":"19.80"},{"mesure":"Humidité","valeur":"29.80","unite":"%","glyph":"glyphicon-tint","precedente":"30.40"},{"mesure":"Pression Atmosphérique","valeur":"984.51","unite":"mbar","glyph":"glyphicon-dashboard","precedente":"984.74"}]);
    }
  })
});

app.use('/graph_temp.json', function (req, res, next) {
  request({url: baseUrl + '/graph_temp.json',timeout:2000}, function (error, response, body) {
    if (!error && response.statusCode == 200) {
      console.log("Graph receptionnées depuis l'ESP8666" + body) // Show the HTML for the Google homepage.
      res.send(body);
    } else {
      console.log("ESP8666 muet, envoi du jeu de données test")
      res.send({"timestamp":[1485273937,1485273938,1485273939,1485273940,1485273941,1485273942,1485273943,1485273944,1485273945,1485273946,1485273947,1485273948,1485273949,1485273950],"t":[23.3,23.3,23.3,23.3,23.3,23.3,23.3,23.3,23.3,23.3,23.3,23.3,23.3,23.3],"h":[35.6,35.6,35.6,35.6,35.6,35.5,35.5,35.4,35.4,35.5,35.5,35.5,35.5,35.5],"pa":[987.7,987.7,987.7,987.8,987.7,987.7,987.7,987.7,987.7,987.8,987.7,987.7,987.7,987.7],"bart":[23.30,23.30,23.30,23.30,23.30,23.30,23.30],"barh":[35.60,35.60,35.50,35.40,35.50,35.50,35.50]});
    }
  })
});

app.use('/gpio', function (req, res, next) {
	var options = {
      url: baseUrl + "/gpio",
      method: 'GET',
      headers: {
    	  'User-Agent':       'Super Agent/0.0.1',
    	  'Content-Type':     'application/x-www-form-urlencoded'
	    },
      qs: {'id': req.param('id'), 'etat': req.param('etat')},
      timeout:2000
    }

	//_url = baseUrl + "/gpio&id=" + req.param('id') + "&etat="+ req.param('etat');
	//console.log(_url);
	request(options, function (error, response, body) {
    if (!error && response.statusCode == 200) {  
      console.log("Retour GPIO ESP8266 : " + body);
      res.send(body);
    } else {
      console.log("ESP8666 muet, envoi du jeu de données test")
      res.send({id:req.param('id'), etat: req.param('etat'), success:1});
    }  
	});
})

app.listen(8080);

Now, if an ESP8266 is connected to the network, you are able to retrieve data to easily develop your project. Messages sent to the Terminal using the console.log () command make it easier to focus more.

nodemon server.js
[nodemon] 1.11.0
[nodemon] reading config /Volumes/Macintosh HD/Users/christophe/DHT22WebserverESP8266_SPIFFS_Tuto3/data/nodemon.json
[nodemon] to restart at any time, enter `rs`
[nodemon] ignoring: .git .nyc_output .sass-cache bower_components coverage node_modules .git node_modules/**/node_modules
[nodemon] watching: /Volumes/Macintosh HD/Users/christophe/DHT22WebserverESP8266_SPIFFS_Tuto3/data/views/**/*
[nodemon] watching extensions: js,json,pug
[nodemon] starting `node --harmony server.js`
[nodemon] child pid: 11071
[nodemon] watching 1 files
Graph receptionnées depuis l'ESP8666{"timestamp":[1485883795],"t":[21.6],"h":[45.8],"pa":[977.9],"bart":[],"barh":[]}
Graph receptionnées depuis l'ESP8666{"timestamp":[1485883795],"t":[21.6],"h":[45.8],"pa":[977.9],"bart":[],"barh":[]}
tabmesures receptionnées depuis l'ESP8666[{"mesure":"Température","valeur":"21.70","unite":"°C","glyph":"glyphicon-indent-left","precedente":"21.60"},{"mesure":"Humidité","valeur":"45.60","unite":"%","glyph":"glyphicon-tint","precedente":"45.80"},{"mesure":"Pression Atmosphérique","valeur":"977.87","unite":"mbar","glyph":"glyphicon-dashboard","precedente":"977.89"}]
express deprecated req.param(name): Use req.params, req.body, or req.query instead server.js:68:22
express deprecated req.param(name): Use req.params, req.body, or req.query instead server.js:68:47
Retour GPIO ESP8266 : {"gpio":"D7","etat":"1","success":"1"}

nodejs esp8266 donnees reelles

So, you can now develop your HTML interfaces and javascript code much faster for your ESP8266 projects. This is an accelerated tutorial. We will return in more detail to the different technologies discussed here in future tutorials.

Subscribe to the weekly newsletter

No spam and no other use will be made of your email. You can unsubscribe anytime.

3 Comments
  1. what does your index.pug file look like?

  2. also what code is the esp8266 running?

Leave a Reply

DIY Projects
%d bloggers like this: