Get started with ArduinoJSON v6, Arduino library for handling JSON objects

debuter get started arduinojson json arduino esp32 esp8266
Share on facebook
Share on twitter
Share on linkedin
Share on pinterest
Share on email
Share on whatsapp

The JSON format is used to organize, store and transmit data. Widely used in the IT world, it also finds its place in the world of micro-controllers. Among all the libraries available for Arduino, ESP32 and ESP8266, ArduinoJSON is the most popular and the most successful.

 

In this tutorial we will also learn how to format and then send data to the serial port without needing any library with standard Arduino code.

Note, this article was written with ArduinoJson 6. Follow the instructions in this  article to migrate code developed with version 5 of the library.

What is the JSON format?

JSON (JavaScript Object Notation) is a textual data format (wikipedia) that allows data to be structured for storage and transfer. It very advantageously replaces the XML format. This is the preferred data format of the JavaScript language.

It is often used in services such as APIs (application programming interfaces) and web services that provide public data.

Without going into too much detail, here’s how it works:

  • Data is represented by key: value pairs
  • The colon separates the key from the value
  • The quotation mark allows to surround a character string
  • key: value pairs are separated by commas  , (except the last line)
  • Braces {} contain objects. You can store whatever you want there as long as you respect the JSON formalist above
  • Square brackets [ ] contain arrays

This gives for example a structure

{
  "key1":"value1",
  "key2":"[1,2,3,4]",
  "key3":{
     "keya":"valuea",
     "keyb":"['a','b','c']"
  }
}

Or an arrayu un tableau

[{
    "temperature": 18.6,
    "humidity": 52.5
  },{
    "temperature": 19.1,
    "humidity": 52.3
}]

Last remark, the JSON format is nothing other than a character string, which gives for the previous example

[{"temperature":"18.6","humidity":"52.5"},{"temperature":"19.1","humidity":"52.3" }]

The JSON vocabulary

Before going any further, let’s see some vocabulary that we come across very often

Sérialiser | Serializer transforms a JSON object into a character string

Désérialiser | Deserializer transforms a string into a JSON object

Parser same as deserialize

How to format data in JSON format without an Arduino library?

Outside of memory, JSON objects are stored (most often) as strings.

So if you don’t need to manipulate the data in your project, no need to burden yourself with a resource consuming library.

Take for example this project (in french) where we collect the moisture content of the soil with a probe. It is very easy to send the measurements to the serial port and then display them on Node-RED in the form of a graph.

yl-69 fc-28 humidite sol arduino node-red_bb

Here are two examples of Arduino code. The code does exactly the same:

  • On the left, the JSON string is sent over the serial port.
  • On the right, we prepare the JSON string and then we send it to the serial port

What is important in both cases is to end with a println() so as to send the newline “\n” control character. In this way Node-RED will be able to detect new incoming measurements.

Without pre-formattingWith pre-formatting
#define wait 5000

void setup(){
  Serial.begin(115200);
}

void loop() {
 //Read YL-69 value
 int a5 = analogRead(A5); 
 
 Serial.print("{\"YL69\":");
 Serial.print(a5);
 Serial.println("}");
 delay(wait);
}
#define wait 5000

void setup(){
  Serial.begin(115200);
}

void loop() {
 //Read YL-69 value
 int a5 = analogRead(A5); 
 
 String jsonString = "{\"YL69\":";
 jsonString += a5;
 jsonString += "}";
 Serial.println(jsonString);
 delay(wait);
}
We get exactly the same result
json arduino send direct string serialjson arduino pre formated string serial

What is the escape character \ for?

Some languages such as C ++ only allow the quotation mark to define a character string. Unfortunately, the JSON format requires enclosing the keys in quotation marks as well as the values as soon as it is a character string.

This is where the escape character \ comes in. You just need to put it in front of a quotation mark like this \” to avoid compilation errors.How to test strings in JSON format online

We have seen small examples previously. In a real project, you might quickly have to deal with large objects. Flattened out, it’s perfectly unreadable and difficult (impossible) to debug.

To test your JSON, there are many online tools. I advise you jsonlint

jsonlint arduino json esp32 sp8266

In which case to use (or not) an Arduino JSON library?

The Arduino language allows you to create structures containing data in different formats. It is an equivalent of the JSON format.

However, it may be useful to use a library allowing to manipulate JSON objects in several cases:

  • When data needs to be stored (on an SD board or Flash memory) and reloaded
  • When retrieving data in this format, for example data or parameters from a server
  • When we have to manipulate the data

And avoid using it in the following cases:

  • A data acquisition system. It is better to “push” the data to a server or a file on an SD board or on Flash memory (LittleFS or SPIFFS)

There are several libraries for handling JSON objects. Benoît Blanchon’s ArduinoJSON library remains the reference.

Install the ArduinoJson library on the Arduino IDE

The ArduinoJson library developed by Benoît Blanchon is the reference library on Arduino

You can install the ArduinoJson library directly from the library manager of the Arduino IDE. Use the selector to choose the desired version.

install ide arduino arduinojson v6 library

Very well documented, it even has a totally dedicated website here.

Only downside, handling JSON objects is not without consequences on the Arduino code. This is because the JSON object is entirely stored in the RAM of the microcontroller, which can limit performance and use cases.

Version 6 improves things, however.

Differences between ArduinoJson 5 and ArduinoJson 6: code migration

All of the examples given in this tutorial use ArduinoJson 6, the most recent version. ArduinoJson is a disruptive version that introduces new concepts and requires migration of existing code.

Benoît Blanchon also made a video which explains how to migrate code from version 5.

Here are the main changes.

Version 5Version 6
JSON object in memory
JsonBuffer

With ArduinoJson 5, it was difficult to use a JsonObject or a JsonArray as a class member because you had to make sure that the JsonBuffer also stayed in memory. The trick was to add the JsonBuffer as a class member as well, but it was more complicated.

JsonDocument

ArduinoJson 6 replaces the concept of JsonBuffer with the concept of JsonDocument. The JsonDocument is stored in memory and contains the root of the object tree. It’s basically the combination of JsonBuffer and JsonVariant.

Because a JsonDocument can contain any type of value, you must convert it to read the content.

There are two versions of the JsonDocument.

It is recommended to use a StaticJsonDocument object for a document smaller than 1 KB and DynamicJsonDocument for a document larger than 1 KB

Measure the size of a JSON object
size_t len = obj.measureLength();
size_t len = measureJson(doc);
Deserialise
JsonObject& obj = jb.parseObject(input);
deserializeJson(doc, input);
Serialise
obj.printTo(Serial);
serializeJson(doc, Serial);

Stack and heap, advantages and disadvantages, how to choose?

ArduioJSON is able to use the batteries (Stack) or the Tas (Heap) of micro-controllers. Here are the main advantages and disadvantages and especially which to choose.

StackHeap
A stack is a special area of microcontroller (or computer) memory allocated by the processor that stores temporary variables created by a function.

In the stack, variables are declared, stored, and initialized during execution. The last item in the stack is the first that can be removed. It is not necessary to remember the location of an item within the stack. When the task is finished, the memory of the variable is automatically erased.

The size of the stack is assigned by the operating system when the task is launched, it cannot be varied. The stack is deleted when the task completes.

The heap is a location in memory used for dynamic allocations. Unlike the stack, any block in this space can be allocated or released at any time. The management of the heap is more complex and slower because it is necessary to know constantly which block is allocated.

The heap is created when a process starts and is linked to it. This means that several tasks can access the same block in this memory.

The size of the heap varies and may increase if the program needs more memory.

Advantages and disadvantages

Size < 1024 bytesSize > 1024 bytes
JSON object stored in stackJSON object stored in the heap
Fast, small sizeSlower, large dimension
StaticJsonDocument<100> firtJSON;
DynamicJsonDocument<2000> firtJSON;

More information on guru99.com.

How to determine the size of the buffer?

Part of the microcontroller’s memory must be reserved for the JSON object. The first thing to do before starting the project is therefore to determine the amount of memory to reserve.

For this, there are two online wizards that allow you to estimate the size of the buffer

Two examples of JSON objects returned by the OpenWeatherMap and Weather Underground APIs provide a better understanding of how the buffer size should be calculated.

I therefore advise you to start by using this tool to estimate the size of the buffer and to reserve at least 20% of additional memory space in order to take into account any variations.

As you can see, the buffer size also varies depending on the platform. The line tool is used to estimate the size of the buffer for Arduino, ESP32, ESP8266, SAMD21 et STM32.

estimation taille size buffer arduinojson v6 esp32 esp8266 stm32 samd21

Is it possible to allocate all the available memory to a JsonDocument?

yes it is possible but it is advisable to free the memory as soon as possible so as not to block other processes. This trick can cause memory fragmentation. More info here.

Create a JSON object of type StaticJsonDocument

As for any library, we start by declaring the ArduinoJSON library at the start of the program

#include <ArduinoJson.h>

The first thing to do is therefore to declare an object of type StaticJsonDocument. This document stored in the memory of the microcontroller will contain the JSON object. We can handle it (write, read, modify a value).

We will start with a simple object that will store the temperature, humidity and atmospheric pressure of a BME280 sensor. Using the online tool, we get between 77 bytes (Arduino) and 111 bytes (ESP32). We will cut the pear in half and allocate 100 bytes.

As we are well below 1KB (1024 bytes), we will use a StaticJsonDocument.

Adding data is now similar to JavaScript. It suffices to indicate between brackets [] the key and the value to be assigned to it without worrying about the type of data. The ArduinoJson library takes care of everything.

for example

firstJSON["key"] = "value"

This gives this object for a BME280 sensor (obviously these are fictitious values).

Objet in memoryReal representation
firtJSON["sensor"] = "BME280";
firtJSON["temperature"] = 22.5;
firtJSON["humidity"] = 65.5;
firtJSON["pressure"] = 988.7;
{ 
  "sensor": "BME280"
  "temperature": 22.5,
  "humidity": 65.5,
  "pressure" : 988.7
}

How to modify a value in the JSON object

Since version 6 of the ArduinoJson library, it is very easy to modify the value of a key. Just use the brackets operator exactly like we did previously to assign the first value

firstJSON["temperature"] = 23.1;

It’s that simple !

Serialize the JSON object, convert it to a character string

In many situations, you will want to store the data (on flash memory or a microSD card), or send it to third party software (or a server). To do this, you must convert the JSON object into a character string.

It is the serialize operation

The character string obtained must be retrieved from a buffer whose size is identical to the JSON object

buffer[100];

Then to get the JSON string, just call the function serializeJson(objet_Json, buffer).

char buffer[firstJsonSize];
serializeJson(firstJSON, buffer);
Serial.println(buffer);

We get the JSON object as a string on the serial monitor

{"sensor":"BME280","temperature":22.6,"humidity":22.6,"pressure":998.5}

Note. The resulting string is minified, that is, all spaces and newlines have been removed. In this example, this is not a problem, but when the JSON is large it becomes difficult (impossible) to read.

You can use an online tool such as jsonlint presented earlier or use the function serializeJsonPretty which does the same. Please note, only use this function for focusing.

Upload the example code

Here is the complete source code which will allow you to test what we have just discovered.

If you are using the Arduino IDE, remove the first line (#include <Arduino.h>) for PlatformIO.

On ESP32, it is possible to know the size of the remaining Heap at any time by calling the method ESP.getFreeHeap().

#include <Arduino.h>  
// load library
#include #include <ArduinoJson.h>

#define firstJsonSize     100
#define displayFreeHeap  false
void setup() {
 
  Serial.begin(115200);
  Serial.println("\n");
  
  #ifdef ESP32
    // Display free heap on startup 
    Serial.print("ESP32 free Heap at startup: ");
    Serial.println(ESP.getFreeHeap());
  #endif  
    
  // Uncomment type of storage: dynamic (heap) or static (stack) 
  // Decommenter le type de stockage : dynamic (heap) ou static (stack) 
  //DynamicJsonDocument firstJSON;
  StaticJsonDocument firstJSON;
   
  firstJSON["sensor"] = "BME280";
  firstJSON["temperature"] = 22.6;
  firstJSON["humidity"] = 22.6;
  firstJSON["pressure"] = 998.5;
 
  char buffer[firstJsonSize];

  serializeJson(firstJSON, buffer);
  // Or for human reading 
  //serializeJsonPretty(firstJSON, buffer);
  
  Serial.print("Data serialised: ");
  Serial.println(buffer);

  // Update Temperature value 
  firstJSON["temperature"] = 23.1;

  serializeJson(firstJSON, buffer);
  Serial.print("Data udpated: ");
  Serial.println(buffer);

  #ifdef ESP32
    if ( displayFreeHeap  ) {
      Serial.print("ESP32 free Heap at startup: ");
      Serial.println(ESP.getFreeHeap());
    }  
  #endif 
}
 
void loop() {
 
}

Deserialize (Parser), how to convert a string to a JSON object

We saw how to create a JSON object with Arduino code and insert and then modify values.

In many cases, we will have to recover data or settings from a third party server or a file stored on a micro-SD board. In this case we get a character string that we will have to convert into a JSON object usable by the library. It’s deserialize or Parse a string.

Create an IQAir account

For this tutorial, we will use the API of the IQAir service which allows to know the air quality and the weather in many cities around the world. The free account allows up to 1 million requests per month.

Go to this page an create a free account.

iqair creation compte

Confirm your email address

creer compte gratuit iqair

Open your account and go to the tab API then new key. Choose the plan Community and validate

create api key iqair json esp32

 

Copy your API Key.

api key iqair airvisual jsonThere are many detailed endpoints here

To build the query, you will need to know the country (country), the region (state) and the city (city).

Use these queries, each time replacing the parameters shown in braces (removing the braces):

  • http://api.airvisual.com/v2/countries?key={{YOUR_API_KEY}} country list
  • http://api.airvisual.com/v2/states?country={{COUNTRY_NAME}}&key={{YOUR_API_KEY}} list of regions for the indicated country
  • http://api.airvisual.com/v2/cities?state={{STATE_NAME}}&country={{COUNTRY_NAME}}&key={{YOUR_API_KEY}} list of cities available in the region indicated

Here is for example the request for Paris (France)

http://api.airvisual.com/v2/city?city=paris&state=Ile-de-France&country=france&key={{YOUR_API_KEY}}

Which gives the response in the following JSON format

{"status":"success","data":{"city":"Paris","state":"Ile-de-France","country":"France","location":{"type":"Point","coordinates":[2.351666,48.859425]},"current":{"weather":{"ts":"2020-09-04T18:00:00.000Z","tp":25,"pr":1019,"hu":57,"ws":4.6,"wd":340,"ic":"01d"},"pollution":{"ts":"2020-09-04T18:00:00.000Z","aqius":6,"mainus":"n2","aqicn":17,"maincn":"n2"}}}}

Deserialize (parse) the response returned by the JSON AI of IQAir

Create a new sketch on the Arduino IDE or a new PlatformIO project and paste the following code.

Modify the following parameters in the code:

  • WiFi network connection credentials
  • API key

The code below is ESP32 and ESP8266 compatible and automatically detects the platform

#include <Arduino.h>
// load libraries
#include <ArduinoJson.h>
#ifdef ESP32
  #include <WiFi.h> 
  #include <HTTPClient.h> 
#else
  #include <ESP8266WiFi.h> 
  #include <ESP8266HTTPClient.h> 
#endif

#define docSize          800
#define displayFreeHeap  false
String iqair_api_key = "enter_your_apikey";
String city = "Paris";
String state = "Ile-de-France";
String country = "france";

int heap_at_startup = 0;

// Replace with your network credentials
const char* SSID = "enter_your_ssid"; 
const char* PASSWORD = "enter_your_password";

String jsonBuffer;

void setup() {
 
  Serial.begin(115200);
  Serial.println("\n");
  
  Serial.print("Connecting to ");
  Serial.println(SSID);
  WiFi.begin(SSID, PASSWORD);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  // Print local IP address 
  Serial.println("");
  Serial.println("WiFi connected at IP address:");
  Serial.println(WiFi.localIP());

  StaticJsonDocument aiqAir;

  // Start Web Server
  if(WiFi.status()== WL_CONNECTED){
    String serverPath = "http://api.airvisual.com/v2/city?city=" + city + "&state=" + state + "&country=" + country + "&key=" + iqair_api_key;

    HTTPClient http;
    http.begin(serverPath.c_str());
    int httpResponseCode = http.GET();
    if ( httpResponseCode == 200 ) {
      Serial.print("HTTP Response code: ");
      Serial.println(httpResponseCode);
      String payload = http.getString();
      Serial.println(payload);
      deserializeJson(aiqAir, payload);
    }
  }    
  
  if ( aiqAir.isNull() ) {
    Serial.println("! No Data returned by server");
  } else {
    Serial.print("Memory used (bytes): ");
    Serial.println(aiqAir.memoryUsage());

    // All data 
    JsonObject data = aiqAir["data"];
    // Only pollution 
    JsonObject data_current_pollution = data["current"]["pollution"];
    const char* data_current_pollution_ts = data_current_pollution["ts"]; // "2020-09-04T18:00:00.000Z"
    int data_current_pollution_aqius = data_current_pollution["aqius"]; // 6
    const char* data_current_pollution_mainus = data_current_pollution["mainus"]; // "n2"
    int data_current_pollution_aqicn = data_current_pollution["aqicn"]; // 17
    const char* data_current_pollution_maincn = data_current_pollution["maincn"]; // "n2"

    Serial.print("Pollution at "); Serial.print(city); Serial.println(": ");
    Serial.print("IAQ "); Serial.print(data_current_pollution_aqius);
    Serial.print(" Polluting "); Serial.println(data_current_pollution_maincn);
  }

  #ifdef ESP32
    // Display free heap on startup 
    Serial.print("ESP32 free Heap at startup: ");
    heap_at_startup = ESP.getFreeHeap();
    Serial.println(heap_at_startup);
  #endif   

  #ifdef ESP32
    if ( displayFreeHeap  ) {
      Serial.print("Heap memory used");
      Serial.println(ESP.getFreeHeap() - heap_at_startup);
    }  
  #endif 
}
 
void loop() {
}

How the code works?

We load the libraries allowing to connect to the WiFi network and create an HTTP client

#ifdef ESP32
  #include <WiFi.h> 
  #include <HTTPClient.h> 
#else
  #include <ESP8266WiFi.h> 
  #include <ESP8266HTTPClient.h> 
#endif

The ArduinoJson wizard recommends between 430 (Arduino) and 630 bytes (ESP32 / ESP8266). For security, 800 bytes are assigned.

#define docSize          800

As before, we create an object of type StaticJsonDocument (size < 1Ko)

StaticJsonDocument aiqAir;

We build the url that points to the exit point (endpoint) of the IQAIr server

String serverPath = "http://api.airvisual.com/v2/city?city=" + city + "&state=" + state + "&country=" + country + "&key=" + iqair_api_key;

We create an object that will contain the HTTP client

HTTPClient http;
http.begin(serverPath.c_str());

We store the response number.

int httpResponseCode = http.GET();

If the response is equal to 200, the server has returned a response. To know all the codes of the HTTP responses go here.

if ( httpResponseCode == 200 ) {
  ...
}

We retrieve the content of the response directly in JSON format using the http.getString() function of the HTTPClient() library on ESP32 and on ESP8266

String payload = http.getString();

Here is an example response returned by the JSON API of IQAir

{"status":"success","data":{"city":"Paris","state":"Ile-de-France","country":"France","location":{"type":"Point","coordinates":[2.351666,48.859425]},"current":{"weather":{"ts":"2020-09-04T18:00:00.000Z","tp":25,"pr":1019,"hu":57,"ws":4.6,"wd":340,"ic":"01d"},"pollution":{"ts":"2020-09-04T18:00:00.000Z","aqius":45,"mainus":"p2","aqicn":22,"maincn":"o3"}}}}

Now all you have to do is apply the deserializeJson() method of the ArduinoJson library on the payload to convert the response (parser) and place it in the previously reserved memory space.

deserializeJson(aiqAir, payload);

The isNull() method is used to check whether the object is empty or contains data.

The memoryUsage() method allows you to find out the space actually occupied in the memory of the microcontroller (Arduino, ESP32, ESP8266 or STM32).

How to extract data stored in a JSON object?

Access to data is similar to the JavaScript language.

Take an excerpt from IQAir’s answer and imagine that we want to retrieve the longitude. The longitude is stored in an array (at index 1) found in data: location: coordinates.

how to comment extraire donnees json esp8266 arduino esp32

 

It is optional but more practical to extract the data portions, here we extract the root

JsonObject data = aiqAir["data"];

Then we access the desired key. Here is position 1 of the GPS coordinates table.

Be careful, you must specify the type of data each time. Here the longitude is of type float.


float data_location_coordinates_1 = data["location"]["coordinates"][1];

If you are not comfortable with accessing data in a JSON, remember that the wizard can chew up your job. 💪

assistant arduinojson v6 a-parser esp32 esp8266 arduino

Tutorials and projects to go further with JSON on microcontrollers

Here are some projects and tutorials that use the JSON format to store or transmit data.

Updates

7/09/2020 First publication of the article

Click to rate this post!
[Total: 0 Average: 0]

Are you having a problem with this topic?

Maybe someone has already found the solution, visit the forum before asking your question
Ask your question

Share on facebook
Share on twitter
Share on linkedin
Share on pinterest
Share on email
Share on whatsapp

Did you like this project ? Don't miss any more projects by subscribing to our weekly newsletter!

We will be happy to hear your thoughts

      Leave a Reply

      DIY Projects