ESP8266 (Web Server – Part 4): ArduinoJson, load, save files (SPIFFS)

In this 4th tutorial on Web Server programming of ESP8266, we will deal with the storage of files in the SPIFFS system file. We will continue to develop the mini weather station and store the measurements in a JSON object using the ArduinoJson library. It is very easy to save a JSON object as a string in a text file and reload it at the start of the ESP8266 to continue data acquisition.

Installing the ArduinoJson library

It is possible to create a structure to manage the data but the JSON format (already explained in the previous tutorial) offers many advantages. The bookshop ArduinoJson developed by Benoît Blanchon is a model of the genre. Very well documented, it is very simple to implement. The only downside, manipulating JSON objects is not without consequences on the Arduino program. Indeed, the JSON object is stored in memory, which greatly limits our ambitions. If you need to create a data acquisition system, this library is not the best. As part of this project, it is very interesting because it is very easy to manipulate the data and send it to the web interface (just in 2 lines of code!).

In a next tutorial, we will see how to save data directly to a file with a separator (‘,’ for example), easily recoverable on a spreadsheet (Excel, Libre Office…).

To install the ArduinoJson library, simply go to the library manager.

arduinojson-esp8266-json-charger-enregistrer-spiffs-historique-acquisition-donnees

Preparing the buffer and the JSON object

Then, a statement is added at the beginning of the program

The ArduinoJson library is very well documented (wiki on GitHub). I propose an example of typical use in which we will carry out the following manipulations:

  • Creating a JSON object
  • Addition of keys: timestamp, t (temperature), h (humidity), pa (atmospheric pressure), bart and barh (data classified to construct a bar graph)
  • Export and save the JSON in the SPIFFS area
  • Reload the data file from the SPIFFS area and recreate the corresponding JSON object

We start by creating a buffer that will contain the JSON object. It is possible to let the library dynamically change the buffer size, but performance will be affected. To determine the buffer size, refer to this page (Memory Usage section). Let’s estimate the size for this project. We will record 12 measurements for 7 hours, or 84 points per physical magnitude.

  • 4 tables of 84 elements (time, temperature, atmospheric pressure) = 4 x (8 + 12 x 84) = 4064 bytes
  • 2 tables of 7 elements (for histogram in bar) = 2 x (8 + 12 x 7) = 184 bytes

That is a total of 4248 bytes. For security, we will round up to 4400 bytes. If a small space is missing, all measuring points can not be recorded.

Now, we create the JSON root  object in this buffer

Then we create the keys that will contain the arrays of data. We have the choice between adding an array (createNestedArray) or an object (createNestedObject). Here, 5 tables are added. The size of the arrays is not fixed in advance, to us to calculate the space necessary at the time of the creation of the buffer. If there is no more space in the buffer, it will not be possible to add additional data.

We also take advantage of this to create a buffer that will be used for exports. If data is to be exported (which is usually the case), it will be necessary to divide the available memory into two, the first part for the JSON, the second for the export buffer.

Manipulating the JSON: Adding, Modifying, Deleting Data

Now that the JSON object is ready, we can add points regularly. Here, we will store in the JSON a point every 5 minutes. There are several functions for manipulating data in the JSON:

  • add(data): adds new data afterwards. A new value is pushed into an array, for example.
  • removeAt(index): Deletes the data at the specified index.
  • set(index, data): refresh the data at the specified index.

The following addPtToHist function adds a measurement point for each variable at regular intervals (here intervalHist = 5000 ms). The time is retrieved from the NTP server using the NTP.getTime() method presented in the previous tutorial. The number of significant digits for decimals is limited by using the double_with_n_digits method(variable, number_significant_number).

Finally, since it is necessary to limit the number of recorded measurements (here 84 per variable), the oldest one must be deleted. To do this, it is sufficient to test the size of the array, if it is greater than the size defined, the first element of each array is removed using the removeAt (index) method. It is thus very easy to set up a circular buffer.



Other useful functions:

  • size(). Returns the size of an array (array.size ())
  • success(). Returns true if an array was successfully parsed (converted from a string to a JSON object) or allocated

Export the JSON and save it in the SPIFFS area

You should have noticed the call to the saveHistory () function in the previous function. Let’s see how to export a JSON object and save it in a simple text file in the SPIFFS space.

Small reminders:

  • The library FS.h is called at the start of the program
  • To access the files stored in the SPIFFS area, you must initialize the library by calling the SPIFFS.begin () function in the setup ().

Now you can access the SPIFFS file system. Exporting the JSON to a text file is three lines of code with the ArduinoJson and FS! Libraries.

  • The file is opened with the SPIFFS.open method (filename, method). Here we will access the file in writing (w). If the file does not exist, it will be created.
  • The printTo method allows to serialize, that is to say to transform the JSON object into a character string. We indicate the output of the function, here the open file (historyFile).
  • We close the file (close).
  • It’s over.

The printTo method is very handy, so it can serialize the JSON and send it both to a file and to the serial port for program debugging. There is also the variant prettyPrintTo which makes it more readable (for a human!) The object JSON as does http://jsonlint.com/.

Import and parse a text file into JSON: reload history

We will reload the history previously saved at the start of the ESP8266 and continue the data acquisition. To parse a character string into a JSON object, we must have a buffer whose size corresponds to the final JSON object. As we will reload the current history, we already have the root buffer. No need to create a new one. We open the file with the SPIFFS.open method but this time in read only (parameter r). Before converting, it is better to test if the file is not empty (file.size ()).

The content of the file (source) is loaded into a temporary buffer (buf). The jsonBuffer.paseObject method is then passed to the buffer (buf.get()).

Warning. It will take enough memory to recharge the JSON.

The success method verifies that the JSON object is correct.

In the next tutorial we will see how to prepare a response from elements extracted from the JSON before sending it to the WEB interface.

Examples used to write the article

JSON :  https://github.com/bblanchon/ArduinoJson/wiki
SPIFFS : https://github.com/esp8266/Arduino/issues/1685
SPIFFS : https://github.com/esp8266/Arduino/blob/master/libraries/esp8266/examples/ConfigFile/ConfigFile.ino

Subscribe to the weekly newsletter

No spam and no other use will be made of your email. You can unsubscribe anytime.

Tags:

DIY Projects