Take the time and answer these questions before you start

When you do an IoT project, the first step is to write down the project requirements. Take the time and think about what you are trying to do. Even a simple project like having a sensor sending you some data, can have complexity. Is it sending data via wire or via the Internet? If it’s via the Internet, do you use a 3G data plan and a SIM card? Or do you have wi-fi and can select something like an ESP8266 that can connect to wi-fi. And how do you configure the device to connect to wi-fi?

Make sure to spend the time now to think about all possibilities and ways you might extend it to avoid getting stuck in a corner later. If you intend on doing something extra with the device, maybe it’s worthwhile to spend a little more time on it now to make sure it’s extensible in the future. If you don’t have any concrete plans to extend it, then maybe it’s not worth optimizing now for something that might never come.

Here are some questions you should ask yourself:

• Does it need configuration via web either mobile app or web app?
• Does it need to be connected to the Internet?
• How will you secure it?
• How much data does it send over the Internet?
• Does it need to store any persistent data?
• Do you need to restore data in case of power failure?
• Does it need to know the time and date?
• How many people will work on the firmware?
• Does it need to keep logs and sync them with a server?
• Does it need to have a web server and API to send data to?
• Does it need OTA (Over The Air updates) ?
• How will you flash the firmware onto the device?

Let’s go through them.

If you or the final user needs to be able to enter some configuration data every couple of days or when you install the device, it’s a good idea to have an interface to do so. Especially if the final user needs to input some metrics that are custom to him. Our device, for example, had several options that were up to the user to input. The water hardness level and the dates when filters are changed. You have several options here. A web interface accessible via the device IP (it can be a fixed IP or a service that announces itself over the network like mDNS). This might not be for all users. A nice option is to have a mobile app that talks to the device. This however requires more work and adds complexity to the project. You will have to balance user friendliness with comDoes it need configuration via web either mobile app or web app?plexity. We went with a mobile app approach which I will discuss in a later chapter.

Does it need to be connected to the Internet?

The first iteration of the device wasn’t an Internet connected device and it worked perfectly. What the Internet added was convenience, meaning more value for the final customer. They could start / stop the device, change its schedule and see live data from anywhere in the world. One destination of the device are cabins. So letting the device plug in while you are away and still be able to see live-data brings a lot of value for the customer. Give it a thought if your device really needs Internet access. Is it worth adding all the extra layers just to have it on the Internet? If the connection to the outside world doesn’t bring some value, just skip it. You will get to a working project so much faster.

How will you secure it?

And I’m not only talking about Internet connected devices. Is your device temper proof? Can a kid mess it up? Are all wires neatly tied together and not reachable by a child? It might not seam important until you get a kid or some friends come over and their kids or pets start looking for things to pull apart. If it’s connected to the Internet, than you have a new set of problems to solve. If you are collecting user data, you have privacy issues to think about. Your device might leak user activity, if his home or not, for example. Do not collect more data than you need, be very explicit on what you are collecting, how you store it and who has access to it. Make sure you do a very good job at it, when asking for the end user to entrust you with his data. You also need to make sure the device can’t be tampered with remotely. This subject as a chapter on its own.

How much data does it send over the Internet?

If you have it connected over wi-fi, then this might not be an issue. But if you are using a data plan and a SIM card, think hard of what data you need to send, in what format and how often. This will save you some money down the line.

Does it need to store any persistent data?

This is a very important thing. The available space to save data is very limited on boards like ESP8266 or ESP32. They have only 1-4 MB of available space, and this includes the firmware itself. The flash memory also has a finite number of writes you can do. So special attention needs to be given to what is persistent to disk. I had to store some counter values and dates, you might have more or none at all. You also need to make sure to have enough space for OTAs to be downloaded. The way a good firmware works is by downling the update, creating a second partition, booting from it, then, if everything is ok, committing that version to disk and deleting the previous partition. This allows you to do rollbacks in case something is messed up. You don’t want to end up with a bricked device.

Does it need to know the time and date?

If the project needs to work with dates and time, like a schedule in my case, you will need Internet access and a way to sync to an NTP server. Otherwise you can’t get the date correctly if the power goes down. I had a schedule for the device to start and stop, so I was very happy that the firmware had the ability to sync time automatically. This was a device feature that simply can’t work with Internet access.

How many people will work on the firmware?

This will dictate how you organize your code. Instead of having everything in one file, it might be worth splitting them in several files to avoid having merge conflicts later down the line. Even if you are not part of a team, use git to version your source code. Make sure to write good comments for the commits. It makes your life easier when looking back at code. Use branches to develop features, before merging into the master branch. There is a chapter for this.

Does it need to keep logs and sync them with a server?

If you need to store logs of events, values, than you need to either consider storage options like flash or SD card or syncing to an external server. The flash and SD card are not the greatest options for logs, especially if it’s something write intensive. I would suggest avoiding the flash for this completely and go to SD card. But these are prone to failures, so it depends on your use case and how important that log data is. If you use an external server, you need to take into account the situation where either the server is unavailable or your connection dropped. You need to either store them locally for some time or discard them and continue logging when the connection comes back on.

Does it need to have a web server and API to send data to?

This ties into the previous question. You need some web API to receive the data and a way to visualize it. You can also offer the end user access to this data. You also need a server in order to allow connection securely to the device from outside and control its state via commands or shadow. This will be explained in more detail in a later chapter.

Does it need OTA (Over The Air updates) ?

OTA is the service where the device checks with some frequency if there is any firmware available, These are usually some sort of archive that can be download, written to a partition and the device can boot from that partition. This service is very important if you plan on fixing bugs, without going to the location of the device to manually flash it. This was not something I could do for the device as there is no way a client would be happy to have me in his house, plug my laptop into the device, flash it and then come back in a couple of months for another update. You might think you don’t need it and it might be true if the device you are working is doing something simple, but add a couple of inputs, outputs and you will start looking into OTA after you have to move your ass and connect a wire to a device several times. There are several platforms and ways you can do OTA, I will go into more detail in its own chapter.

How will you flash the firmware onto the device?

If you are mostly a web/mobile developer like I am, with not much time or a desire to have the smallest number of wires possible on the desk. I recommend you buy a more expensive board that has a micro-usb port or a separate programmer board with this connection already on board. It makes things so much easier than having an extra wire to worry you will break if you are not careful. The though of having to solder wires to get a connection going just makes me… balk at the thought. Did I mention I never soldered anything in my life and is not a skill I am eager to acquire unless my life depends on it? :D