How to build an internet-connected traffic meter

This project demonstrates how to build an internet-connected flow meter that can continuously measure the flow of a river and provide real-time data to the user online.

The system can be deployed to remote rivers with a one-time setup and can run on its own, automatically collecting data without the need for a user to be present. Multiple meters can be placed throughout a river to collect many data points.

The measurement data is pushed to the cloud via the system’s cellular connection, enabled by a Soracom Global SIM card, and can be viewed online via Soracom Harvest. AWS IoT and CloudWatch services are used to send email notifications to users when river flows reach critical levels.

The flow rate of a river measures the amount of water flowing past a specific point over time. This rate rises and falls as water from rainfall flows in and out of the river system.

Measuring the flow rate of a river can provide valuable information for a variety of purposes:

Hydrology: Long-term trends in flow rate provide insight into the watershed systems that feed the river and how they change over time.

Ecology: The speed at which a river flows determines whether it can support different types of plants or animals.

Flood warnings: Flow peaks can provide early warning of impending flooding and can be linked to rainfall reports.

Recreational safety: Real-time measurements can alert canoeists/kayakers if it is safe to travel along a river.

Currently, measuring flow in remote river areas is a very hands-off process. Surveyors need to carry their measuring equipment on foot to the river of interest, set it up, take a measurement, dismantle it, and then hike back.

Because of the effort required to collect the measurements, they are only done periodically for certain rivers. Without constant monitoring, many of the insights that flow provides are lost.

This project aims to address this problem by making traffic data easier to collect and more easily available to users who need it.

How it works

The IoT flow meter consists of sensors and electronics attached to a PVC pipe support structure that can be deployed in rivers to collect flow data.

The flow meter sensor is connected at an adjustable height to a vertical pipe of the structure, which is submerged under water. The electronics are contained in an upper housing on top of the structure outside the water.

The system uses the YF-S201 liquid flow meter sensor, which contains a magnetic pinwheel that is rotated by the incoming water.

As more water enters the sensor, the wheel spins faster, and this rotation speed (Hz) can be used to calculate how many litres of water flow through the sensor per minute (L/min), known as the flow rate.

Note: This flow rate only represents the liters of water passing through the sensor itself, not the entire river. The flow rate at the sensor is closely related to the actual flow of the river, but it is not directly measured. The IoT flow meter only provides the flow rate at the sensor and uses it as an approximation for the entire river.

A 3D-printed bracket is used to attach the sensor to the PVC pipe structure, and a 3D-printed funnel directs river water to the sensor's inlet.

The system's electronics include a Raspberry Pi, 3G USB modem with a Soracom Global SIM, and external battery, all contained within a 3D printed enclosure. The sensors are connected to the Pi via water-sealed jumper wires that extend the length of the vertical pipe into the enclosure.

The Pi runs a Python script that collects flow measurements from the sensors every minute and pushes them to the cloud using the cellular connection provided by the modem and SIM card.

The script makes an HTTP POST request to Soracom's unified endpoint, which forwards the data to two services: Soracom Harvest and Soracom Funnel.

Soracom Harvest allows for easy real-time visualization of incoming data. The console displays a real-time graph showing updated data values ​​as they flow in. Users can monitor how data changes over a selected time window.

Soracom Funnel takes the data and forwards it further by sending it to AWS IoT endpoint, which then pushes the data to AWS CloudWatch using AWS IoT rules.

A CloudWatch alarm is configured to monitor when the flow exceeds a critical value and automatically sends an email notification to any subscribed users to let them know that the alarm has been triggered. This feature allows users to stay informed of important changes in river flow without having to monitor it in real time.

Architecture and Operational Excellence

IoT flow meters are designed to solve problems that have traditionally required more hands-on, expensive equipment to collect data. While this prototype has shown that it can come in handy and deliver results, there are more things to consider when moving to a production version.

Cost: The total cost of the system is about $120, with the largest expenses being the Raspberry Pi ($35) and the 3G modem ($60). This price will likely drop as volume becomes available. This total cost is reasonable considering the man-hours saved by using this IoT device instead of paying a surveyor/engineer to manually collect measurements.

Scalability: The system can be expanded to more units as demand demands. By transitioning to prefabricated circuits containing all the necessary electronics and developing better pre-assembled pipe structures, it will become easier to build large numbers of units.

If you scale to 1000s of devices running around the world, Soracom and AWS IoT could still be the backbone of how data is collected and distributed. It may be necessary to set up individual accounts and access permissions for each user, AWS IoT can achieve this through IAM user groups, which only grant permissions to specific users to view/edit data.

Reliability: Consistent and accurate measurements that reflect actual flow are important for this problem. More testing is needed to better understand the different influences that can affect sensor readings, such as depth and location in the river.

The system also relies on a reliable cellular network. Currently, if a request attempt to Soracom fails due to a network timeout, that data is lost. Further updates to the tool will include backing up measurements so they can be saved and pushed to the cloud when a connection is reestablished.

Security: IoT flow meters are designed to be unattended, which puts their physical security at risk. Currently, there is no way to determine what will happen to the system if it goes offline or is lost, but future versions may include a GPS module that contains the current coordinates of the system at each data push. This will help track the location of the system at all times.

Build Instructions

Below are instructions on how to prototype, build, and receive notifications from an IoT flow meter.

Start this project by first building a prototype circuit to test the functionality of the liquid flow meter, enabling internet connectivity and pushing measurements to the cloud. Next build the pipeline structure and attach components so it can be deployed. Finally, set up AWS CloudWatch and subscribe to email alerts when critical flows occur.

These instructions assume that you have access to the Rasberry Pi's terminal via SSH or a keyboard/mouse/monitor setup. The Pi must also be connected to the internet via WiFi or Ethernet for initial setup.

Prototype Circuit

Step 1: Set up the test circuit

Using male to female jumper wires, make the following connections from the wires from the YF-S201 Liquid Flow Meter to the Raspberry Pi GPIO pins:

Red (DC power) to pin 1 (3.3V)

Black (Ground) to Pin 6 (Ground)

Yellow (output) to pin 7 (GPIO4)

Step 2: Test Sensor Measurements

You can run a Python script to get readings from the now connected sensor. Open the Raspberry Pi terminal and enter the following commands to create and navigate to the flowmeter directory on the Pi desktop where the script will be stored:

Using Python IDLE or another text editor, copy the following Python code and save it as flowmeter.py in the folder you just created:

import json
import time
from datetime import datetime
import RPi.GPIO as GPIO

class FlowMeter():
 ''' Class representing the flow meter sensor which handles input pulses
and calculates current flow rate (L/min) measurement
 '''

 def __init__(self ):
self.flow_rate = 0.0
self.last_time = datetime.now()

 def pulseCallback(self, p):
''' Callback that is executed with each pulse
 received from the sensor
'''
 
# Calculate the time difference since last pulse recieved
current_time = datetime.now()
diff = (current_time - self.last_time).total_seconds()
 
# Calculate current flow rate
hertz = 1. / diff
self.flow_rate = hertz / 7.5
 
# Reset time of last pulse
self.last_time = current_time

 def getFlowRate(self):
''' Return the current flow rate measurement.
 If a pulse has not been received in more than one second,
 assume that flow has stopped and set flow rate to 0.0
'''
 
if (datetime.now() - self.last_time).total_seconds() > 1:
 self.flow_rate = 0.0
 
return self.flow_rate

def main():
 ''' Main function for repeatedly collecting flow rate measurements
and sending them to the SORACOM API
 '''

 # Configure GPIO pins
 INPUT_PIN = 7
 GPIO.setmode(GPIO.BOARD)
 GPIO.setup(INPUT_PIN, GPIO.IN, pull_up_down=GPIO.PUD_UP)

 # Init FlowMeter instance and pulse callback
 flow_meter = FlowMeter()
 GPIO.add_event_detect(INPUT_PIN,
GPIO.RISING,
callback=flow_meter.pulseCallback,
bouncetime=20)

 # Begin infinite loop
 while True:

# Get current timestamp and flow meter reading
timestamp = str(datetime.now())
flow_rate = flow_meter.getFlowRate()
print('Timestamp: %s' % timestamp)
print('Flow rate: %f' % flow_rate)
 
# Delay
time.sleep(5)

if __name__ == '__main__':
main()

This script defines a FlowMeter() object class that tracks the pulses received from the sensor as its pinwheel rotates and calculates the current flow rate.

Each new pulse triggers the pulseCallback() method, where the rate of pulses per second (Hz) is divided by a constant to calculate the flow rate in liters per minute (L/min). This conversion is defined in the sensor's datasheet. The getFlowRate() method returns the current flow rate.

The main() function, which is called when the script starts, defines the GPIO pins used by the sensor and starts an instance of the FlowMeter() class.

An infinite loop is entered, where flow rate measurements and their timestamps are collected and printed to the console on each pass. The time.sleep(5) statement that controls the delay between loops is set to 5 seconds. The loop will run continuously until canceled by the user.

Before deploying the final IoT stream meter to the river, edit this script to adjust this sleep time to how often you want to collect river measurements. A value of 60 will take a measurement every one minute.

Start the Python script using the following command:

The flow rate measurements and timestamps are printed to the console every 5 seconds:

When the sensor's windmill stops, the flow rate should be zero. Try blowing air into the sensor to spin the wheel and observe how the flow rate value changes.

Press Ctrl+C to stop the script.

Step 3: Enable cellular connectivity

With the sensor working, the next step is to send its readings to the cloud. This is achieved via a 3G USB modem with a Soracom Global SIM card plugged into the Pi and Soracom's Harvest service.

Soracom's own IoT State Machine Kit tutorial provides good instructions for setting up with a Soracom account, configuring the USB modem, and enabling the Harvest service. Complete the following sections from their tutorial:

Set up your Soracom account

Setting up a 3G USB modem

Enable Soracom Harvest

Step 4: Send data to Soracom Harvest

The flowmeter.py script can now be updated with additional code to send measurements to Soracom Harvester.

First install the requestsPython package, which allows making HTTP requests from Python, using the following command:

Open flowmeter.py and add the following line to the list of import statements:

Add the following code inside the main() function above the statement time.sleep(5):

This additional code wraps the timestamp and flow rate values ​​into a JSON string and sends it to the Soracom unified endpoint using a POST request.

A completed version, flowmeter.py, is attached to this tutorial. Run the script again using the following command:

With the Harvest console open, you should start to see data flowing in. The flow data is plotted on a graph, which updates each time a new value is received.

Step 5: Run the script at startup

In the final setup of this project, the Raspberry Pi will be disconnected from the monitor, mouse, and keyboard and powered by an external battery. Therefore, it is necessary for the Python script to start automatically without user interaction.

Configure the Pi to run the script at boot time by editing the file /etc/rc.local. Edit this file with the following command:

Add the following line above the last line of the file, exit 0:

Press Ctrl+X, Y, and Enter to save and close the file.

The full path to flowmeter.py is required because the Pi will not be in the directory when flowmeter is started. Including & at the end of the line ensures that the script runs as its own process and does not block other started processes while it is running.

Disconnect the Pi from any monitor, keyboard, and mouse, and plug the external battery pack into the Pi's micro USB port. It may take a few moments for the Pi to boot up and the USB modem to connect. Once connected, open the Soracom Harvest console to view incoming data.

Assembling the structure

Step 1: Build the structural foundation

A PVC pipe structure holds the components in place, keeping the sensor underwater and the electronics above water.

Begin by using a hand saw to cut the PVC pipe into four 15.5-inch sections and two 7-inch sections. Using tee fittings, assemble them into the "H" base of the structure.

Use a 5' foot piece of PVC pipe as the vertical bar that will be inserted into the center of the "H" base.

Step 2: Waterproof the Sensor

Since the flow sensor will be submerged underwater, it needs to be protected from any water damage. The length of its connecting wire also needs to be extended to extend the vertical pipe up into the electronics housing.

Begin by cutting the end caps off the wires. Then, using three 4-foot jumper wires, extend the length of the DC power (red), ground (black), and output (yellow) wires by soldering the ends together. Individually wrap each soldered connection with a small piece of heat shrink tubing.

The ends of the full-length wires should be individually connected to female jumper end caps so that they can easily connect to the GPIO pins of the Raspberry Pi. Cut 3 male-to-female jumper wires and solder the female jumper end caps to the three full-length wires. Wrap each connection individually with heat shrink tubing.

Use a 4 foot length of heat shrink tubing to wrap all three full length wires that are now coming out of the sensor. Make sure the tubing extends all the way to the bottom of the sensor to help prevent water from entering the sensor housing.

Attach the sensor to the 3D bracket by removing the faceplate screws from the sensor, placing the sensor inside the bracket, and then attaching it to the bracket wall using the same screws.

Silicone caulk should be placed around the exterior of the sensor and the wire openings for added waterproof protection.

Attach the bracket and sensor to the vertical pipe structure and secure the extension cord to the pipe using nylon tie wraps. Place the 3D printed funnel over the mouth of the sensor.

Step 3: Electronics Enclosure

Once the electronics enclosure is 3D printed, attach it to the top of the vertical duct using rigid conduit straps and 1/4" hex bolts and nuts. The bolts will go through the strap holes on the back of the enclosure.

Once you are ready to deploy the sensor, place the Rasberry Pi, battery, USB extension cable and 3G USB modem into the enclosure and power on the Raspberry Pi. Feed the sensor's heat shrink wiring through the holes in the bottom of the enclosure and connect to the Pi's pins.

Attach the faceplate to the enclosure box using 4-40 1/2” screws.

The structure is now ready to be lowered into the river. Find a spot that will keep the house above water, and a spot with a relatively flat bottom so the structure can remain stable.

Email Notification

Step 1: Create a new managed user in AWS

First create a new managed user that will be granted permissions to access AWS IoT programmatically. Soracom Funnel will access AWS IoT through this user.

Open the AWS IAM Management Console and click Users in the left menu, then click Add User.

Enter the User Name (e.g. Funnel) and check Programmatic Access in the Access Type section, then click Next: Permissions.

Click Attach existing policies directly and type "iotdata" in the search box. From the results, check the "AWSIoTDataAccess" permission. Click Next: Tags in the following page, then Next: Review and Create User.

On the next page, copy the Access Key ID and Secret Access Key into a text file.

Step 2: Get the AWS IoT endpoint

Open the AWS IoT console and click Settings in the left menu. Copy the Endpoint URL from the Custom Endpoints section.

Step 3: Configure Soracom Funnel

Open the Soracom User Console and click on the group name of the SIM card used for this project. In the "Soracom Funnel" section, enable the service by clicking the toggle button to "On".

Select AWS IoT as the Service.

Set the “Destination” to “<your_custom_endpoint> /funnel/flowmeter ”, replacing the custom url you copied in the previous step (without the quotes). This is the endpoint and AWS topic (funnel/flowmeter) that the data will be forwarded to.

Click the "+" button to register a new credential set.

In the pop-up window, set the Credential Set ID to the new ID (eg Soracom) and fill in the Description (eg Soracom IAM User in AWS IoT).

Select AWS Credentials as the Type and paste the AWS Access Key ID and AWS Secret Access Key copied from step 1 into the corresponding fields.

Click the Register button.

From the Credential Set drop-down list, select the credential you just registered (for example, Soracom (Soracom IAM User in AWS IoT)).

Click the Save button.

Step 4: View the data in AWS IoT

Verify that data is being forwarded from the Soracom Funnel to AWS IoT by subscribing to the topic we assigned in the previous step.

Open the AWS IoT console and click Test in the left menu. In the MQTT client window, enter "Funnel/Flowmeter" as the Subscribe Topic and click Subscribe Topic.

Messages published from the hopper to this topic should start to appear. These messages contain the original payload from the sensor, as well as some additional metadata from Soracom.

Step 5: Send data to CloudWatch

The next step is to send this incoming data to AWS CloudWatch as a custom metric. CloudWatch allows for visualization of metrics and notifications.

Open the AWS IoT console and click on "Act" in the left menu. Click "Create" on the following screen.

From the Create Rule menu, enter a rule Name (for example, CloudWatch) and Description.

Enter the following statement in the Rule Query Statement: SELECT * FROM 'funnel/flowmeter'. This statement will apply this rule to all messages sent to the "funnel/flowmeter" topic.

Click on “Add Action”. On the next screen select “Send message data to CloudWatch” and click on “Configure Action”.

In the next menu fill in the following:

Metric name: flow_rate

Metric namespace: flowmeter

Unit: None

Value: ${payloads.flow_rate}

Click Create Role and type a name for the new role (e.g. CloudWatchRole), click Create Role again, and use the Select button to select the newly created role from the drop-down menu.

Click the Add Action button. The flow_rate data from the incoming message will be forwarded to CloudWatch.

Step 6: Set up notifications

Open the AWS CloudWatch console and click Metrics in the left menu. Under Custom Namespace, click Traffic Meters.

Click "Metrics without dimensions" and select "flow_rate". The CloudWatch graph will populate with the flow rate values ​​for the selected time window. The graph can be set to automatically update so that new data is plotted as it flows in.

Click the bell icon to set an alert for that metric. Fill in the alert Name (e.g. FlowRateAlarm) and Description on the next screen.

The Alert Details section defines when the metric is in an alert state. For this project, this means when the flow_rate reaches a value that is important for the user to know. For this example, a value of 1.0 was chosen, but a more realistic number could be determined by looking at the incoming data to determine what flow rate a normal river flow generates and setting it higher than that number.

In the Actions section, select New List and enter a name for the new notification list group (for example, cloudwatch_alarm). Select the list and enter any email addresses you want to include in this list.

A confirmation email will be sent to these addresses to confirm their subscription to notifications.

Click the Create Alert button.

Now that the alarm is created, CloudWatch will continuously monitor incoming data for this metric and if it enters the alarm state (e.g. flow_rate >= 1.0), it will send a notification email to the subscribed users.