What intrigued me about azure IOT suite was how easy it was to create a templated solution, connect it to a hardware device and be able to visualize the live streaming data on a chart within minutes.
All the device had to do was to send its metadata and telemetry data. Both the data packages were in JSON format. The data was self-describing.
That raised my curiosity about how this is done under the hood. This blog explores various azure resources used as well as the flow of metadata and telemetry data within the system.
Azure Resources:
The generated resources consist of:
Azure IOT Hub: (1 Unit)
This is the interaction point for the connected device(s). It consumes data from the device(s) and sends the message to device. It is a bi-directional communication hub. The data that is consumed is sent to Stream Analytics job(s).
$ Tip: This is created at the S2 layer. You can change it to the S1 layer for 1/10th of the cost. It cannot be converted to a free layer.
Azure Stream Analytics: (3 instances)
Remote monitoring solution creates three azure stream analytics instances.
1. Device Info
This consumes the metadata stream of the device and creates events that are processed by event processor.
$Tip – If all devices are registered and need not be updated, this job can be shutdown.
2. Telemetry
This consumes the telemetry data from the device. The output is raw data as well as summary data. It is stored in blobs storage.
3. Rules
This consumes telemetry data from the device and obtains device rules from the blob. Based on the values, it invokes rules as necessary. If a rule is invoked, it is stored in the blob as history. The rules can integrate with logic apps to send notifications.
Azure Document DB (1 S2 unit)
Document DB stores one document per device to store the metadata of the device. Apart from the metadata, the document also stores the commands for the device and the command history of the device.
Azure Event Hub(2 basic throughput units)
One unit is intended for device metadata information. DeviceInfo stream analytics job outputs events to this hub. The events are consumed by the events processor.
The second unit is intended for storing the history of invocation of rules. The rules stream analytics job outputs events to this hub. The events are consumed by the events processor.
Azure Storage (1 account)
The generated Azure storage consists of two tables and several blobs.
1. Azure Tables
The DeviceList table stores the Name, web address and key of the device.
The DeviceRulesNormalized table stores rules for the devices.
2. Azure Blobs
The storage solution creates 5 blob containers.
ActionMappings to store relation between ActionID and RuleOutput.(JSON format)
DeviceRules to store rules for devices.(JSON format)
DeviceTelemetry to store both raw data as well as Summary data.(CSV format)
<name>-ehout to store output of the event hub
<name>-ehruleout to store output of the rule event hub
Azure App Services (2 S1 and 2 P1 instances)
1. One App Service (with 2 instances) is created for hosting the portal web site.
$Tip – This is a premium service. Turn it’s Always On feature to off and then you can lower it to free level.
2. Another App Service is created to host two web jobs
- Event Processor – This processes events from both the event hubs and stores the data accordingly.
- Device Simulator – This hosts 4 simulated devices. These devices create events and send the data to the IOT hub.
$Tip – Shut device emulator down if not required.
$Tip –This is created with two S2 standard levels. Bringing the levels down would save your cost.
Apart from the above, the solution also creates the Azure Active Directory application and a Bing maps application.
Data Flow – Device to IOT Hub:
A device sends two types of data to IOT hub.
1. Metadata
When a device is registered (or updated), it sends metadata to the IOT Hub. The IOT hub redirects it to the Stream Analytics job that is looking for “DeviceInfo”. The Stream analytics job creates events in the event hub. The events are processed by the Event processor. Eventually, it is converted to JSON document and stored it in the DocumentDB.
The portal uses this data to display the information about the device. The portal also checks the commands available for the device and enables the user to send them to the device.
2. Telemetry data
The device periodically sends raw telemetry data to IOT hub. The IOT hub exposes event hub compatible endpoints. This data is observed by two Stream analytics jobs.
a. Telemetry Stream analytics job consumes the stream, converts the raw data to csv files and stores it in the devicetelemetry blob. This data is consumed by Power BI to display the graph.
It also creates aggregate data and stores it in the DeviceTelemetry-summary blob. The portal consumes the data to display the summary information.
b. The rules stream analytics jobs consumes the IOTStream, devicerules blob and monitors the data to activate the rules as necessary.
It sends events to the DeviceRules hub that are processed by the Event processor. It also stores the history of rules in DeviceRulesMonitor blob.
Data Flow – IOT Hub to Device:
When the user wants to send a command to a device from the portal, the JavaScript relays command to the DeviceCommandController. The controller sends it to the device logic that adds it to the command history. The command is sent to device by the IOTHubRespository using the DeviceManager that utilizes ServiceClient.
When the device receives the command, it can accept and process it or reject it. The server waits for accept/reject signal. If the signal is not received, it will retry to send the command.
For more info, check-out these links.
Sameer's Technology Blog 
Interesting,I have interest in that Microsoft Azure IoT too
LikeLike