Trukdoc – In-transit Vehicle Predictive Maintenance

About the project

An AI-based IoT solution for in-transit vehicle monitoring on Nordic Thingy:91

Project info

Difficulty: Expert

Platforms: ArduinoSilicon Labs

Estimated time: 2 weeks

License: GNU Lesser General Public License version 3 or later (LGPL3+)

Items used in this project

Hardware components

Sensor Tool W Nrf52840 & Nrf9160 Sensor Tool W Nrf52840 & Nrf9160 x 1
Arduino Uno - R3 Arduino Uno - R3 x 1
Raspberry Pi Zero W Raspberry Pi Zero W x 1
Raspberry Pi Camera Module V2 Raspberry Pi Camera Module V2 x 1
Transceiver Breakout - Nrf24l01+ Transceiver Breakout - Nrf24l01+ x 1
Lithium Ion Polymer Battery 3.7v 1200mah Lithium Ion Polymer Battery 3.7v 1200mah x 1
Tiny Battery Charger Tiny Battery Charger x 1
Rotary Encoder Rotary Encoder x 1
Gas sensor - MQ135 Gas sensor - MQ135 x 1
Air Pressure sensor Air Pressure sensor x 1
Magnetic Field Sensor Magnetic Field Sensor x 1

View all

Software apps and online services

Arduino IDE Arduino IDE
Simplicity Studio Simplicity Studio
Edge Impulse Edge Impulse
Thonny IDE Thonny IDE
IFTTT IFTTT
Monogoto Cloud Monogoto Cloud

Hand tools and fabrication machines

3d printer 3d printer x 1
soldering iron soldering iron x 1
Jumper wires Jumper wires x 1

Story

Trukdoc – an all-in-one in-transit vehicle predictive maintenance AIoT solution

The Problem

Fleet owners in India suffer a huge loss of time and money because of unwanted breakdowns and sudden component failures in their trucks. This is caused by the undetected problems in truck components due to the ignorance of both the fleet owners and truck drivers, high cost of regular maintenance and undesirable downtime experienced with each checkup. Breakdowns consequently lead to major downtime, exacerbated by a lack of mechanical assistance on highways and at night. Another common issue faced by Indian truckers is drowsiness due to long journeys at night hauling cargo, which leads to accidents and deaths.

Despite being such a critical component of the Indian supply chain, the trucking industry is plagued with problems ranging from monetary and safety constraints to environmental issues. Today, many fleet management solutions exist which solve the problems encountered by a fleet owner in pre-transit and post-transit. However, there are a multitude of problems faced by the driver and the owner in the most critical part of the whole journey i.e. the “in-transit” part, which remain unsolved.

Poor truck maintenance and driver drowsiness are the major causes of truck accidents. Trucks were involved in 13% of all road accidents in India in 2018, which amounts to around 57000 accidents, and 150 thousand casualties

Pictorial Representation of the problems faced by various stakeholders in the trucking industry

A video interview with a truck driver was what opened my eyes to this huge problem.

With the above problem definition, it is clear that there is a pressing need for an in-transit vehicle telematics system specialised for Indian trucks. We studied several competitors in the vehicle telematics sector and found that none of them are solving the on-ground problems faced by Indian fleet owners and truck drivers. They are either creating generic solutions catering to all vehicles which cannot solve specific problems present only in Indian trucks, or providing telematics for newer models of trucks which don’t cater to the 7.6 million trucks running on Indian roads today.

Through our suite of solutions, we bridge the gaps in the existing solutions to reduce cost incurred per truck and increase safety of drivers in transit. Our solutions include driver drowsiness detection, DEF (Diesel Exhaust Fluid) level detection and coolant pipe monitoring, tyre pressure and wheel alignment monitoring, brake and clutch health monitoring and axle bending detection, with an overarching feature of updates provided even without legacy cellular connectivity through the NBIoT protocol.

Additionally, I benchmarked our solution with international players such as ScanGauge and FieldLogix where problems specific to the Indian market such as lack of connectivity and urea detection were not addressed. My findings have been presented in the following table.


From the detailed analysis and competitor benchmarking, it is very clear that there is a gap in the Indian trucking industry and consequently a need for the comprehensive yet user-friendly IOT-based solution that we are offering.

TrukDoc aims to support fleet owners and empower truck drivers by tracking vehicle health in-transit to provide early detection of common vehicle problems and driver monitoring, using our truck telematics solution to reduce cost incurred per truck and boost safety through decreased risk of accidents.


Features

Our solution comes with a wide range of features including monitoring of truck components like the brake, clutch, axle, tires, detecting driver drowsiness, and getting updates even without cellular network, among others. The following figure lists down the key functionalities offered.


The User Interface

A key component of the solution is the user interface that acts as an abstraction layer between the underlying deeo tech and the technology agnostic driver or fleet owner. The three components of the UI are –

-        Dashboard display

-        Website

-        Mobile Application for driver

Driver Application

Driver app informs the truck driver about any component damage and provides nearby mechanic contacts. It is equipped with assistive UI that provides audio-visual assistance to help drivers in navigating the app. The picture below gives an idea of the look and feel of the app.


Dashboard display

In case of component damage or other problems, a warning is displayed on this dashboard installed near the steering, and a red LED starts to blink. The picture below gives an idea of the look and feel of the dashboard (Sorry about my horrible CAD skills)


Website Dashboard for fleet owner

In order to track the location, issues and warnings for all trucks in the fleet, the fleet owner is equipped with a dashboard. Through this, they receive live updates of the location of the truck, component damage issues or warnings and contacts of nearby mechanics in case of vehicle breakdown. The picture below gives an idea of the look and feel of the webapp that can be accessed at -

https://trukdoc.netlify.app/dashboard

The video below shows a demonstration of fleet management and realtime alerts on the website.



Overall Architecture

The overall architecture of the solution is shown in the figure below. It displays a Master-slave architecture where various slave nodes collect sensory data and send it to the master wirelessly which then uploads warnings and key datapoints to the cloud from where notifications are sent to end-users.


Master Node

The Nordic Thingy:91 was chosen as the processing unit for the MASTER node. The selection criteria in favor of the NRF9160-based microcontroller were its NB-IoT communication features, low cost, rich peripheral set, low sleep mode current, and compatibility with the nRF5 SDK(with which I am quite familiar by now) helping me achieve optimal performance while reducing BOM and time to market.


Schematic of Master Node


Software Architecture

The flowchart below explains the algorithm for the master node. All the slaves send warnings to the MASTER if a fault is detected. Some slaves keep sending data to the MASTER at regular intervals based on which secondary checks are performed. The GPS data is continuously streamed. A component check is performed at the beginning of each trip. All the vehicle’s logged health data is analyzed and sent to the cloud through the gateway at the end of the trip.


Software API Description


Slave Nodes

There are five slave nodes in total that monitor different parameters related to vehicle health. The following table gives a summary of the functionalities of the master and the slave nodes.Let me discuss the features packed into Truckdoc individually and how each of the nodes is realised in practice.


Power Consumption

The power consumption for a particular module is calculated based on its estimated uptime. The uptime is further divided into active & passive phases. In the active mode, most system resources are enabled & wireless transmission links are active. Passively, the microcontroller is sent to the deep sleep mode and woken up at certain predefined intervals of time to send or receive data. The duty ratio quantifies the fraction of the time for which the microcontroller is active. From the current consumption in both the modes, the power is calculated, which is then used to predict the approximate battery life of each sensor node.

The life of almost all the modules is more than one year. After that, the modules can easily be charged through a standard Micro-USB port with a portable power bank. Note that the power consumption of the Master node and the LoRaWAN gateway has not been considered since they are powered directly off the 12V power supply of the truck and only set in active mode on battery power in emergencies when the fleet owner wants to know the truck status while the truck is powered down.

Features

Since the number of nodes were so many, I thought it would be better to take the feature implementations one by one, rather than describing the nodes. The list of features along with the technology is shown below.

Driver Drowsiness Detection

Problem:

Truck drivers in India are often subject to inadequate sleep or have jobs requiring them to alternate sleep cycles which is often taxing. This results in drivers becoming drowsy while driving and potentially meeting with an accident.

Technology:

The dashboard consists of a Raspberry Pi with an infrared camera to monitor the driver's eyes. IR LEDs ensure uninterrupted monitoring even during low-light conditions and at night when most accidents occur. A pre-trained face landmark detection model is used to determine if the eyes are closed or open using EAR (Eye Aspect Ratio). If the eyes remain closed (EAR < 2.0) beyond a specific time interval (200 μs), an alarm is raised to alert the driver. If the number of warnings exceeds a certain threshold, the fleet owner is also alerted.


Software flow diagram for the drowsiness detection module

For training the model, Edge Impulse was used. The entire process is documented in a later section. A neural network based image classifier predicted if the driver`s eyes were open or closed in real-time and sent it over I2C to the Nordic Thingy:91 – the master controller.


Circuit interfacing of the detection mechanism


Final position of the drowsiness detection module


Brake System Monitoring

Problem:

For medium and heavy-load transport vehicles, it is critical for the braking system to function correctly and be maintained regularly.

Technology:

  A 6 DoF Inertial Measurement Unit (IMU) is interfaced with the MASTER to measure the vehicle’s deceleration when the brakes are pressed. At full braking pedal position, a limit switch interfaced with the MASTER detects a pedal press, after which the acceleration value is polled.

If the deceleration is below the required threshold a certain number of times, an alert is raised with the fleet owner. The threshold value is dynamically adjusted depending on the road conditions by monitoring the acceleration value of the truck over an overlapping time window.

The fleet owner can observe timely monitoring of deceleration magnitudes to decide on maintenance-related issues.


Circuit interfacing of the brake system monitoring mechanism


Software architecture of Brake system and Driving Behavior Monitor


Axle Bending Monitoring

Problem:

Trucks are often loaded beyond the maximum limit, which leads to over-the-mark stresses on the leaf springs and, consequently, the axles of the rear wheel pairs. Prolonged driving with bent axles leads to bending and breaking, leading to fatal accidents.

Technology:

  SLAVE nodes fixed at the ends of the axle quantify the bending by measuring the linear displacement of the ends using a Linear Variable Differential Transformer (LVDT) sensor to sense the bending in the axle. A warning signal is sent to the MASTER node if the linear displacement exceeds a certain threshold. The MASTER node then raises an alert with the fleet owner about the vehicle’s overloading.


Circuit interfacing of the axle bend monitoring mechanism


Placement of Axle Bend Monitor

Software architecture of Axle Bend Monitor


Clutch Health Monitoring

Problem:

Clutch plate wear due to improper shifting techniques leads to accidents. Late detection leads to higher repair costs.

Technology:

A SLAVE node reads the rotational speed of the input shaft to the gearbox. The position of a magnetic sticker on the gearbox input shaft is encoded using a hall effect sensor (AH49E). The rotational speed is calculated locally and transmitted wirelessly to the MASTER.  The MASTER compares the input shaft RPM with the engine RPM received from the ECU OBD bus. If the RPMs differ beyond a certain magnitude for a standard response time interval (1 - 1.5 s), an alert is raised with the fleet owner.

Clutch bites are recorded and stored to assess the health of the friction material over time and suggest minor repairs. The frequency of bites also gives a reasonable measure of the shifting skills of the driver.


Circuit interfacing of the clutch health monitoring mechanism


Placement of clutch health Monitor

Software architecture of clutch health Monitor


Coolant Pipe Monitoring

Problem:

Air bubbles trapped in the cooling system lead to overheating of the vehicle. Due to this, trucks can go over the safe operating temperature range leading to a blown head gasket, cracked engine block, damaged pistons, bursting hoses, or a blown radiator.

Technology:

The SLAVE node has a fluid pressure sensor that relays coolant pressure data to the main module wirelessly via nRF communication at regular intervals. This data can be used to detect possible leakage and further analyze the condition of the coolant fluids. The driver and the fleet owner are intimated if immediate servicing has to be done.


Tire Pressure Monitoring

Problem:

Operating within an optimum tire pressure range is integral to having good maneuvering characteristics and  fuel economy. 

Technology:

Our system provides one air pressure sensing SLAVE node (fixed near the nozzle on the rim) per wheel, relaying pressure data to the main module wirelessly via nRF communication at regular intervals. The main module reads the pressure data from all the tires to detect leaks or punctures and informs the driver by a message on the dashboard display about refilling air or getting the tires checked for holes. The fleet owner is also notified about possible flat tire situations to take immediate action.


Circuit interfacing of the Tyre Pressure monitoring mechanism


Placement of Tyre Pressure Monitor

Software architecture of Tyre Pressure Monitor


Wheel Alignment Monitoring

Problem:

Proper wheel alignment is crucial for the control and stability of the vehicle, and misalignment can lead to severe accidents.. 

Technology:

Improper alignment is detected by observing the variation of the rotation of the front tires about the ends of the steering rack. Our module uses three rotary encoders - one for the steering wheel rod and one each for the two front wheels - to measure rotation angles. The angles turned by the wheels and the angle turned by the steering rod are individually calculated by the respective SLAVE nodes.

and sent wirelessly to the MASTER node. The MASTER node correlates the steering rotation in a particular direction to the rotation of the individual front tires in that direction. Discrepancies beyond tolerable limits are reported to the fleet owner on high priority with a  suggestion for a wheel alignment inspection.


Circuit interfacing of the Wheel Alignment monitoring mechanism


Placement of Wheel Alignment Monitor

Software architecture of Wheel Alignment Monitor


Fuel Quality Monitoring

Problem:

An optimum air-to-fuel ratio is required for the engine’s clean and efficient combustion of fuel. The engine efficiency is heavily degraded by suboptimal grade fuel which is often substituted by opportunistic drivers in place of recommended fuel.

Technology:

The lambda sensor (connected to the ECU) measures the amount of unburnt oxygen in the exhaust gases. The MASTER module taps into the ECU OBD bus to read this data and analyzes it over time to calculate the fuel quality and engine efficiency. Failure to reduce levels of unburnt oxygen by the ECU (by air-to-fuel ratio calibration) indicates fault either in ECU or fuel pump. This is notified to the fleet owner, and an alert for maintenance is raised.  


Circuit interfacing of the Fuel Quality monitoring mechanism


Software architecture of Fuel Quality Monitor


DEF Level Monitoring

Problem:

Heavy and medium load vehicles generally use diesel for fuel. Combustion of diesel leads to a significantly high release of NOX in the exhaust gas, which is highly hazardous. Hence, exhaust gas is treated with DEF (Diesel Exhaust Fluid) to reduce NOX emissions. DEF consists of urea and deionized water. The ammonia released by the urea solution reacts with NOX to produce much less harmful gases (N2, CO2, & H2O) into the exhaust. If the level of DEF decreases below a certain threshold, the concentration of harmful gases in the exhaust increases polluting the air and attracting hefty fines from pollution control enforcement.

Technology:

The ammonia sensor (MQ-135) in our SLAVE node detects urea depletion in the DEF tank. The ultrasound sensor (HC-SR04) monitors the liquid level and assists in assessing the evaporation of the liquid from the DEF tank. Combined inputs from both the sensors are analysed on a timely basis and the fleet owner is alerted regarding the DEF replacement as and when required.


Circuit interfacing of the DEF Level monitoring mechanism


Placement of DEF Level Monitor

Software architecture of DEF Level Monitor

Model Development using Edge Impulse

Edge Impulse was used to develop two different neural networks. The first one was for detection of driver drowsiness while the second one was detection of harsh driving by the driver. For the first model, imagery data was collected manually using the data acquisition tab in Edge Impulse. For proper training of the model, I collected images of ten different people (my friends, of course) with eyes closed and open. To make it much more realisitic, I collected images with heads bowed down, sideways and even wearing sunglasses!!


Project Dashboard on EI


Data Acquisition – I used the Raspberry Pi NOIR camera

After the data was collected, I created a standard train/test split of 79% is to 21% and moved on to training. Something important to note here is that the data has to be labelled during collection. Otherwise, it becomes a pain to view each sample individually on the console and tag it.

For the training of the data, I first designed the impulse. I used the recommended settings as it had brought good results to many hacksters in the past. I scaled the images captured to 96x96 size for reducing the feature size. After that I used a Transfer Learning block which is essentially a transfer learning network that generates two outputs – open or closed indicating whether the driver`s eyes are open or closed.


Impulse Design on Edge Impulse

For the next step, I tweaked some parameters to train the model. Here comes the interesting part. I had to juggle through a lot of parameters to finally arrive at the best implementation. I must say that the EON Tuner helped a lot. I used the MobilNet V2 with a 0.1 dropout as the final model since it gave the best tradeoff of inference time, which is the time taken by the model to process results and RAM usage. Note that as the dropout is increased, the size of the network and its accuracy increases. As you would see from the EON Tuner results, increasing the depth of the networks does not always results in better accuracy.

Remember that the Raspberry Pi just has 512MB of RAM out of which a significant portion is needed to run the operating system as well. From my experiments, I advise keeping the RAM usage below 200 KB. The Model size or the Flash usage is not much of a concern for the Raspberry Pi, but it was a concern for the Nordic Thingy used for the Driver Behaviour Prediction Model.

As you can see from the picture below, I achieved an accuracy of over 90% which is great for a minion of a processing platform, that is the Raspberry Pi Zero.

Schematics, diagrams and documents

CAD, enclosures and custom parts

CAD files for all nodes

Go to download

Code

Code files for the prototypes

Credits

Leave your feedback...