Digital twin and centralized system for heavy vehicle automation
By LeddarTech | Published on March 11, 2022
Problem and objectives
Problem
- Automate the operation of heavy-duty and off-track vehicles.
Objectives
- Enable automation while maintaining safety and reliability;
- Create a model capable of making decisions based on different types of vehicles and weather conditions;
- Develop a solution that does not require full retraining for each change that is made.
Proposed solution
Solution
LeddarTech’s mission is to use AI techniques to provide customers with environmental sensing solutions for autonomous vehicles and advanced driver assistance systems.
LeddarTech’s mission is to develop decentralized, safe and reliable technologies to automate the operation of heavy vehicles.
Automation of heavy vehicles, such as those used in mining and farming, is a significant market development. The firm Trimble, which previously had a navigation system that required operator intervention, used LeddarTech’s services.
Principle
To ensure safe and reliable vehicle optimization, LeddarTech chose to use data from a multitude of sources. The systems that are developed use radar images, cameras, LiDAR and navigation systems. The algorithm reconstitutes and rebuilds the environment by creating a digital twin in order to identify the objects in it and make decisions accordingly.
The external software developed by LeddarTech is not contained in the actual tool (such as in the camera). This makes it possible to replicate automated operation for several heavy vehicles through a centralized system. Since the model is not found in the sensor, Trimble is able to obtain equipment at a lower cost, which avoids having to support AI processes directly in the detection tool. The sensors are then connected through the system developed by LeddarTech. In fact, this technology can work in a cloud-based system.
User experience
LeddarTech’s technology allows the model to be implemented in different types of vehicles, without having to retrain the algorithm with each change. This is practical for users with many vehicles with different functions (such as a farmer with tractors and combines). The user thus benefits from significant time savings. When necessary, the models are flexible and allow the sensors to be moved without having to go through a long retraining process.
Role of AI
Before starting the project with LeddarTech, Trimble used camera-based navigation systems that required an operator. Using AI will enable them to offer an automation solution, enhance their services, and save on equipment costs.
In developing the algorithm, LeddarTech used a variety of techniques, including:
- Deep learning;
- Deep neural networks;
- Convolutional neural networks;
- Graph convolutional networks;
- Recurrent neural networks;
- Encoders and decoders;
- Reinforcement learning;
- Computer vision;
- Automatic natural language processing;
- Classification;
- Simulation;
- Regression;
- Clustering.
Impacts
Main outcomes
Impact on Trimble:
- Reduced sensor and camera costs;
- Automated operation of heavy vehicles that does not require operator intervention and can work in different weather conditions;
- Sensors can be moved without having to completely retrain the model;
- Solution that can be replicated for different types of vehicles;
- Increased productivity of customers who use their technology.
Impact on LeddarTech:
- Development of expertise in the field of heavy-duty and off-track vehicles;
- Showcasing of the company’s ability to solve complex industrial problems.
Generation and dissemination of new knowledge
- Sharing knowledge with Trimble throughout the project;
- System maintained by LeddarTech post-implementation;
- Training taken by LeddarTech’s team for the project: Nvidia AI development kit: deep learning for deploying AI and computer vision with NVIDIA Jetson AGX Xavier.
Economic development
- High recruitment at the local and international level;
- Solution implemented in four countries.
Challenges takled
- Development of telework technology during the COVID-19 pandemic;
- Coordination between the various Trimble sites worldwide;
- Financial risk associated with investments to develop the program.
Conditions for success
Team mobilised
LeddarTech’s AI project team consists of:
- Data analysts;
- Solution architects;
- Data scientists;
- Data engineers;
- Software developers;
- UI/UX designers;
- Integrators;
- Machine learning scientists.
Collaborations
The following collaborations contributed to the success of the project:
- LeddarTech and Trimble project managers;
- Partnership with Amazon Web Services;
- University partners such as Polytechnique Montréal, University of Ottawa, University of Toronto and Université Laval.
As part of the project, LeddarTech received external support in the following areas:
- Compliance with heavy-vehicle regulations;
- Data annotation;
- Data management;
- Simulation (e.g. Xspace).
Project stages
Proof of concept (set up at Trimble’s operational site);
Quantitative and qualitative analysis of collected data;
Development of a proof of concept in real time;
Deployment of the model in thousands of vehicles in 2023.