An Analysis of Machine Learning Algorithm in Autonomous Vehicle Navigation System
Keywords:
Autonomous vehicle, Machine Learning, Dynamic Vehicle Navigation, Traffic Signal Control, Mixed Autonomy Traffic Control, Reinforcement Learning, Multi-Agent System, Intelligent Transportation System, Proximal Policy Optimization, Deep Q-network, ConvolutAbstract
Machine learning (ML) algorithms play a pivotal role in the key functional areas of autonomous vehicle (AV) navigation, including perception, localization, mapping, trajectory prediction, planning, and control. Despite the advancements in sensor technologies such as high-definition cameras, LiDAR, and radar—commonly used for mapping, obstacle detection, and localization—autonomous vehicles still face significant challenges in reliably navigating unfamiliar environments with unpredictable dynamics. These challenges stem from real-world factors such as weather variability, traffic congestion, pedestrian activity, and erratic behaviour of other drivers. Machine learning offers a promising solution to address these complexities. As one of the most rapidly evolving technologies, ML enables autonomous navigation systems to better interpret sensory data, adapt to dynamic surroundings, and make informed driving decisions. In addition to enhancing traffic safety and minimizing human error-related accidents, connected and autonomous vehicles (CAVs) can also fulfil a wide range of smart functions—from last-mile delivery services to urban surveillance in smart cities. To achieve these benefits, autonomous vehicles must be capable of independently reaching their destinations while cooperating with road infrastructure. Recent advancements in Cooperative Vehicle-Infrastructure Systems (CVIS) facilitate seamless communication between AVs and elements like traffic lights (TLs), promoting safer and more efficient transportation systems. CVIS allows for the integration of real-time information sharing between infrastructure and vehicles, thus supporting more accurate navigation and situational awareness. This study focuses on evaluating the practicality of two well-known reinforcement learning techniques—Proximal Policy Optimization (PPO) and Deep Q-Network (DQN)—within autonomous navigation systems. The assessment began with training the models in a low-fidelity driving simulator, followed by testing in a high-fidelity traffic simulation environment to replicate more realistic driving conditions. Multiple driving scenarios were considered to evaluate the robustness, adaptability, and performance of each algorithm. Results indicate that both PPO and DQN outperform traditional models, with PPO showing superior performance in maintaining consistent speed, navigating efficiently, and minimizing idle or ineffective movements. In autonomous driving, vehicles must constantly evaluate the state of surrounding objects, whether static or in motion, and adapt their behaviour accordingly. To support this, machine learning techniques such as convolutional neural networks (CNNs) optimized with Adaptive Moment Estimation (Adam), and neuro-fuzzy systems fine-tuned through Particle Swarm Optimization (PSO), are employed. These methods enable real-time decision-making and smooth control, empowering AVs to respond swiftly and accurately based on learned patterns from extensive training data
Downloads
References
Fernandes L C, Souza J R, PESSIN G, Shinzato P Y, Sales D, Mendes C, Prado M, KLASER R, MAGALHAES A C, HATA A and PIGATTO D 2014 CARINA intelligent robotic car: architectural design and applications Journal of Systems Architecture 60 372-92
KUDERER M, Gulati S and BURGARD W 2015 Learning driving styles for autonomous vehicles from demonstration In2015 IEEE International Conference on Robotics and Automation (ICRA) IEEE p 2641-2646
Cathy Wu, ABOUDY KREIDIEH, KANAAD PARVATE, Eugene VINITSKY, and Alexandre M BAYEN. 2017. Flow: Architecture and benchmarking for reinforcement learning in traffic control. ARXIV preprint arXiv:1710.05465 10 (2017).
Liu S, Tang J, Zhang Z and GAUDIOT J L 2017 Computer architectures for autonomous driving Computer 50 18-25.
Koopman P and Wagner M 2017 Autonomous vehicle safety: An interdisciplinary challenge. IEEE Intelligent Transportation Systems Magazine 9 90-6
OFIR Nachum, SHIXIANG Shane Gu, HONGLAK Lee, and Sergey Levine. 2018. Data efficient hierarchical reinforcement learning. Advances in neural information processing systems 31 (2018).
Richard S Sutton and Andrew G BARTO. 2018. Reinforcement learning: An intro duction. MIT press.
FANGYU Zou, Li Shen, ZEQUN JIE, WEIZHONG Zhang, and Wei Liu. 2019. A sufficient condition for convergences of ADAM and RMSPROP. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 11127–11135.
Hua Wei, Nan Xu, HUICHU Zhang, GUANJIE Zheng, XINSHI Zang, CHACHA Chen, WEINAN Zhang, YANMIN Zhu, Kai Xu, and ZHENHUI Li. 2019. COLIGHT: Learning network-level cooperation for traffic signal control. In Proceedings of the 28thACM International Conference on Information and Knowledge Management. 1913–1922.
JINMING Ma and Feng Wu. 2020. Feudal multi-agent deep reinforcement learning for traffic signal control. In Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems (AAMAS). 816–824.
SONGSANG Koh, Bo Zhou, Hui Fang, Po Yang, ZALLI Yang, QIANG Yang, Lin Guan, and ZHIGANG Ji. 2020. Real-time deep reinforcement learning based vehicle navigation. Applied soft computing 96 (Nov 2020), 106694.
Shubham PATERIA, BUDHIAMA SUBAGDJA, Ah-HWEE Tan, and Chai Quek. 2021. Hierarchical reinforcement learning: A comprehensive survey. ACM Computing Surveys (CSUR) 54, 5 (2021), 1–35.
James Ault and GUNI Sharon. 2021. Reinforcement learning benchmarks for traffic signal control. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1).
YUANZHE GENG, ERWU Liu, Rui Wang, YIMING Liu, WEIXIONG Rao, SHAOJUN Feng, Zhao Dong, ZHIREN Fu, and YANFEN Chen.2021. Deep reinforcement learning based dynamic route planning for minimizing travel time. In 2021 IEEE International Conference on Communications Workshops (ICC Workshops). IEEE, 1–6.
Hua Wei, GUANJIE Zheng, Vikash GAYAH, and ZHENHUI Li. 2021. Recent advances in reinforcement learning for traffic signal control: A survey of models and evaluation. ACM SIGKDD Explorations Newsletter 22, 2 (2021), 12–18.
Matthias HUTSEBAUT-BUYSSE, Kevin Mets, and Steven LATRE. 2022. Hierarchical reinforcement learning: A survey and open research challenges. Machine Learning and Knowledge Extraction 4, 1 (2022), 172–221.
YANAN Wang, Tong Xu, Xin NIU, Chang Tan, ENHONG Chen, and Hui XIONG. 2022. STMARL: ASPATIO-Temporal Multi-Agent Reinforcement Learning Approach for Cooperative Traffic Light Control. ,2228-2242 pages. https://doi.org/10.1109/ TMC.2020.3033782
ZhengWangandShenWang.Jan01,2022. XROUTING: Explainable Vehicle REROUTING for Urban Road Congestion Avoidance using Deep Reinforcement Learning. The Institute of Electrical and Electronics Engineers, Inc. (IEEE), Piscataway.
WEIJIA Zhang, Hao Liu, Jindong Han, Yong Ge, and Hui XIONG. 2022. Multi -agent graph convolutional reinforcement learning for dynamic electric vehicle charging pricing. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. 2471–2481.
Hao Mei, XIAOLIANG Lei, LONGCHAO Da, Bin Shi, and Hua Wei. 2022. LIBSIGNAL: An Open Library for Traffic Signal Control. ARXIV preprint arXiv:2211.10649 (2022).
Q. Meng, H. Guo, Y. Liu, H. Chen, and D. Cao, “Trajectory prediction for automated vehicles on roads with lanes partially covered by ice or snow,” IEEE Transactions on Vehicular Technology, 2023.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
You are free to:
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
Terms:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
