research paper on machine tools

We apologize for the inconvenience...

To ensure we keep this website safe, please can you confirm you are a human by ticking the box below.

If you are unable to complete the above request please contact us using the below link, providing a screenshot of your experience.

https://ioppublishing.org/contacts/

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

An agent-based method for feature recognition and path optimization of computer numerical control machining trajectories.

1. Introduction

• Introduction of an agent-based CNC system architecture, leveraging artificial intelligence technology to enhance the performance and efficiency of CNC systems.
• Proposal of a deep neural network integrating multiple neural network models and linear attention mechanisms for improved feature recognition efficiency.
• The honey badger algorithm initializes the coati population, thereby enhancing the initial population quality and optimization efficiency.
• Information such as the population size, iteration count, and fitness function is embedded into the improved path update rules, which allows the new rules to optimize the search process based on the current population status, thereby improving convergence speed.
• A dynamic multi-population strategy is used to comprehensively explore the search space and maintain population diversity.
• A gradient descent fitness-guided strategy dynamically adjusts the learning rate, thereby controlling the magnitude of each path point update for quicker convergence to the optimal solution.

2. Related Works

2.1. feature recognition in cnc machining based on deep learning, 2.2. cnc machining path optimization, 3. agent-based cnc system architecture, 3.1. intelligent requirements, 3.2. system model and structure.

• CAD/CAM: The process of design and manufacturing using NX software (Version 12.0, Siemens Digital Industries Software, Plano, TX, USA). This is the input part of the intelligent system, providing design data and manufacturing instructions.
• Learning: Extracts useful information from data to optimize models and manufacturing processes.
• Digital Twin: Provides a virtual environment for testing and optimization, enhancing efficiency and precision.
• Sense: Monitors various parameters during the manufacturing process, providing real-time feedback to the digital twin and optimization modules.
• Optimization: The process of optimizing system performance based on learning and sensing data, reducing resource consumption, and refining manufacturing processes.
• NC System and Machine Tool: Receives instructions from the optimization module and performs machining and manufacturing according to CNC system directives.

3.3. Assembly Line Work Mode

4. mathematical model, 4.1. machining path feature design.

• v i · v i − 1 is the dot product of vectors v i and v i − 1 .
• | v i | and | v i − 1 | are the Euclidean lengths of vectors v i and v i − 1 respectively.

4.2. Path Optimization Design

4.2.1. honey badger algorithm for population initialization.

• A two-dimensional point set path = { P 1 , P 2 , … , P n } represents a set of points on the path, where P i = ( x i , y i ) .
• Set the start and end points: s t a r t _ p o i n t = P 1 and e n d _ p o i n t = P n .
• Randomly insert points: Randomly select the remaining points and randomly insert them into a certain position on the path until all points are inserted.
• p k = { P 1 , P k , 2 , P k , 3 , … , P k , n − 1 , P n } , where { P k , 2 , P k , 3 , … , P k , n − 1 } are random permutations of { P 2 , P 3 , … , P n − 1 } , and the population size is p o p _ s i z e = N .
• r 1 is a random number, uniformly distributed in (0,1).
• S is the intensity.
• p k [ i ] is the i -th point of individual k .
• g b e s t [ i ] is the i -th point of the global best position.
• d i represents the distance between the global best position and the current point.
• α 1 is a constant.
• β is a constant indicating the honey badger’s ability to obtain food.
• p k [ i ] ′ represents the updated position.
• r 2 , r 3 , r 4 , r 5 , r 6 are random numbers between 0 and 1.

4.2.2. Enhanced Path Update Rule

• α max is the initial maximum value of α 2 .
• α min is the minimum value of α 2 .
• g is the current iteration number, incrementing from 1 to G .
• G is the total number of iterations in the algorithm.

4.2.3. Dynamic Multi-Population Strategy

• M min : Minimum sub-population size
• M max : Maximum sub-population size

4.2.4. Gradient Descent-Based Adaptive Guidance Strategy

• η 0 : Initial learning rate
• λ : Decay rate
• g : Current iteration number
• Step 1: Algorithm Parameter Initialization: The algorithm begins by initializing a series of parameters, including a 1 , β , M 0 , M m i n , M m a x , and the number of iterations G . This initialization establishes the foundation for subsequent path optimization and sub-population operations.
• Step 2: HBA Path Optimization: During the path optimization phase, the algorithm uses the HBA path list as the initial input. For each iteration, the algorithm removes the first and last elements from the path list and deletes a specific point p k [ i ] using a randomly generated t value. The remaining points in the path are updated according to a specific equation (e.g., Equation ( 5 )). After updating the path, the algorithm calculates the fitness function using a series of equations (e.g., Equations (6)–(11)) and determines the current optimal path.
• Step 3: NACOA Path List Initialization and Update: In the second stage, the algorithm initializes the NACOA path list based on the HBA path list, with M g = M 0 and N g = N M 0 . As the iterations progress, the number of sub-populations M g is dynamically adjusted according to Equation (16). During each update, the algorithm clears the current sub-population and assigns path points p t i to the sub-population s u b p o p a . The algorithm then optimizes the path using a gradient descent strategy and specified equations (Equations (12)–(14)), incorporating the optimized path points into the population list.
• Step 4: Sub-population Exchange and Global Optimization: At specific iteration counts (e.g., g % T = = 0 ), the algorithm randomly exchanges individuals between two sub-populations to increase diversity. At the end of each iteration, the algorithm identifies the current global best individual g b e s t based on the fitness function.
• Step 5: Output Optimal Path: After all iterations are completed, the algorithm outputs the globally optimal path obtained through computation. This final output represents the optimal solution under the given constraints.

5. Experiments

5.1. network experiments and results, 5.1.1. network architecture, 5.1.2. experimental results.

• Hyperparameter settings
• Learning Rate: This parameter was set to either 0.0001 or 0.0005, depending on the dataset. The learning rates for the gear and pentagram datasets were 0.0001, while the rates for the maple leaf datasets were 0.0005.
• Decay Rate: The decay rate varied between datasets, ranging from 0.8 to 0.9. The decay rates for the gear datasets were higher at 0.9, indicating a slower reduction in the learning rate over time compared to the 0.8 rate for the pentagram and maple leaf datasets.
• Decay Steps: The decay steps were either 100 or 500. The gear and maple leaf datasets used a larger number of decay steps (500), implying a lower frequency of learning rate decay application, while the pentagram datasets employed 100 decay steps.
• Batch Size: The batch sizes were set to either 32 or 64. The gear and maple leaf datasets used a batch size of 64, which generally provides a more stable gradient estimate, whereas the pentagram dataset used a smaller batch size of 32.
• Iterations: The number of iterations required varied significantly between datasets. The gear dataset required the most iterations, totaling 497, while the pentagram dataset required 203. The maple leaf datasets required 234 and 340 iterations, respectively.
• Time: The training times for each dataset also varied. The gear dataset required the longest training time, at 212 units, while the pentagram dataset required 169 units, and the maple leaf dataset required the shortest time of 40 units.
• TP (True Positives): The number of samples that are truly positive and predicted as positive.
• TN (True Negatives): The number of samples that are truly negative and predicted as negative.
• FP (False Positives): The number of samples that are truly negative but predicted as positive.
• FN (False Negatives): The number of samples that are truly positive but predicted as negative.
• N is the total number of samples.
• C is the total number of classes.
• y i , j is the true label of the i -th sample, where y i , j = 1 if the true label of the i -th sample is class j , and 0 otherwise.
• p i , j is the predicted probability that the i -th sample belongs to class j .
• log ( p i , j ) is the natural logarithm of the predicted probability of class j .
• FPR: False Positive Rate
• TPR: True Positive Rate
• Gear Dataset
• The MCRL model achieved the best performance on the gear dataset, with an accuracy of 94.75%, a loss of 14.26, a precision of 96.23%, a recall of 93.52%, an F1 score of 94.85%, and an AUC of 97.71%. These results underscore MCRL’s superior predictive performance on this dataset and its ability to effectively balance various metrics.
• The ConvMixer and ConvNeXt models also performed relatively well, with accuracies of 92.22% and 93.92%, respectively. While their other metrics were comparable, they still fell short compared to MCRL.
• MaxViT slightly lagged behind ConvNeXt in overall performance but maintained high precision and F1 score.
• The MLP, CNN, RNN, and LSTM models showed mediocre performance, particularly the MLP and RNN models, which fell below 90% across all metrics, highlighting their limitations on this dataset.
• Pentagram Dataset
• On the pentagram dataset, MCRL again excelled with an accuracy of 94.98%, a loss of 13.55, a precision of 96.47%, a recall of 93.35%, an F1 score of 95.63%, and an AUC of 97.58%.
• The ConvMixer and ConvNeXt models performed well on the pentagram dataset, although they slightly lagged behind MCRL in recall and F1 score, with values around 93% and 92%, respectively.
• MaxViT’s performance was fairly balanced, but it fell short in accuracy and AUC.
• Other models, such as MLP, CNN, and RNN, showed relatively poor performance, with RNN notably underperforming, achieving an accuracy of only 86.34% and significant deficiencies in recall and F1 score.
• Maple Leaf Dataset
• On the maple leaf dataset, the MCRL model achieved the highest accuracy of 96.32%, the lowest loss of 9.81, and precision and F1 scores of 96.52% and 96.35%, respectively, with an AUC of 98.66%. These results highlight MCRL’s exceptional performance in path classification tasks.
• The ConvMixer and ConvNeXt models followed closely, with accuracies of 95.97% and 95.27%, and F1 scores exceeding 95%, although they still fell slightly short of MCRL.
• MaxViT also showed stable performance, but its AUC was somewhat lower at 98.00.
• Traditional models, such as MLP, CNN, and RNN, generally fell short of the advanced models mentioned above, with RNN showing particularly poor performance on this dataset, achieving an accuracy of only 85.63%.
• MCRL: The inference times across various datasets ranged from 3.1 to 3.9 s.
• MCRL with QAT: This variant of MCRL utilized quantization-aware training (QAT) technology, which simulates the effects of reduced precision during training, lowering network weights and activations from 32-bit floating-point numbers to a minimum of 8-bit integers. This reduction in precision decreased memory usage and accelerated data transfer, thereby reducing inference time compared to the standard MCRL model. For instance, on the gear dataset, the inference time for MCRL with QAT was 2.8 s, while the standard MCRL took 3.2 s. Similar improvements were observed across all datasets.
• ConvMixer (2022), ConvNeXt (2022), MaxViT (2022): These models, introduced in 2022, achieved inference times comparable to MCRL but showed variability across datasets, with ConvMixer and RNN exhibiting slightly higher inference times.
• MLP, CNN, RNN, LSTM: These traditional models generally had higher inference times, with MLP showing the highest inference times across all datasets, particularly on the pentagram and maple leaf datasets.

5.2. Optimization Experiments and Results

5.2.1. complexity analysis, 5.2.2. comparison with other algorithms, 5.2.3. comparison of algorithm optimization details, 5.2.4. the performance of algorithms in path optimization, 5.3. integration of the preprocessing module in the cnc system, 6. conclusions, author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest.

• Du, S. Analysis of the application of intelligent CNC technology in machinery manufacturing. Proc. J. Phys. Conf. Ser. Iop Publ. 2023 , 649 , 012010. [ Google Scholar ] [ CrossRef ]
• Soori, M.; Arezoo, B.; Dastres, R. Machine learning and artificial intelligence in CNC machine tools, a review. Sustain. Manuf. Serv. Econ. 2023 , 2 , 100009. [ Google Scholar ] [ CrossRef ]
• Molina, L.M.; Teti, R.; Alvir, E.M.R. Quality, efficiency and sustainability improvement in machining processes using artificial intelligence. Procedia CIRP 2023 , 118 , 501–506. [ Google Scholar ] [ CrossRef ]
• Ning, F.; Shi, Y.; Cai, M.; Xu, W. Part machining feature recognition based on a deep learning method. J. Intell. Manuf. 2023 , 34 , 809–821. [ Google Scholar ] [ CrossRef ]
• Yao, X.; Wang, D.; Yu, T.; Luan, C.; Fu, J. A machining feature recognition approach based on hierarchical neural network for multi-feature point cloud models. J. Intell. Manuf. 2023 , 34 , 2599–2610. [ Google Scholar ] [ CrossRef ]
• Guangwen, Y.; ZHANG, D.; Jinting, X.; Yuwen, S. Corner smoothing for CNC machining of linear tool path: A review. J. Adv. Manuf. Sci. Technol. 2023 , 3 , 2023001. [ Google Scholar ]
• Zhang, H.; Wu, Y.; Shen, Z.; Zhang, P.; Li, H.; Lou, F. Five-axis path smoothing based on sliding convolution windows for CNC machining. CIRP J. Manuf. Sci. Technol. 2024 , 49 , 80–94. [ Google Scholar ] [ CrossRef ]
• Shen, Z.; Wu, Y.; Guo, P.; Zhang, H.; Zhang, P.; Li, H.; Lou, F. Convolution synchronous smoothing for tool position and posture of continuous line-segment path in 5-axis machining. J. Manuf. Process. 2024 , 112 , 136–149. [ Google Scholar ] [ CrossRef ]
• Zhang, H.; Wu, Y.; Shen, Z.; Zhang, P.; Li, H.; Lou, F.; Guo, P. Mixed tiny path smoothing method based on sliding convolution windows for CNC machining. J. Manuf. Process. 2023 , 102 , 685–699. [ Google Scholar ] [ CrossRef ]
• Xu, Y.; Chen, Z.; Deng, C.; Wang, S.; Wang, J. LCDL: Towards Dynamic Localization for Autonomous Landing of Unmanned Aerial Vehicle Based on LiDAR-Camera Fusion. IEEE Sens. J. 2024 , 24 , 26407–26415. [ Google Scholar ] [ CrossRef ]
• Deng, C.; Wang, S.; Wang, J.; Xu, Y.; Chen, Z. LiDAR Depth Cluster Active Detection and Localization for a UAV with Partial Information Loss in GNSS. Unmanned Syst. 2024 , 1–13. [ Google Scholar ] [ CrossRef ]
• Zhang, S.; Guan, Z.; Jiang, H.; Wang, X.; Tan, P. BrepMFR: Enhancing machining feature recognition in B-rep models through deep learning and domain adaptation. Comput. Aided Geom. Des. 2024 , 111 , 102318. [ Google Scholar ] [ CrossRef ]
• Wang, P.; Yang, W.A.; You, Y. A hybrid learning framework for manufacturing feature recognition using graph neural networks. J. Manuf. Process. 2023 , 85 , 387–404. [ Google Scholar ] [ CrossRef ]
• Wu, H.; Lei, R.; Huang, P.; Peng, Y. A semi-supervised learning framework for machining feature recognition on small labeled sample. Appl. Sci. 2023 , 13 , 3181. [ Google Scholar ] [ CrossRef ]
• Kaigom, E.G.; Roßmann, J. Value-driven robotic digital twins in cyber–physical applications. IEEE Trans. Ind. Inform. 2020 , 17 , 3609–3619. [ Google Scholar ] [ CrossRef ]
• Fang, J.; Li, B.; Zhang, H.; Ye, P. Real-time smooth trajectory generation for 3-axis blending tool-paths based on FIR filtering. Int. J. Adv. Manuf. Technol. 2023 , 126 , 3401–3416. [ Google Scholar ] [ CrossRef ]
• Kučera, D.; Linkeová, I.; Stejskal, M. The influence of CAD model continuity on accuracy and productivity of CNC machining. Int. J. Adv. Manuf. Technol. 2023 , 124 , 1115–1128. [ Google Scholar ] [ CrossRef ]
• Hua, L.; Huang, N.; Yi, B.; Zhao, Y.; Zhu, L. Global toolpath smoothing for CNC machining based on B-spline approximation with tool tip position adjustment. Int. J. Adv. Manuf. Technol. 2023 , 125 , 3651–3665. [ Google Scholar ] [ CrossRef ]
• Zhang, L.; Lu, J.; Ou, C.; Ma, J.; Liao, X. Smoothing and compressing algorithm of toolpath with complex contour in NC machining. Int. J. Adv. Manuf. Technol. 2023 , 125 , 4841–4854. [ Google Scholar ] [ CrossRef ]
• Feng, K.; Ji, J.; Ni, Q. A novel gear fatigue monitoring indicator and its application to remaining useful life prediction for spur gear in intelligent manufacturing systems. Int. J. Fatigue 2023 , 168 , 107459. [ Google Scholar ] [ CrossRef ]
• Yao, Y.; Liu, M.; Du, J.; Zhou, L. Design of a machine tool control system for function reconfiguration and reuse in network environment. Robot.-Comput.-Integr. Manuf. 2019 , 56 , 117–126. [ Google Scholar ] [ CrossRef ]
• Wu, S.F.; Lee, C.H.; Tseng, B.R. Intelligent Servo Tuning of High-Speed Machine Tools Using Circular Test. IEEE Sens. J. 2023 , 23 , 12084–12092. [ Google Scholar ] [ CrossRef ]
• Böttjer, T.; Tola, D.; Kakavandi, F.; Wewer, C.R.; Ramanujan, D.; Gomes, C.; Larsen, P.G.; Iosifidis, A. A review of unit level digital twin applications in the manufacturing industry. CIRP J. Manuf. Sci. Technol. 2023 , 45 , 162–189. [ Google Scholar ] [ CrossRef ]
• Liu, Z.; Lang, Z.Q.; Gui, Y.; Zhu, Y.P.; Laalej, H. Digital twin-based anomaly detection for real-time tool condition monitoring in machining. J. Manuf. Syst. 2024 , 75 , 163–173. [ Google Scholar ] [ CrossRef ]
• Iliyas Ahmad, M.; Yusof, Y.; Mustapa, M.S.; Daud, M.E.; Latif, K.; Kadir, A.Z.A.; Saif, Y.; Adam, A.; Hatem, N. A novel integration between service-oriented IoT-based monitoring with open architecture of CNC system monitoring. Int. J. Adv. Manuf. Technol. 2024 , 131 , 5625–5636. [ Google Scholar ] [ CrossRef ]
• Liu, C.; Zheng, P.; Xu, X. Digitalisation and servitisation of machine tools in the era of Industry 4.0: A review. Int. J. Prod. Res. 2023 , 61 , 4069–4101. [ Google Scholar ] [ CrossRef ]
• Dehghani, M.; Montazeri, Z.; Trojovská, E.; Trojovskỳ, P. Coati Optimization Algorithm: A new bio-inspired metaheuristic algorithm for solving optimization problems. Knowl.-Based Syst. 2023 , 259 , 110011. [ Google Scholar ] [ CrossRef ]
• Hashim, F.A.; Houssein, E.H.; Hussain, K.; Mabrouk, M.S.; Al-Atabany, W. Honey Badger Algorithm: New metaheuristic algorithm for solving optimization problems. Math. Comput. Simul. 2022 , 192 , 84–110. [ Google Scholar ] [ CrossRef ]
• Trockman, A.; Kolter, J.Z. Patches are all you need? arXiv 2022 , arXiv:2201.09792. [ Google Scholar ]
• Liu, Z.; Mao, H.; Wu, C.Y.; Feichtenhofer, C.; Darrell, T.; Xie, S. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–21 June 2022; pp. 11976–11986. [ Google Scholar ]
• Tu, Z.; Talebi, H.; Zhang, H.; Yang, F.; Milanfar, P.; Bovik, A.; Li, Y. Maxvit: Multi-axis vision transformer. In Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel, 23–27 October 2022; Springer: Berlin/Heidelberg, Germany, 2022; pp. 459–479. [ Google Scholar ]
• Abdollahzadeh, B.; Gharehchopogh, F.S.; Mirjalili, S. African vultures optimization algorithm: A new nature-inspired metaheuristic algorithm for global optimization problems. Comput. Ind. Eng. 2021 , 158 , 107408. [ Google Scholar ] [ CrossRef ]
• Seyyedabbasi, A.; Kiani, F. Sand Cat swarm optimization: A nature-inspired algorithm to solve global optimization problems. Eng. Comput. 2023 , 39 , 2627–2651. [ Google Scholar ] [ CrossRef ]
• Chen, Z.; Francis, A.; Li, S.; Liao, B.; Xiao, D.; Ha, T.T.; Li, J.; Ding, L.; Cao, X. Egret swarm optimization algorithm: An evolutionary computation approach for model free optimization. Biomimetics 2022 , 7 , 144. [ Google Scholar ] [ CrossRef ] [ PubMed ]

Click here to enlarge figure

Share and Cite

Li, P.; Chen, M.; Ji, C.; Zhou, Z.; Lin, X.; Yu, D. An Agent-Based Method for Feature Recognition and Path Optimization of Computer Numerical Control Machining Trajectories. Sensors 2024 , 24 , 5720. https://doi.org/10.3390/s24175720

Li P, Chen M, Ji C, Zhou Z, Lin X, Yu D. An Agent-Based Method for Feature Recognition and Path Optimization of Computer Numerical Control Machining Trajectories. Sensors . 2024; 24(17):5720. https://doi.org/10.3390/s24175720

Li, Purui, Meng Chen, Chuanhao Ji, Zheng Zhou, Xusheng Lin, and Dong Yu. 2024. "An Agent-Based Method for Feature Recognition and Path Optimization of Computer Numerical Control Machining Trajectories" Sensors 24, no. 17: 5720. https://doi.org/10.3390/s24175720

Article Metrics

Further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Introduction to machine and machine tools

• Al-Falah School of Engineering & Technology

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

•  We're Hiring!
•  Help Center

Machine Tools

• Most Cited Papers
• Most Downloaded Papers
• Newest Papers
• Last »
• Metal Cutting Follow Following
• Numerical Control Follow Following
• Technical History Follow Following
• Mechanical Engineering History Follow Following
• Manufacturing Engineering Follow Following
• Manufacture Follow Following
• Phase Diagrams Follow Following
• Electrical Technology Follow Following
• Micromachining Follow Following
• CNC Machine tools Follow Following

Enter the email address you signed up with and we'll email you a reset link.

• Academia.edu Journals
•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

• Search Research
• Eindhoven Artificial Intelligence Systems Institute
• Institute for Complex Molecular Systems
• Eindhoven Hendrik Casimir Institute
• Eindhoven Institute for Renewable Energy Systems
• Artificial Intelligence
• Smart Mobility
• Engineering Health
• Integrated Photonics
• Quantum Technology
• High Tech Systems Center
• Data Science
• Humans and Technology
• Future Chips
• Research Groups
• Other labs and facilities
• Researchers
• Applied Physics and Science Education
• Biomedical Engineering
• Built Environment
• Chemical Engineering and Chemistry
• Eindhoven School of Education
• Electrical Engineering
• Industrial Design
• Industrial Engineering and Innovation Sciences
• Mathematics and Computer Science
• Mechanical Engineering
• National Grants
• International Grants
• TU/e Distinctions
• Sectorplans
• Research assessments
• Winners TU/e Science Awards
• Research Support Network

Information Systems IE&IS

The Information Systems (IS) group studies novel tools and techniques that help organizations use their information systems to support better operational decision making.

Create value through intelligent processing of business information

Information Systems are at the core of modern-day organizations. Both within and between organizations. The Information Systems group studies tools and techniques that help to use them in the best possible way, to get the most value out of them.

In order to do that, the IS group helps organizations to: (i) understand the business needs and value propositions and accordingly design the required business and information system architecture; (ii) design, implement, and improve the operational processes and supporting (information) systems that address the business need, and (iii) use advanced data analytics methods and techniques to support decision making for improving the operation of the system and continuously reevaluating its effectiveness.

We do so in various sectors from transportation and logistics, mobility services, high-tech manufacturing, service industry, and e-commerce to healthcare.

Against this background, IS research concentrates on the following topics:

• Business model design and service systems engineering for digital services.
• Managing digital transformation.
• Data-driven business process engineering and execution.
• Innovative process modeling techniques and execution engines.
• Human aspects of information systems engineering.
• Intelligent decision support through Artificial Intelligence and Computational Intelligence.
• Data-driven decision making.
• Machine learning to optimize resource allocation.
• All IS news

Research Areas

We work on Information Systems topics in three related research areas.

Process Engineering

Process Engineering (PE) develops integrated tools and techniques for data-driven decision support in the design and execution of…

AI for decision-making

AI for Decision-Making (AI4DM) develops methods, techniques and tools for AI-driven decision making in operational business process.

Business Engineering

Business Engineering (BE) investigates and develops new concepts, methods, and techniques - including novel data-driven approaches - for the…

Application domains

We focus on the application of Information Systems in the following domains.

Information Systems are the backbone of modern health(care) ecosystems. They are critical for clinical research, clinical operations, and…

Smart Industry

The digital transformation of industry is leveraged by Information Systems providing integrated data and process management and AI-enabled…

Transportation and Logistics

Information Systems facilitate monitoring and planning of transportation and logistics resources. By doing so, they ultimately help to…

Information Systems focuses on the business architecture design of new mobility solutions that are safe, efficient, affordable and…

Service Industry

Service organizations, including banks, insurance companies, and governmental bodies, fully rely on information provisioning to do their…

Meet some of our researchers

Sybren de kinderen, isel grau garcia, yingqian zhang, laura genga, pieter van gorp, konstantinos tsilionis, remco dijkman, baris ozkan, karolin winter, oktay türetken, laurens bliek, alexia athanasopoulou.

• Meet all our researchers

ENFIELD & EAISI event: Human Centric AI

Together with EAISI, ENFIELD will present key findings on ongoing projects, available funding for researchers and collaboration…

EAISI lecture of Visiting Professor Chiara Ghidini

Process, Data, Conceptual Knowledge, and AI: What can they do together? Chiara Ghidini is a full professor at the Free University of…

Annual AI Conference ELA - Siemens 2024

The Euregio AI Triangle (RWTH Aachen, KU Leuven and TU Eindhoven) and Siemens are cordially inviting all AI enthusiasts and interested…

Recent Publications

• See all publications

Our most recent peer reviewed publications

Acceptance of Mobility-as-a-Service: Insights from empirical studies on influential factors

A revised cognitive mapping methodology for modeling and simulation, topic specificity, business models and process models, a reference architecture for reverse logistics in the high-tech industry.

Open source

We encourage innovation from our research. This is why we share the open-source codes from our research projects.

• Link to our open source codes

Work with us!

Please check out the TU/e vacancy pages for opportunities within our group. 

If you are a student, potential sponsor or industrial partner and want to work with us, please contact the IS secretariat or the Information Systems group chair,  dr.ir. Remco Dijkman

Visiting address

Postal address.

• Data Science
• Data Analysis
• Data Visualization
• Machine Learning
• Deep Learning
• Computer Vision
• Artificial Intelligence
• AI ML DS Interview Series
• AI ML DS Projects series
• Data Engineering
• Web Scrapping

10 Must Read Machine Learning Research Papers

Machine learning is a rapidly evolving field with research papers often serving as the foundation for discoveries and advancements. For anyone keen to delve into the theoretical and practical aspects of machine learning, the following ten research papers are essential reads. They cover foundational concepts, groundbreaking techniques, and key advancements in the field.

This article highlights 10 must-read machine learning research papers that have significantly contributed to the development and understanding of machine learning. Whether you’re a beginner or an experienced practitioner, these papers provide invaluable insights that will help you grasp the complexities of machine learning and its potential to transform industries.

Table of Content

1. “A Few Useful Things to Know About Machine Learning” by Pedro Domingos

2. “imagenet classification with deep convolutional neural networks” by alex krizhevsky, ilya sutskever, and geoffrey e. hinton, 3. “playing atari with deep reinforcement learning” by volodymyr mnih et al., 4. “sequence to sequence learning with neural networks” by ilya sutskever, oriol vinyals, and quoc v. le, 5. “attention is all you need” by ashish vaswani et al., 6. “generative adversarial nets” by ian goodfellow et al., 7. “bert: pre-training of deep bidirectional transformers for language understanding” by jacob devlin et al., 8. “deep residual learning for image recognition” by kaiming he et al., 9. “a survey on deep learning in medical image analysis” by geert litjens et al., 10. “alphago: mastering the game of go with deep neural networks and tree search” by silver et al..

Summary : Pedro Domingos provides a comprehensive overview of essential machine learning concepts and common pitfalls. This paper is a great starting point for understanding the broader landscape of machine learning.

Key Contributions:

• Distills core principles and practical advice.
• Discusses overfitting, feature engineering, and model selection.
• Offers insights into the trade-offs between different machine learning algorithms.

Access: Read the Paper

Summary : Often referred to as the “AlexNet” paper, this work introduced a deep convolutional neural network that significantly improved image classification benchmarks, marking a turning point in computer vision.

• Demonstrated the power of deep learning for image classification.
• Introduced techniques like dropout and ReLU activations.
• Showed the importance of large-scale datasets and GPU acceleration.

Summary : This paper from DeepMind presents the use of deep Q-networks (DQN) to play Atari games . It was a seminal work in applying deep learning to reinforcement learning.

• Introduced the concept of using deep learning for Q-learning.
• Showcased the ability of DQNs to learn complex behaviors from raw pixel data.
• Paved the way for further research in reinforcement learning.

Summary : This paper introduced the sequence-to-sequence (seq2seq) learning framework , which has become fundamental for tasks such as machine translation and text summarization.

• Proposed an encoder-decoder architecture for sequence tasks.
• Demonstrated effective training of neural networks for sequence modeling.
• Laid the groundwork for subsequent advancements in natural language processing.

Summary : This paper introduces the Transformer model, which relies solely on attention mechanisms, discarding recurrent layers used in previous models. It has become the backbone of many modern NLP systems.

• Proposed the Transformer architecture, which uses self-attention to capture dependencies.
• Demonstrated improvements in training efficiency and performance over RNN-based models.
• Led to the development of models like BERT, GPT, and others.

Summary : Ian Goodfellow and his colleagues introduced Generative Adversarial Networks (GANs) , a revolutionary framework for generating realistic data through adversarial training.

• Proposed a novel approach where two neural networks compete against each other.
• Enabled the generation of high-quality images, text, and other data types.
• Spurred a plethora of research on GAN variations and applications.

Summary : BERT (Bidirectional Encoder Representations from Transformers) introduced a new way of pre-training language models, significantly improving performance on various NLP benchmarks.

• Proposed bidirectional training of transformers to capture context from both directions.
• Achieved state-of-the-art results on several NLP tasks.
• Set the stage for subsequent models like RoBERTa, ALBERT, and DistilBERT.

Summary : This paper introduces Residual Networks (ResNets), which utilize residual learning to train very deep neural networks effectively.

• Addressed the issue of vanishing gradients in very deep networks.
• Demonstrated that extremely deep networks can be trained successfully.
• Improved performance on image classification tasks and influenced subsequent network architectures.

Summary : This survey provides a comprehensive review of deep learning techniques applied to medical image analysis, summarizing the state of the art in this specialized field.

• Reviewed various deep learning methods used in medical imaging.
• Discussed challenges and future directions in the field.
• Provided insights into applications such as disease detection and image segmentation.

Summary : This paper describes AlphaGo, the first AI to defeat a world champion in the game of Go, using a combination of deep neural networks and Monte Carlo tree search.

• Demonstrated the effectiveness of combining deep learning with traditional search techniques.
• Achieved a major milestone in AI by mastering a complex game.
• Influenced research in AI and its application to other complex decision-making problems.

These ten research papers cover a broad spectrum of machine learning advancements, from foundational concepts to cutting-edge techniques. They provide valuable insights into the development and application of machine learning technologies, making them essential reads for anyone looking to deepen their understanding of the field. By exploring these papers, you can gain a comprehensive view of how machine learning has evolved and where it might be heading in the future.

10 Must Read Machine Learning Research Papers – FAQ’s

What are large language models (llms) and why are they important.

Large Language Models (LLMs) are advanced AI systems designed to understand and generate human language. They are built using deep learning techniques, particularly transformer architectures. LLMs are important because they enable applications such as text generation, translation, and sentiment analysis, significantly advancing the field of natural language processing (NLP).

Why should I read “A Few Useful Things to Know About Machine Learning” by Pedro Domingos?

Pedro Domingos’ paper provides a broad overview of key machine learning concepts, common challenges, and practical advice. It’s an excellent resource for both beginners and experienced practitioners to understand the underlying principles of machine learning and avoid common pitfalls.

What impact did “ImageNet Classification with Deep Convolutional Neural Networks” have on the field?

The “AlexNet” paper revolutionized image classification by demonstrating the effectiveness of deep convolutional neural networks. It significantly improved benchmark results on ImageNet and introduced techniques like dropout and ReLU activations, which are now standard in deep learning.

Please Login to comment...

Similar reads.

• AI-ML-DS Blogs
• How to Delete Discord Servers: Step by Step Guide
• Google increases YouTube Premium price in India: Check our the latest plans
• California Lawmakers Pass Bill to Limit AI Replicas
• Best 10 IPTV Service Providers in Germany
• 15 Most Important Aptitude Topics For Placements [2024]

Improve your Coding Skills with Practice

What kind of Experience do you want to share?

Suggestions or feedback?

MIT News | Massachusetts Institute of Technology

• Machine learning
• Sustainability
• Black holes
• Classes and programs

Departments

• Aeronautics and Astronautics
• Brain and Cognitive Sciences
• Architecture
• Political Science
• Mechanical Engineering

Centers, Labs, & Programs

• Abdul Latif Jameel Poverty Action Lab (J-PAL)
• Picower Institute for Learning and Memory
• Lincoln Laboratory
• School of Architecture + Planning
• School of Engineering
• School of Humanities, Arts, and Social Sciences
• Sloan School of Management
• School of Science
• MIT Schwarzman College of Computing

Study: Transparency is often lacking in datasets used to train large language models

Press contact :, media download.

*Terms of Use:

Images for download on the MIT News office website are made available to non-commercial entities, press and the general public under a Creative Commons Attribution Non-Commercial No Derivatives license . You may not alter the images provided, other than to crop them to size. A credit line must be used when reproducing images; if one is not provided below, credit the images to "MIT."

Previous image Next image

In order to train more powerful large language models, researchers use vast dataset collections that blend diverse data from thousands of web sources.

But as these datasets are combined and recombined into multiple collections, important information about their origins and restrictions on how they can be used are often lost or confounded in the shuffle.

Not only does this raise legal and ethical concerns, it can also damage a model’s performance. For instance, if a dataset is miscategorized, someone training a machine-learning model for a certain task may end up unwittingly using data that are not designed for that task.

In addition, data from unknown sources could contain biases that cause a model to make unfair predictions when deployed.

To improve data transparency, a team of multidisciplinary researchers from MIT and elsewhere launched a systematic audit of more than 1,800 text datasets on popular hosting sites. They found that more than 70 percent of these datasets omitted some licensing information, while about 50 percent had information that contained errors.

Building off these insights, they developed a user-friendly tool called the  Data Provenance Explorer that automatically generates easy-to-read summaries of a dataset’s creators, sources, licenses, and allowable uses.

“These types of tools can help regulators and practitioners make informed decisions about AI deployment, and further the responsible development of AI,” says Alex “Sandy” Pentland, an MIT professor, leader of the Human Dynamics Group in the MIT Media Lab, and co-author of a new open-access paper about the project .

The Data Provenance Explorer could help AI practitioners build more effective models by enabling them to select training datasets that fit their model’s intended purpose. In the long run, this could improve the accuracy of AI models in real-world situations, such as those used to evaluate loan applications or respond to customer queries.

“One of the best ways to understand the capabilities and limitations of an AI model is understanding what data it was trained on. When you have misattribution and confusion about where data came from, you have a serious transparency issue,” says Robert Mahari, a graduate student in the MIT Human Dynamics Group, a JD candidate at Harvard Law School, and co-lead author on the paper.

Mahari and Pentland are joined on the paper by co-lead author Shayne Longpre, a graduate student in the Media Lab; Sara Hooker, who leads the research lab Cohere for AI; as well as others at MIT, the University of California at Irvine, the University of Lille in France, the University of Colorado at Boulder, Olin College, Carnegie Mellon University, Contextual AI, ML Commons, and Tidelift. The research is published today in Nature Machine Intelligence .

Focus on finetuning

Researchers often use a technique called fine-tuning to improve the capabilities of a large language model that will be deployed for a specific task, like question-answering. For finetuning, they carefully build curated datasets designed to boost a model’s performance for this one task.

The MIT researchers focused on these fine-tuning datasets, which are often developed by researchers, academic organizations, or companies and licensed for specific uses.

When crowdsourced platforms aggregate such datasets into larger collections for practitioners to use for fine-tuning, some of that original license information is often left behind.

“These licenses ought to matter, and they should be enforceable,” Mahari says.

For instance, if the licensing terms of a dataset are wrong or missing, someone could spend a great deal of money and time developing a model they might be forced to take down later because some training data contained private information.

“People can end up training models where they don’t even understand the capabilities, concerns, or risk of those models, which ultimately stem from the data,” Longpre adds.

To begin this study, the researchers formally defined data provenance as the combination of a dataset’s sourcing, creating, and licensing heritage, as well as its characteristics. From there, they developed a structured auditing procedure to trace the data provenance of more than 1,800 text dataset collections from popular online repositories.

After finding that more than 70 percent of these datasets contained “unspecified” licenses that omitted much information, the researchers worked backward to fill in the blanks. Through their efforts, they reduced the number of datasets with “unspecified” licenses to around 30 percent.

Their work also revealed that the correct licenses were often more restrictive than those assigned by the repositories.   

In addition, they found that nearly all dataset creators were concentrated in the global north, which could limit a model’s capabilities if it is trained for deployment in a different region. For instance, a Turkish language dataset created predominantly by people in the U.S. and China might not contain any culturally significant aspects, Mahari explains.

“We almost delude ourselves into thinking the datasets are more diverse than they actually are,” he says.

Interestingly, the researchers also saw a dramatic spike in restrictions placed on datasets created in 2023 and 2024, which might be driven by concerns from academics that their datasets could be used for unintended commercial purposes.

A user-friendly tool

To help others obtain this information without the need for a manual audit, the researchers built the Data Provenance Explorer. In addition to sorting and filtering datasets based on certain criteria, the tool allows users to download a data provenance card that provides a succinct, structured overview of dataset characteristics.

“We are hoping this is a step, not just to understand the landscape, but also help people going forward to make more informed choices about what data they are training on,” Mahari says.

In the future, the researchers want to expand their analysis to investigate data provenance for multimodal data, including video and speech. They also want to study how terms of service on websites that serve as data sources are echoed in datasets.

As they expand their research, they are also reaching out to regulators to discuss their findings and the unique copyright implications of fine-tuning data.

“We need data provenance and transparency from the outset, when people are creating and releasing these datasets, to make it easier for others to derive these insights,” Longpre says.

“Many proposed policy interventions assume that we can correctly assign and identify licenses associated with data, and this work first shows that this is not the case, and then significantly improves the provenance information available,” says Stella Biderman, executive director of EleutherAI, who was not involved with this work. “In addition, section 3 contains relevant legal discussion. This is very valuable to machine learning practitioners outside companies large enough to have dedicated legal teams. Many people who want to build AI systems for public good are currently quietly struggling to figure out how to handle data licensing, because the internet is not designed in a way that makes data provenance easy to figure out.”

Share this news article on:

Related links.

• Data Provenance Initiative
• Robert Mahari
• Shayne Longpre
• Alex “Sandy” Pentland
• School of Architecture and Planning

Related Topics

• Computer science and technology
• Artificial intelligence
• Human-computer interaction
• Technology and society

Related Articles

If art is how we express our humanity, where does AI fit in?

Large language models don’t behave like people, even though we may expect them to

Large language models use a surprisingly simple mechanism to retrieve some stored knowledge

Previous item Next item

More MIT News

For developing designers, there’s magic in 2.737 (Mechatronics)

Read full story →

How MIT’s online resources provide a “highly motivating, even transformative experience”

Students learn theater design through the power of play

A framework for solving parabolic partial differential equations

Designing better delivery for medical therapies

Making a measurable economic impact

• More news on MIT News homepage →

Massachusetts Institute of Technology 77 Massachusetts Avenue, Cambridge, MA, USA

• Map (opens in new window)
• Events (opens in new window)
• People (opens in new window)
• Careers (opens in new window)
• Accessibility
• Social Media Hub
• MIT on Facebook
• MIT on YouTube
• MIT on Instagram

A TinyML model for sidewalk obstacle detection: aiding the blind and visually impaired people

• Published: 03 September 2024

Cite this article

• Ahmed Boussihmed 1 ,
• Khalid El Makkaoui   ORCID: orcid.org/0000-0002-9569-9162 2 ,
• Ibrahim Ouahbi 2 ,
• Yassine Maleh   ORCID: orcid.org/0000-0003-4704-5364 3 &
• Abdelaziz Chetouani   ORCID: orcid.org/0000-0002-0239-2832 1  

This paper presents a pioneering study on the feasibility of implementing deep learning on resource-restricted IoT devices for real-world applications. We introduce a TinyML model configured for sidewalk obstacle detection tailored explicitly to assist those with visual impairments-a demographic often hindered by urban navigation challenges. Our investigation primarily focuses on adapting traditionally computationally intensive deep learning models to the stringent confines of IoT systems, where both memory and processing power are markedly limited. With a remarkably small footprint of just 1.93 MB and a robust mean average precision (mAP) of 50%, the proposed model achieves breakthrough outcomes, making it particularly well-suited for lightweight IoT devices. We demonstrate an exceptional inference speed of 96.2 milliseconds on a standard CPU, signifying a substantial step toward real-time processing in assistive technologies. The implications of this research are profound, emphasizing TinyML’s potential to bridge the gap between advanced machine learning capabilities and the accessibility demands of assistive devices for visually impaired individuals.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Explore related subjects

• Artificial Intelligence
• Medical Imaging

Data Availibility Statement

The SOD-v2 image dataset is available at: “ https://universe.roboflow.com/lamao/sod-enect/dataset/2 ” The training scripts and deployment code are available at: “ https://github.com/lamao-ab/object-detection-in-iot-devices ”

Divya K, Kannadasan R (2023) A systematic review and applications of how AI evolved in healthcare. Opt Quan Elect 56(3). https://doi.org/10.1007/s11082-023-05798-2

Rustam F, Imtiaz Z, Mehmood A, Rupapara V, Choi GS, Din S, Ashraf I (2022) Automated disease diagnosis and precaution recommender system using supervised machine learning. Multim Tool Appl 81(22):31929–31952. https://doi.org/10.1007/s11042-022-12897-x

Article   Google Scholar  

Saif S, Karmakar K, Biswas S, Neogy S (2022) MLIDS: Machine Learning Enabled Intrusion Detection System for Health Monitoring Framework Using BA-WSN. Int J Wireless Inf Networks 29(4):491–502. https://doi.org/10.1007/s10776-022-00574-7

Singh KN, Mantri JK (2023) Clinical decision support system based on RST with machine learning for medical data classification. Multimed Tool Appl 83(13):39707–39730. https://doi.org/10.1007/s11042-023-16802-y

An Q, Rahman S, Zhou J, Kang JJ (2023) A Comprehensive Review on Machine Learning in Healthcare Industry: Classification, Restrictions. Opportunities and Challenges. Sensors 23(9):4178. https://doi.org/10.3390/s23094178

Kuriakose B, Shrestha R, Sandnes FE (2020) Tools and Technologies for Blind and Visually Impaired Navigation Support: A Review. IETE Tech Rev 39(1):3–18. https://doi.org/10.1080/02564602.2020.1819893

Tapu R, Mocanu B, Zaharia T (2020) Wearable assistive devices for visually impaired: A state of the art survey. Pattern Recogn Lett 137:37–52. https://doi.org/10.1016/j.patrec.2018.10.031

Albusac J, Herrera V, Schez-Sobrino S, Grande R, Monekosso DN, Vallejo D (2024) Innovative haptic-based system for upper limb rehabilitation in visually impaired individuals: a multilayer approach. Multimed Tool Appl 83(21):60537–60563. https://doi.org/10.1007/s11042-023-17892-4

Patel I, Kulkarni M, Mehendale N (2024) Review of sensor-driven assistive device technologies for enhancing navigation for the visually impaired. Multimed Tool Appl . https://doi.org/10.1007/s11042-023-17552-7

Chit SM, Yap KM, Ahmad A (2024) Multi-sensory learning framework for visually impaired learners: Use of 3D, haptic, audio, olfactory media. Multimed Tools Appl. https://doi.org/10.1007/s11042-024-18249-1

Abadade Y, Temouden A, Bamoumen H et al (2023) A Comprehensive Survey on TinyML. IEEE Access 11:96892–96922. https://doi.org/10.1109/access.2023.3294111

Schizas N, Karras A, Karras C, Sioutas S (2022) TinyML for Ultra-Low Power AI and Large Scale IoT Deployments: A Systematic Review. Future Internet 14(12):363. https://doi.org/10.3390/fi14120363

Gubbi J, Buyya R, Marusic S, Palaniswami M (2013) Internet of Things (IoT): A vision, architectural elements, and future directions. Futur Gener Comput Syst 29:1645–1660. https://doi.org/10.1016/j.future.2013.01.010

Liu D, Wang Y, Chen T, Matson ET (2020) Accurate Lane Detection for Self-Driving Cars: An Approach Based on Color Filter Adjustment and K-Means Clustering Filter. Int J Semantic Comput 14(01):153–168. https://doi.org/10.1142/s1793351x20500038

Cui Y, Yan L, Cao Z, Liu D (2021) TF-Blender: Temporal Feature Blender for Video Object Detection. 2021 IEEE/CVF International Conference on Computer Vision (ICCV). https://doi.org/10.1109/iccv48922.2021.00803

Liu D, Cui Y, Chen Y, Zhang J, Fan B (2020) Video object detection for autonomous driving: Motion-aid feature calibration. Neurocomputing 409:1–11. https://doi.org/10.1016/j.neucom.2020.05.027

Liu D, Cui Y, Tan W, Chen Y (2021) SG-Net: Spatial Granularity Network for One-Stage Video Instance Segmentation. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2103.10284

Farcy R, Leroux R, Jucha A, Damaschini R, Grégoire C, Zogaghi A (2006) Electronic travel aids and electronic orientation aids for blind people: Technical, rehabilitation and everyday life points of view. In Conference Workshop on Assistive Technologies for People with Vision And Hearing Impairments Technology for Inclusion (Vol. 12, p. 12)

Garcia AR, Fonseca R, Duran A (2011) Electronic long cane for locomotion improving on visual impaired people. A case study. 2011 Pan American Health Care Exchanges. https://doi.org/10.1109/pahce.2011.5871848

Yiting Yi, Lunfu Dong (2015) A design of blind-guide crutch based on multi-sensors. 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD). https://doi.org/10.1109/fskd.2015.7382309

Mahendran JK, Barry DT, Nivedha AK, Bhandarkar SM (2021) Computer Vision-based Assistance System for the Visually Impaired Using Mobile Edge Artificial Intelligence. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). https://doi.org/10.1109/cvprw53098.2021.00274

Mukhiddinov M, Cho J (2021) Smart Glass System Using Deep Learning for the Blind and Visually Impaired. Electronics 10:2756. https://doi.org/10.3390/electronics10222756

Li J, Xie L, Chen Z et al (2023) An AIoT-Based Assistance System for Visually Impaired People. Electronics 12:3760. https://doi.org/10.3390/electronics12183760

Hsieh Y-Z, Lin S-S, Xu F-X (2020) Development of a wearable guide device based on convolutional neural network for blind or visually impaired persons. Multimed Tools Appl 79:29473–29491. https://doi.org/10.1007/s11042-020-09464-7

Chen S, Yao D, Cao H, Shen C (2019) A Novel Approach to Wearable Image Recognition Systems to Aid Visually Impaired People. Appl Sci 9:3350. https://doi.org/10.3390/app9163350

Bauer Z, Dominguez A, Cruz E et al (2020) Enhancing perception for the visually impaired with deep learning techniques and low-cost wearable sensors. Pattern Recogn Lett 137:27–36. https://doi.org/10.1016/j.patrec.2019.03.008

Shadi S, Hadi S, Nazari MA, Hardt W (2019). Outdoor navigation for visually impaired based on deep learning. In Proc. CEUR Workshop Proc (Vol. 2514, pp. 97-406)

Venkat Ragavan S, Tarun AH, Yogeeshwar S et al (2023) A realtime portable and accessible aiding system for the blind - a cloud based approach. Multimed Tools Appl 82:20641–20654. https://doi.org/10.1007/s11042-023-14419-9

Hong K, He W, Tang H, Zhang X, Li Q, Zhou B (2024) SPVINet: A Lightweight Multitask Learning Network for Assisting Visually Impaired People in Multiscene Perception. IEEE Int Things J 1. https://doi.org/10.1109/jiot.2024.3371978

Cao Z, Xu X, Hu B, Zhou M (2021) Rapid Detection of Blind Roads and Crosswalks by Using a Lightweight Semantic Segmentation Network. IEEE Trans Intell Transp Syst 22(10):6188–6197. https://doi.org/10.1109/tits.2020.2989129

Russakovsky O, Deng J, Su H et al (2015) ImageNet Large Scale Visual Recognition Challenge. Int J Comput Vision 115:211–252. https://doi.org/10.1007/s11263-015-0816-y

Article   MathSciNet   Google Scholar  

Open Images V7. https://storage.googleapis.com/openimages/web/index.html

Lin T-Y, Maire M, Belongie S et al (2014) Microsoft COCO: Common Objects in Context. Comput Vision - ECCV 2014:740–755. https://doi.org/10.1007/978-3-319-10602-1_48

Everingham M, Van Gool L, Williams CKI et al (2009) The Pascal Visual Object Classes (VOC) Challenge. Int J Comput Vision 88:303–338. https://doi.org/10.1007/s11263-009-0275-4

Ding J, Xue N, Xia G-S et al (2022) Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges. IEEE Trans Pattern Anal Mach Intell 44:7778–7796. https://doi.org/10.1109/tpami.2021.3117983

Geiger A, Lenz P, Urtasun R (2012) Are we ready for autonomous driving? The KITTI vision benchmark suite. 2012 IEEE conference on computer vision and pattern recognition. https://doi.org/10.1109/cvpr.2012.6248074

Boussihmed A, El Makkaoui K, Chetouani A, et al (2024) An intelligent model for detecting obstacles on sidewalks for visually impaired people. Lecture notes in networks and systems. pp 398–404. https://doi.org/10.1007/978-3-031-48573-2_57

Roboflow: Give your software the power to see objects in images and video. https://roboflow.com/

Zoph B, Cubuk ED, Ghiasi G et al (2020) Learning data augmentation strategies for object detection. Comput Vision - ECCV 2020:566–583. https://doi.org/10.1007/978-3-030-58583-9_34

Moreno-Barea FJ, Strazzera F, Jerez JM, et al (2018) Forward Noise Adjustment Scheme for Data Augmentation. 2018 IEEE Symposium Series on Computational Intelligence (SSCI). https://doi.org/10.1109/ssci.2018.8628917

Rodriguez-Conde I, Campos C, Fdez-Riverola F (2021) Optimized convolutional neural network architectures for efficient on-device vision-based object detection. Neural Comput Appl 34:10469–10501. https://doi.org/10.1007/s00521-021-06830-w

Girshick R, Donahue J, Darrell T, Malik J (2014). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. https://doi.org/10.1109/cvpr.2014.81

Girshick R (2015). Fast R-CNN. https://doi.org/10.1109/iccv.2015.169

Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149. https://doi.org/10.1109/tpami.2016.2577031

Liu W, Anguelov D, Erhan D, Szegedy C, Reed SE, Fu C-Y, Berg AC (2016) SSD: Single Shot MultiBox Detector. In: Lecture notes in computer science. pp 21–37

Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. https://doi.org/10.1109/cvpr.2016.91

Tan M, Pang R, Le QV (2020) EfficientDet: scalable and efficient object detection. https://doi.org/10.1109/cvpr42600.2020.01079

Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861

Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely Connected Convolutional Networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr.2017.243

Baker B, Gupta O, Naik N, Raskar R (2016) Designing neural network architectures using reinforcement learning. arXiv preprint arXiv:1611.02167

Redmon J, Farhadi A (2017) YOLO9000: Better, Faster. Stronger IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr.2017.690

Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size. arXiv preprint arXiv:1602.07360

Zhang X, Zhou X, Lin M, Sun J (2018) ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. IEEE/CVF Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1109/cvpr.2018.00716

Han K, Wang Y, Tian Q, Guo J, Xu C, Xu C. (2020) Ghostnet: More features from cheap operations. IEEE/CVF conference on computer vision and pattern recognition. pp 1580–1589

Jocher G, Chaurasia A, Qiu J (2023) Ultralytics YOLO (Version 8.0.0). https://github.com/ultralytics/ultralytics

Jocher, G (2020). YOLOv5 by Ultralytics (Version 7.0). https://doi.org/10.5281/zenodo.3908559.

Zhang S, Wen L, Bian X, et al (2017) Single-Shot Refinement Neural Network for Object Detection. In: arXiv.org. https://doi.org/10.48550/arXiv.1711.06897

Wang N, Gao Y, Chen H et al (2021) NAS-FCOS: Efficient Search for Object Detection Architectures. Int J Comput Vision 129:3299–3312. https://doi.org/10.1007/s11263-021-01523-2

Law H, Teng Y, Russakovsky O, Deng, J (2019) Cornernet-lite: Efficient keypoint based object detection. arXiv preprint arXiv:1904.08900

Sandler M, Howard A, Zhu M, et al (2018) MobileNetV2: Inverted Residuals and Linear Bottlenecks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1109/cvpr.2018.00474

Qin Z, Li Z, Zhang Z, et al (2019) ThunderNet: Towards Real-Time Generic Object Detection on Mobile Devices. 2019 IEEE/CVF International Conference on Computer Vision (ICCV). https://doi.org/10.1109/iccv.2019.00682

Chen C, Liu M, Meng X, et al (2020) RefineDetLite: A Lightweight One-stage Object Detection Framework for CPU-only Devices. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). https://doi.org/10.1109/cvprw50498.2020.00358

Ghiasi G, Lin T-Y, Le QV (2019) NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr.2019.00720

Tang Q, Li J, Shi Z, Hu Y (2020) Lightdet: A Lightweight and Accurate Object Detection Network. ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). https://doi.org/10.1109/icassp40776.2020.9054101

Fan B, Chen Y, Qu J, et al (2019) FFBNet: Lightweight Backbone for Object Detection Based Feature Fusion Block. 2019 IEEE International Conference on Image Processing (ICIP). https://doi.org/10.1109/icip.2019.8803683

Wang RJ, Li X, Ling CX (2018) Pelee: A real-time object detection system on mobile devices. Adv Neural Inf Process Sys 31

Tan M, Chen B, Pang R, et al (2019) MnasNet: Platform-Aware Neural Architecture Search for Mobile. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr.2019.00293

Howard A, Sandler M, Chen B, et al (2019) Searching for MobileNetV3. 2019 IEEE/CVF International Conference on Computer Vision (ICCV). https://doi.org/10.1109/iccv.2019.00140

Ling H, Zhang L, Qin Y, Shi Y, Wu L, Chen J, Zhang B (2019). BMNet: A Reconstructed Network for Lightweight Object Detection via Branch Merging. British Machine Vision Conference

Li Y, Li J, Lin W, Li J (2018) Tiny-DSOD: Lightweight object detection for resource-restricted usages. arXiv:1807.11013

Padilla R, Netto SL, da Silva EAB (2020) A Survey on Performance Metrics for Object-Detection Algorithms. 2020 International Conference on Systems, Signals and Image Processing (IWSSIP). https://doi.org/10.1109/iwssip48289.2020.9145130

Berthelier A, Chateau T, Duffner S et al (2020) Deep Model Compression and Architecture Optimization for Embedded Systems: A Survey. J Sig Process Sys 93:863–878. https://doi.org/10.1007/s11265-020-01596-1

Hinton G, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. arXiv:1503.02531

Han S, Mao H, Dally WJ (2015) Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv:1510.00149

Gholami A., Kim S, Dong Z, Yao Z, Mahoney M W, Keutzer K (2022) A survey of quantization methods for efficient neural network inference. In: Low-power computer vision. Chapman and Hall/CRC, pp 291–326

Dev Board Micro. In: Coral. https://coral.ai/products/dev-board-micro/#tech-specs

ESP-WHO Face Detection Solution. In: Espressif. https://www.espressif.com/en/products/devkits/esp-eye/overview

Himax WE-I Plus. In: Himax Technologies. https://www.himax.com.tw/product-brief/HX6537.39.40-A_product_brief.pdf

xG24-DK2601B EFR32xG24 Wireless Dev Kit. In: Silicon Labs. https://www.silabs.com/development-tools/wireless/efr32xg24-dev-kit?tab=overview

Sony’s Spresense. In: Sony Developer World. https://developer.sony.com/spresense/product-specifications#secondary-menu-desktop

Nicla Vision. In: Arduino Official Store. https://store.arduino.cc/products/nicla-vision

Arduino Nano 33 BLE Sense Rev2 with headers. In: Arduino Online Shop. https://store-usa.arduino.cc/products/nano-33-ble-sense-rev2-with-headers

Loh YP, Chan CS (2019) Getting to know low-light images with the Exclusively Dark dataset. Comput Vis Image Underst 178:30–42. https://doi.org/10.1016/j.cviu.2018.10.010

Shi Z, Zhu MM, Guo B, Zhao M (2018) Zhang C (2018) Nighttime low illumination image enhancement with single image using bright/dark channel prior. EURASIP J Image Video Process 1. https://doi.org/10.1186/s13640-018-0251-4

Download references

Author information

Authors and affiliations.

LaMAO, ORAS team, ENCG, Mohammed Premier University, Oujda, Morocco

Ahmed Boussihmed & Abdelaziz Chetouani

Multidisciplinary Faculty of Nador, Mohammed Premier University, Oujda, Morocco

Khalid El Makkaoui & Ibrahim Ouahbi

Laboratory LaSTI, ENSAK, Sultan Moulay Slimane University, Beni Mellal, Morocco

Yassine Maleh

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Yassine Maleh .

Ethics declarations

Conflicts of interest.

The authors declare they have no potential competing interests in this article’s research, authorship, or publication.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Boussihmed, A., El Makkaoui, K., Ouahbi, I. et al. A TinyML model for sidewalk obstacle detection: aiding the blind and visually impaired people. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-20070-9

Download citation

Received : 08 May 2024

Revised : 09 July 2024

Accepted : 09 August 2024

Published : 03 September 2024

DOI : https://doi.org/10.1007/s11042-024-20070-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Blind and visually impaired people (BVIP)
• Computer vision (CV)
• Deep learning (DL)
• Internet of Things (IoT)
• Object detection
• Tiny machine learning (TinyML)

Advertisement

• Find a journal
• Publish with us
• Track your research