recent research papers in image processing

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts

Image processing articles within Nature Methods

Article | 03 July 2024

Gapr for large-scale collaborative single-neuron reconstruction

Gapr is an efficient platform for reconstructing neurons in large-scale light microscopy datasets. It enables various proofreading modes as well as collaboration among many annotators.

• Lingfeng Gou
• , Yanzhi Wang
•  &  Jun Yan

Correspondence | 10 June 2024

Omega — harnessing the power of large language models for bioimage analysis

• Loïc A. Royer

Correspondence | 17 May 2024

DL4MicEverywhere: deep learning for microscopy made flexible, shareable and reproducible

• Iván Hidalgo-Cenalmor
• , Joanna W. Pylvänäinen
•  &  Estibaliz Gómez-de-Mariscal

Article | 12 April 2024

Pretraining a foundation model for generalizable fluorescence microscopy-based image restoration

A pretrained foundation model (UniFMIR) enables versatile and generalizable performance across diverse fluorescence microscopy image reconstruction tasks.

• , Weimin Tan
•  &  Bo Yan

Resource 09 April 2024 | Open Access

Three million images and morphological profiles of cells treated with matched chemical and genetic perturbations

The CPJUMP1 Resource comprises Cell Painting images and profiles of 75 million cells treated with hundreds of chemical and genetic perturbations. The dataset enables exploration of their relationships and lays the foundation for the development of advanced methods to match perturbations.

• Srinivas Niranj Chandrasekaran
• , Beth A. Cimini
•  &  Anne E. Carpenter

Research Briefing | 01 April 2024

Creating a universal cell segmentation algorithm

Cell segmentation currently involves the use of various bespoke algorithms designed for specific cell types, tissues, staining methods and microscopy technologies. We present a universal algorithm that can segment all kinds of microscopy images and cell types across diverse imaging protocols.

Analysis | 26 March 2024

The multimodality cell segmentation challenge: toward universal solutions

Cell segmentation is crucial in many image analysis pipelines. This analysis compares many tools on a multimodal cell segmentation benchmark. A Transformer-based model performed best in terms of performance and general applicability.

• , Ronald Xie
•  &  Bo Wang

Editorial | 12 February 2024

Where imaging and metrics meet

When it comes to bioimaging and image analysis, details matter. Papers in this issue offer guidance for improved robustness and reproducibility.

Correspondence | 24 January 2024

EfficientBioAI: making bioimaging AI models efficient in energy and latency

• , Jiajun Cao
•  &  Jianxu Chen

Correspondence | 08 January 2024

JDLL: a library to run deep learning models on Java bioimage informatics platforms

• Carlos García López de Haro
• , Stéphane Dallongeville
•  &  Jean-Christophe Olivo-Marin

Article 08 January 2024 | Open Access

Unsupervised and supervised discovery of tissue cellular neighborhoods from cell phenotypes

CytoCommunity enables both supervised and unsupervised analyses of spatial omics data in order to identify complex tissue cellular neighborhoods based on cell phenotypes and spatial distributions.

• , Jiazhen Rong
•  &  Kai Tan

Article 04 January 2024 | Open Access

Image restoration of degraded time-lapse microscopy data mediated by near-infrared imaging

InfraRed-mediated Image Restoration (IR 2 ) uses deep learning to combine the benefits of deep-tissue imaging with NIR probes and the convenience of imaging with GFP for improved time-lapse imaging of embryogenesis.

• Nicola Gritti
• , Rory M. Power
•  &  Jan Huisken

Method to Watch | 06 December 2023

Imaging across scales

New twists on established methods and multimodal imaging are poised to bridge gaps between cellular and organismal imaging.

• Rita Strack

Visual proteomics

Advances will enable proteome-scale structure determination in cells.

Article 06 December 2023 | Open Access

Embryo mechanics cartography: inference of 3D force atlases from fluorescence microscopy

Foambryo is an analysis pipeline for three-dimensional force-inference measurements in developing embryos.

• Sacha Ichbiah
• , Fabrice Delbary
•  &  Hervé Turlier

Article | 06 December 2023

TubULAR: tracking in toto deformations of dynamic tissues via constrained maps

TubULAR is an in toto tissue cartography method for mapping complex dynamic surfaces

• Noah P. Mitchell
•  &  Dillon J. Cislo

Research Briefing | 05 December 2023

Inferring how animals deform improves cell tracking

Tracking cells is a time-consuming part of biological image analysis, and traditional manual annotation methods are prohibitively laborious for tracking neurons in the deforming and moving Caenorhabditis elegans brain. By leveraging machine learning to develop a ‘targeted augmentation’ method, we substantially reduced the number of labeled images required for tracking.

Article | 05 December 2023

Automated neuron tracking inside moving and deforming C. elegans using deep learning and targeted augmentation

Targettrack is a deep-learning-based pipeline for automatic tracking of neurons within freely moving C. elegans . Using targeted augmentation, the pipeline has a reduced need for manually annotated training data.

• Core Francisco Park
• , Mahsa Barzegar-Keshteli
•  &  Sahand Jamal Rahi

Brief Communication | 16 November 2023

Improving resolution and resolvability of single-particle cryoEM structures using Gaussian mixture models

This manuscript describes a refinement protocol that extends the e2gmm method to optimize both the orientation and conformation estimation of particles to improve the alignment for flexible domains of proteins.

• Muyuan Chen
• , Michael F. Schmid
•  &  Wah Chiu

Article 13 November 2023 | Open Access

Bio-friendly long-term subcellular dynamic recording by self-supervised image enhancement microscopy

DeepSeMi is a self-supervised denoising framework that can enhance SNR over 12 dB across diverse samples and imaging modalities. DeepSeMi enables extended longitudinal imaging of subcellular dynamics with high spatiotemporal resolution.

• Guoxun Zhang
• , Xiaopeng Li
•  &  Qionghai Dai

High-fidelity 3D live-cell nanoscopy through data-driven enhanced super-resolution radial fluctuation

Enhanced super-resolution radial fluctuations (eSRRF) offers improved image fidelity and resolution compared to the popular SRRF method and further enables volumetric live-cell super-resolution imaging at high speeds.

• Romain F. Laine
• , Hannah S. Heil
•  &  Ricardo Henriques

Article 26 October 2023 | Open Access

nextPYP: a comprehensive and scalable platform for characterizing protein variability in situ using single-particle cryo-electron tomography

nextPYP is a turn-key framework for single-particle cryo-electron tomography that streamlines complex data analysis pipelines, from pre-processing of tilt series to high-resolution refinement, for efficient analysis and visualization of large datasets.

• Hsuan-Fu Liu
•  &  Alberto Bartesaghi

Article | 07 September 2023

FIOLA: an accelerated pipeline for fluorescence imaging online analysis

FIOLA is a pipeline for processing calcium or voltage imaging data. Its advantages include the fast speed and online processing.

• Changjia Cai
• , Cynthia Dong
•  &  Andrea Giovannucci

Correspondence | 18 August 2023

napari-imagej: ImageJ ecosystem access from napari

• Gabriel J. Selzer
• , Curtis T. Rueden
•  &  Kevin W. Eliceiri

Article 17 August 2023 | Open Access

Alignment of spatial genomics data using deep Gaussian processes

Gaussian Process Spatial Alignment (GPSA) aligns multiple spatially resolved genomics and histology datasets and improves downstream analysis.

• Andrew Jones
• , F. William Townes
•  &  Barbara E. Engelhardt

Brief Communication 27 July 2023 | Open Access

Segmentation metric misinterpretations in bioimage analysis

This study shows the importance of proper metrics for comparing algorithms for bioimage segmentation and object detection by exploring the impact of metrics on the relative performance of algorithms in three image analysis competitions.

• Dominik Hirling
• , Ervin Tasnadi
•  &  Peter Horvath

Article | 27 July 2023

DBlink: dynamic localization microscopy in super spatiotemporal resolution via deep learning

DBlink uses deep learning to capture long-term dependencies between different frames in single-molecule localization microscopy data, yielding super spatiotemporal resolution videos of fast dynamic processes in living cells.

• , Onit Alalouf
•  &  Yoav Shechtman

Editorial | 11 July 2023

What’s next for bioimage analysis?

Advanced bioimage analysis tools are poised to disrupt the way in which microscopy images are acquired and analyzed. This Focus issue shares the hopes and opinions of experts on the near and distant future of image analysis.

Comment | 11 July 2023

The future of bioimage analysis: a dialog between mind and machine

The field of bioimage analysis is poised for a major transformation, owing to advancements in imaging technologies and artificial intelligence. The emergence of multimodal foundation models — which are akin to large language models (such as ChatGPT) but are capable of comprehending and processing biological images — holds great potential for ushering in a revolutionary era in bioimage analysis.

Unveiling the vision: exploring the potential of image analysis in Africa

Here we discuss the prospects of bioimage analysis in the context of the African research landscape as well as challenges faced in the development of bioimage analysis in countries on the continent. We also speculate about potential approaches and areas of focus to overcome these challenges and thus build the communities, infrastructure and initiatives that are required to grow image analysis in African research.

• Mai Atef Rahmoon
• , Gizeaddis Lamesgin Simegn
•  &  Michael A. Reiche

The Twenty Questions of bioimage object analysis

The language used by microscopists who wish to find and measure objects in an image often differs in critical ways from that used by computer scientists who create tools to help them do this, making communication hard across disciplines. This work proposes a set of standardized questions that can guide analyses and shows how it can improve the future of bioimage analysis as a whole by making image analysis workflows and tools more FAIR (findable, accessible, interoperable and reusable).

• Beth A. Cimini

Smart microscopes of the future

We dream of a future where light microscopes have new capabilities: language-guided image acquisition, automatic image analysis based on extensive prior training from biologist experts, and language-guided image analysis for custom analyses. Most capabilities have reached the proof-of-principle stage, but implementation would be accelerated by efforts to gather appropriate training sets and make user-friendly interfaces.

• Anne E. Carpenter

Using AI in bioimage analysis to elevate the rate of scientific discovery as a community

The future of bioimage analysis is increasingly defined by the development and use of tools that rely on deep learning and artificial intelligence (AI). For this trend to continue in a way most useful for stimulating scientific progress, it will require our multidisciplinary community to work together, establish FAIR (findable, accessible, interoperable and reusable) data sharing and deliver usable and reproducible analytical tools.

• Damian Dalle Nogare
• , Matthew Hartley
•  &  Florian Jug

Scaling biological discovery at the interface of deep learning and cellular imaging

Concurrent advances in imaging technologies and deep learning have transformed the nature and scale of data that can now be collected with imaging. Here we discuss the progress that has been made and outline potential research directions at the intersection of deep learning and imaging-based measurements of living systems.

• Morgan Schwartz
• , Uriah Israel
•  &  David Van Valen

Towards effective adoption of novel image analysis methods

The bridging of domains such as deep learning-driven image analysis and biology brings exciting promises of previously impossible discoveries as well as perils of misinterpretation and misapplication. We encourage continual communication between method developers and application scientists that emphases likely pitfalls and provides validation tools in conjunction with new techniques.

• Talley Lambert
•  &  Jennifer Waters

Towards foundation models of biological image segmentation

In the ever-evolving landscape of biological imaging technology, it is crucial to develop foundation models capable of adapting to various imaging modalities and tackling complex segmentation tasks.

When seeing is not believing: application-appropriate validation matters for quantitative bioimage analysis

A key step toward biologically interpretable analysis of microscopy image-based assays is rigorous quantitative validation with metrics appropriate for the particular application in use. Here we describe this challenge for both classical and modern deep learning-based image analysis approaches and discuss possible solutions for automating and streamlining the validation process in the next five to ten years.

• Jianxu Chen
• , Matheus P. Viana
•  &  Susanne M. Rafelski

Article | 10 July 2023

SCS: cell segmentation for high-resolution spatial transcriptomics

Subcellular spatial transcriptomics cell segmentation (SCS) combines information from stained images and sequencing data to improve cell segmentation in high-resolution spatial transcriptomics data.

• , Dongshunyi Li
•  &  Ziv Bar-Joseph

Research Highlight | 09 June 2023

Capturing hyperspectral images

A single-shot hyperspectral phasor camera (SHy-Cam) enables fast, multiplexed volumetric imaging.

Correspondence | 05 June 2023

Distributed-Something: scripts to leverage AWS storage and computing for distributed workflows at scale

• Erin Weisbart
•  &  Beth A. Cimini

Brief Communication | 29 May 2023

New measures of anisotropy of cryo-EM maps

This paper proposes two new anisotropy metrics—the Fourier shell occupancy and the Bingham test—that can be used to understand the quality of cryogenic electron microscopy maps.

• Jose-Luis Vilas
•  &  Hemant D. Tagare

Analysis 18 May 2023 | Open Access

The Cell Tracking Challenge: 10 years of objective benchmarking

This updated analysis of the Cell Tracking Challenge explores how algorithms for cell segmentation and tracking in both 2D and 3D have advanced in recent years, pointing users to high-performing tools and developers to open challenges.

• Martin Maška
• , Vladimír Ulman
•  &  Carlos Ortiz-de-Solórzano

Article 15 May 2023 | Open Access

TomoTwin: generalized 3D localization of macromolecules in cryo-electron tomograms with structural data mining

TomoTwin is a deep metric learning-based particle picking method for cryo-electron tomograms. TomoTwin obviates the need for annotating training data and retraining a picking model for each protein.

• , Thorsten Wagner
•  &  Stefan Raunser

Research Briefing | 12 May 2023

Mapping the motion and structure of flexible proteins from cryo-EM data

A deep learning algorithm maps out the continuous conformational changes of flexible protein molecules from single-particle cryo-electron microscopy images, allowing the visualization of the conformational landscape of a protein with improved resolution of its moving parts.

Article 11 May 2023 | Open Access

Cross-modality supervised image restoration enables nanoscale tracking of synaptic plasticity in living mice

XTC is a supervised deep-learning-based image-restoration approach that is trained with images from different modalities and applied to an in vivo modality with no ground truth. XTC’s capabilities are demonstrated in synapse tracking in the mouse brain.

• Yu Kang T. Xu
• , Austin R. Graves
•  &  Jeremias Sulam

3DFlex: determining structure and motion of flexible proteins from cryo-EM

3D Flexible Refinement (3DFlex) is a generative neural network model for continuous molecular heterogeneity for cryo-EM data that can be used to determine the structure and motion of flexible biomolecules. It enables visualization of nonrigid motion and improves 3D structure resolution by aggregating information from particle images spanning the conformational landscape of the target molecule.

• Ali Punjani
•  &  David J. Fleet

Resource 08 May 2023 | Open Access

EmbryoNet: using deep learning to link embryonic phenotypes to signaling pathways

EmbryoNet is an automated approach to the phenotyping of developing embryos that surpasses experts in terms of speed, accuracy and sensitivity. A large annotated image dataset of zebrafish, medaka and stickleback development rounds out this resource.

• Daniel Čapek
• , Matvey Safroshkin
•  &  Patrick Müller

Article 01 April 2023 | Open Access

Rapid detection of neurons in widefield calcium imaging datasets after training with synthetic data

DeepWonder removes background signals from widefield calcium recordings and enables accurate and efficient neuronal segmentation with high throughput.

• Yuanlong Zhang
• , Guoxun Zhang

Correspondence | 10 February 2023

MoBIE: a Fiji plugin for sharing and exploration of multi-modal cloud-hosted big image data

• Constantin Pape
• , Kimberly Meechan
•  &  Christian Tischer

Article 23 January 2023 | Open Access

Convolutional networks for supervised mining of molecular patterns within cellular context

DeePiCt (deep picker in context) is a versatile, open-source deep-learning framework for supervised segmentation and localization of subcellular organelles and biomolecular complexes in cryo-electron tomography.

• Irene de Teresa-Trueba
• , Sara K. Goetz
•  &  Judith B. Zaugg

Browse broader subjects

• Computational biology and bioinformatics

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• PubMed/Medline
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Developments in image processing using deep learning and reinforcement learning.

1. Introduction

2. methodology, 2.1. search process and sources of information, 2.2. inclusion and exclusion criteria for article selection, 3. technical background, 3.1. graphics processing units, 3.2. image processing, 3.3. machine learning overview.

• In supervised learning, we can determine predictive functions using labeled training datasets, meaning each data object instance must include an input for both the values and the expected labels or output values [ 21 ]. This class of algorithms tries to identify the relationships between input and output values and generate a predictive model able to determine the result based only on the corresponding input data [ 3 , 21 ]. Supervised learning methods are suitable for regression and data classification, being primarily used for a variety of algorithms like linear regression, artificial neural networks (ANNs), decision trees (DTs), support vector machines (SVMs), k-nearest neighbors (KNNs), random forest (RF), and others [ 3 ]. As an example, systems using RF and DT algorithms have developed a huge impact on areas such as computational biology and disease prediction, while SVM has also been used to study drug–target interactions and to predict several life-threatening diseases, such as cancer or diabetes [ 23 ].
• Unsupervised learning is typically used to solve several problems in pattern recognition based on unlabeled training datasets. Unsupervised learning algorithms are able to classify the training data into different categories according to their different characteristics [ 21 , 24 ], mainly based on clustering algorithms [ 24 ]. The number of categories is unknown, and the meaning of each category is unclear; therefore, unsupervised learning is usually used for classification problems and for association mining. Some commonly employed algorithms include K-means [ 3 ], SVM, or DT classifiers. Data processing tools like PCA, which is used for dimensionality reduction, are often necessary prerequisites before attempting to cluster a set of data.

3.3.1. Deep Learning Concepts

• Training a DNN implies the definition of a loss function, which is responsible for calculating the error made in the process given by the difference between the expected output value and that produced by the network. One of the most used loss functions in regression problems is the mean squared error (MSE) [ 30 ]. In the training phase, the weight vector that minimizes the loss function is adjusted, meaning it is not possible to obtain analytical solutions effectively. The loss function minimization method usually used is gradient descent [ 30 ].
• Activation functions are fundamental in the process of learning neural network models, as well as in the interpretation of complex nonlinear functions. The activation function adds nonlinear features to the model, allowing it to represent more than one linear function, which would not happen otherwise, no matter how many layers it had. The Sigmoid function is the most commonly used activation function in the early stages of studying neural networks [ 30 ].
• As their capacity to learn and adjust to data is greater than that of traditional ML models, it is more likely that overfitting situations will occur in DL models. For this reason, regularization represents a crucial and highly effective set of techniques used to reduce the generalization errors in ML. Some other techniques that can contribute to achieving this goal are increasing the size of the training dataset, stopping at an early point in the training phase, or randomly discarding a portion of the output of neurons during the training phase [ 30 ].
• In order to increase stability and reduce convergence times in DL algorithms, optimizers are used, with which greater efficiency in the hyperparameter adjustment process is also possible [ 30 ].

3.3.2. Reinforcement Learning Concepts

3.4. current challenges, 4. image processing developments, 4.1. domains, 4.1.1. research using deep learning.

• One of the first DL models used for video prediction, inspired by the sequence-to-sequence model usually used in natural language processing [ 97 ], uses a recurrent long and short term memory network (LSTM) to predict future images based on a sequence of images encoded during video data processing [ 97 ].
• In their research, Salahzadeh et al. [ 98 ] presented a novel mechatronics platform for static and real-time posture analysis, combining 3 complex components. The components included a mechanical structure with cameras, a software module for data collection and semi-automatic image analysis, and a network to provide the raw data to the DL server. The authors concluded that their device, in addition to being inexpensive and easy to use, is a method that allows postural assessment with great stability and in a non-invasive way, proving to be a useful tool in the rehabilitation of patients.
• Studies in graphical search engines and content-based image retrieval (CBIR) systems have also been successfully developed recently [ 11 , 82 , 99 , 100 ], with processing times that might be compatible with real-time applications. Most importantly, the corresponding results of these studies appeared to show adequate image retrieval capabilities, displaying an undisputed similarity between input and output, both on a semantic basis and a graphical basis [ 82 ]. In a review by Latif et al. [ 101 ], the authors concluded that image feature representation, as it is performed, is impossible to be represented by using a unique feature representation. Instead, it should be achieved by a combination of said low-level features, considering they represent the image in the form of patches and, as such, the performance is increased.
• In their publication, Rani et al. [ 102 ] reviewed the current literature found on this topic from the period from 1995 to 2021. The authors found that researchers in microbiology have employed ML techniques for the image recognition of four types of micro-organisms: bacteria, algae, protozoa, and fungi. In their research work, Kasinathan and Uyyala [ 17 ] apply computer vision and knowledge-based approaches to improve insect detection and classification in dense image scenarios. In this work, image processing techniques were applied to extract features, and classification models were built using ML algorithms. The proposed approach used different feature descriptors, such as texture, color, shape, histograms of oriented gradients (HOG) and global image descriptors (GIST). ML was used to analyze multivariety insect data to obtain the efficient utilization of resources and improved classification accuracy for field crop insects with a similar appearance.

4.1.2. Research Using Reinforcement Learning

5. discussion and future directions, 6. conclusions.

• Interest in image-processing systems using DL methods has exponentially increased over the last few years. The most common research disciplines for image processing and AI are medicine, computer science, and engineering.
• Traditional ML methods are still extremely relevant and are frequently used in fields such as computational biology and disease diagnosis and prediction or to assist in specific tasks when coupled with other more complex methods. DL methods have become of particular interest in many image-processing problems, particularly because of their ability to circumvent some of the challenges that more traditional approaches face.
• A lot of attention from researchers seems to focus on improving model performance, reducing computational resources and time, and expanding the application of ML models to solve concrete real-world problems.
• The medical field seems to have developed a particular interest in research using multiple classes and methods of learning algorithms. DL image processing has been useful in analyzing medical exams and other imaging applications. Some areas have also still found success using more traditional ML methods.
• Another area of interest appears to be autonomous driving and driver profiling, possibly powered by the increased access to information available both for the drivers and the vehicles alike. Indeed, modern driving assistance systems have already implemented features such as (a) road lane finding, (b) free driving space finding, (c) traffic sign detection and recognition, (d) traffic light detection and recognition, and (e) road-object detection and tracking. This research field will undoubtedly be responsible for many more studies in the near future.
• Graphical search engines and content-based image retrieval systems also present themselves as an interesting topic of research for image processing, with a diverse body of work and innovative approaches.

Author Contributions

Institutional review board statement, informed consent statement, data availability statement, acknowledgments, conflicts of interest, abbreviations.

• Raschka, S.; Patterson, J.; Nolet, C. Machine Learning in Python: Main Developments and Technology Trends in Data Science, Machine Learning, and Artificial Intelligence. Information 2020 , 11 , 193. [ Google Scholar ] [ CrossRef ]
• Barros, D.; Moura, J.; Freire, C.; Taleb, A.; Valentim, R.; Morais, P. Machine learning applied to retinal image processing for glaucoma detection: Review and perspective. BioMed. Eng. OnLine 2020 , 19 , 20. [ Google Scholar ] [ CrossRef ]
• Zhu, M.; Wang, J.; Yang, X.; Zhang, Y.; Zhang, L.; Ren, H.; Wu, B.; Ye, L. A review of the application of machine learning in water quality evaluation. Eco-Environ. Health 2022 , 1 , 107–116. [ Google Scholar ] [ CrossRef ]
• Singh, V.; Chen, S.S.; Singhania, M.; Nanavati, B.; kumar kar, A.; Gupta, A. How are reinforcement learning and deep learning algorithms used for big data based decision making in financial industries–A review and research agenda. Int. J. Inf. Manag. Data Insights 2022 , 2 , 100094. [ Google Scholar ] [ CrossRef ]
• Moscalu, M.; Moscalu, R.; Dascălu, C.G.; Țarcă, V.; Cojocaru, E.; Costin, I.M.; Țarcă, E.; Șerban, I.L. Histopathological Images Analysis and Predictive Modeling Implemented in Digital Pathology—Current Affairs and Perspectives. Diagnostics 2023 , 13 , 2379. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Wang, S.; Yang, D.M.; Rong, R.; Zhan, X.; Fujimoto, J.; Liu, H.; Minna, J.; Wistuba, I.I.; Xie, Y.; Xiao, G. Artificial Intelligence in Lung Cancer Pathology Image Analysis. Cancers 2019 , 11 , 1673. [ Google Scholar ] [ CrossRef ]
• van der Velden, B.H.M.; Kuijf, H.J.; Gilhuijs, K.G.; Viergever, M.A. Explainable artificial intelligence (XAI) in deep learning-based medical image analysis. Med. Image Anal. 2022 , 79 , 102470. [ Google Scholar ] [ CrossRef ]
• Prevedello, L.M.; Halabi, S.S.; Shih, G.; Wu, C.C.; Kohli, M.D.; Chokshi, F.H.; Erickson, B.J.; Kalpathy-Cramer, J.; Andriole, K.P.; Flanders, A.E. Challenges related to artificial intelligence research in medical imaging and the importance of image analysis competitions. Radiol. Artif. Intell. 2019 , 1 , e180031. [ Google Scholar ] [ CrossRef ]
• Smith, K.P.; Kirby, J.E. Image analysis and artificial intelligence in infectious disease diagnostics. Clin. Microbiol. Infect. 2020 , 26 , 1318–1323. [ Google Scholar ] [ CrossRef ]
• Wu, Q. Research on deep learning image processing technology of second-order partial differential equations. Neural Comput. Appl. 2023 , 35 , 2183–2195. [ Google Scholar ] [ CrossRef ]
• Jardim, S.; António, J.; Mora, C. Graphical Image Region Extraction with K-Means Clustering and Watershed. J. Imaging 2022 , 8 , 163. [ Google Scholar ] [ CrossRef ]
• Ying, C.; Huang, Z.; Ying, C. Accelerating the image processing by the optimization strategy for deep learning algorithm DBN. EURASIP J. Wirel. Commun. Netw. 2018 , 232 , 232. [ Google Scholar ] [ CrossRef ]
• Protopapadakis, E.; Voulodimos, A.; Doulamis, A.; Doulamis, N.; Stathaki, T. Automatic crack detection for tunnel inspection using deep learning and heuristic image post-processing. Appl. Intell. 2019 , 49 , 2793–2806. [ Google Scholar ] [ CrossRef ]
• Yong, B.; Wang, C.; Shen, J.; Li, F.; Yin, H.; Zhou, R. Automatic ventricular nuclear magnetic resonance image processing with deep learning. Multimed. Tools Appl. 2021 , 80 , 34103–34119. [ Google Scholar ] [ CrossRef ]
• Freeman, W.; Jones, T.; Pasztor, E. Example-based super-resolution. IEEE Comput. Graph. Appl. 2002 , 22 , 56–65. [ Google Scholar ] [ CrossRef ]
• Rodellar, J.; Alférez, S.; Acevedo, A.; Molina, A.; Merino, A. Image processing and machine learning in the morphological analysis of blood cells. Int. J. Lab. Hematol. 2018 , 40 , 46–53. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Kasinathan, T.; Uyyala, S.R. Machine learning ensemble with image processing for pest identification and classification in field crops. Neural Comput. Appl. 2021 , 33 , 7491–7504. [ Google Scholar ] [ CrossRef ]
• Yadav, P.; Gupta, N.; Sharma, P.K. A comprehensive study towards high-level approaches for weapon detection using classical machine learning and deep learning methods. Expert Syst. Appl. 2023 , 212 , 118698. [ Google Scholar ] [ CrossRef ]
• Suganyadevi, S.; Seethalakshmi, V.; Balasamy, K. Reinforcement learning coupled with finite element modeling for facial motion learning. Int. J. Multimed. Inf. Retr. 2022 , 11 , 19–38. [ Google Scholar ] [ CrossRef ]
• Zeng, Y.; Guo, Y.; Li, J. Recognition and extraction of high-resolution satellite remote sensing image buildings based on deep learning. Neural Comput. Appl. 2022 , 34 , 2691–2706. [ Google Scholar ] [ CrossRef ]
• Pratap, A.; Sardana, N. Machine learning-based image processing in materials science and engineering: A review. Mater. Today Proc. 2022 , 62 , 7341–7347. [ Google Scholar ] [ CrossRef ]
• Mahesh, B. Machine Learning Algorithms—A Review. Int. J. Sci. Res. 2020 , 9 , 1–6. [ Google Scholar ] [ CrossRef ]
• Singh, D.P.; Kaushik, B. Machine learning concepts and its applications for prediction of diseases based on drug behaviour: An extensive review. Chemom. Intell. Lab. Syst. 2022 , 229 , 104637. [ Google Scholar ] [ CrossRef ]
• Lillicrap, T.P.; Hunt, J.J.; Pritzel, A.; Heess, N.; Erez, T.; Tassa, Y.; Silver, D.; Wierstra, D. Continuous control with deep reinforcement learning. In Proceedings of the 4th International Conference on Learning Representations 2016, San Juan, Puerto Rico, 2–4 May 2016. [ Google Scholar ] [ CrossRef ]
• Dworschak, F.; Dietze, S.; Wittmann, M.; Schleich, B.; Wartzack, S. Reinforcement Learning for Engineering Design Automation. Adv. Eng. Inform. 2022 , 52 , 101612. [ Google Scholar ] [ CrossRef ]
• Khan, T.; Tian, W.; Zhou, G.; Ilager, S.; Gong, M.; Buyya, R. Machine learning (ML)-centric resource management in cloud computing: A review and future directions. J. Netw. Comput. Appl. 2022 , 204 , 103405. [ Google Scholar ] [ CrossRef ]
• Botvinick, M.; Ritter, S.; Wang, J.X.; Kurth-Nelson, Z.; Blundell, C.; Hassabis, D. Reinforcement Learning, Fast and Slow. Trends Cogn. Sci. 2019 , 23 , 408–422. [ Google Scholar ] [ CrossRef ]
• Moravčík, M.; Schmid, M.; Burch, N.; Lisý, V.; Morrill, D.; Bard, N.; Davis, T.; Waugh, K.; Johanson, M.; Bowling, M. DeepStack: Expert-level artificial intelligence in heads-up no-limit poker. Science 2017 , 356 , 508–513. [ Google Scholar ] [ CrossRef ]
• ElDahshan, K.A.; Farouk, H.; Mofreh, E. Deep Reinforcement Learning based Video Games: A Review. In Proceedings of the 2nd International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC), Cairo, Egypt, 8–9 May 2022. [ Google Scholar ] [ CrossRef ]
• Huawei Technologies Co., Ltd. Overview of Deep Learning. In Artificial Intelligence Technology ; Springer: Singapore, 2023; Chapter 1–4; pp. 87–122. [ Google Scholar ] [ CrossRef ]
• Le, N.; Rathour, V.S.; Yamazaki, K.; Luu, K.; Savvides, M. Deep reinforcement learning in computer vision: A comprehensive survey. Artif. Intell. Rev. 2022 , 55 , 2733–2819. [ Google Scholar ] [ CrossRef ]
• Melanthota, S.K.; Gopal, D.; Chakrabarti, S.; Kashyap, A.A.; Radhakrishnan, R.; Mazumder, N. Deep learning-based image processing in optical microscopy. Biophys. Rev. 2022 , 14 , 463–481. [ Google Scholar ] [ CrossRef ]
• Winovich, N.; Ramani, K.; Lin, G. ConvPDE-UQ: Convolutional neural networks with quantified uncertainty for heterogeneous elliptic partial differential equations on varied domains. J. Comput. Phys. 2019 , 394 , 263–279. [ Google Scholar ] [ CrossRef ]
• Pham, H.; Warin, X.; Germain, M. Neural networks-based backward scheme for fully nonlinear PDEs. SN Partial. Differ. Equ. Appl. 2021 , 2 , 16. [ Google Scholar ] [ CrossRef ]
• Wei, X.; Jiang, S.; Li, Y.; Li, C.; Jia, L.; Li, Y. Defect Detection of Pantograph Slide Based on Deep Learning and Image Processing Technology. IEEE Trans. Intell. Transp. Syst. 2020 , 21 , 947–958. [ Google Scholar ] [ CrossRef ]
• E, W.; Yu, B. The deep ritz method: A deep learning based numerical algorithm for solving variational problems. Commun. Math. Stat. 2018 , 6 , 1–12. [ Google Scholar ] [ CrossRef ]
• Yamashita, R.; Nishio, M.; Do, R.K.G.; Togashi, K. Convolutional neural networks: An overview and application in radiology. Insights Imaging 2018 , 9 , 611–629. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Archarya, U.; Oh, S.; Hagiwara, Y.; Tan, J.; Adam, M.; Gertych, A.; Tan, R. A deep convolutional neural network model to classify heartbeats. Comput. Biol. Med. 2021 , 89 , 389–396. [ Google Scholar ] [ CrossRef ]
• Ha, V.K.; Ren, J.C.; Xu, X.Y.; Zhao, S.; Xie, G.; Masero, V.; Hussain, A. Deep Learning Based Single Image Super-resolution: A Survey. Int. J. Autom. Comput. 2019 , 16 , 413–426. [ Google Scholar ] [ CrossRef ]
• Jeong, C.Y.; Yang, H.S.; Moon, K. Fast horizon detection in maritime images using region-of-interest. Int. J. Distrib. Sens. Netw. 2018 , 14 , 1550147718790753. [ Google Scholar ] [ CrossRef ]
• Olmos, R.; Tabik, S.; Lamas, A.; Pérez-Hernández, F.; Herrera, F. A binocular image fusion approach for minimizing false positives in handgun detection with deep learning. Inf. Fusion 2019 , 49 , 271–280. [ Google Scholar ] [ CrossRef ]
• Zhao, X.; Wu, Y.; Tian, J.; Zhang, H. Single Image Super-Resolution via Blind Blurring Estimation and Dictionary Learning. Neurocomputing 2016 , 212 , 3–11. [ Google Scholar ] [ CrossRef ]
• Qi, C.; Song, C.; Xiao, F.; Song, S. Generalization ability of hybrid electric vehicle energy management strategy based on reinforcement learning method. Energy 2022 , 250 , 123826. [ Google Scholar ] [ CrossRef ]
• Ritto, T.; Beregi, S.; Barton, D. Reinforcement learning and approximate Bayesian computation for model selection and parameter calibration applied to a nonlinear dynamical system. Mech. Syst. Signal Process. 2022 , 181 , 109485. [ Google Scholar ] [ CrossRef ]
• Hwang, R.; Lee, H.; Hwang, H.J. Option compatible reward inverse reinforcement learning. Pattern Recognit. Lett. 2022 , 154 , 83–89. [ Google Scholar ] [ CrossRef ]
• Ladosz, P.; Weng, L.; Kim, M.; Oh, H. Exploration in deep reinforcement learning: A survey. Inf. Fusion 2022 , 85 , 1–22. [ Google Scholar ] [ CrossRef ]
• Khayyat, M.M.; Elrefaei, L.A. Deep reinforcement learning approach for manuscripts image classification and retrieval. Multimed. Tools Appl. 2022 , 81 , 15395–15417. [ Google Scholar ] [ CrossRef ]
• Nguyen, D.P.; Ho Ba Tho, M.C.; Dao, T.T. A review on deep learning in medical image analysis. Comput. Methods Programs Biomed. 2022 , 221 , 106904. [ Google Scholar ] [ CrossRef ]
• Laskin, M.; Lee, K.; Stooke, A.; Pinto, L.; Abbeel, P.; Srinivas, A. Reinforcement Learning with Augmented Data. In Proceedings of the 34th Conference on Neural Information Processing Systems 2020, Vancouver, BC, Canada, 6–12 December 2020; pp. 19884–19895. [ Google Scholar ]
• Li, H.; Xu, H. Deep reinforcement learning for robust emotional classification in facial expression recognition. Knowl.-Based Syst. 2020 , 204 , 106172. [ Google Scholar ] [ CrossRef ]
• Gomes, G.; Vidal, C.A.; Cavalcante-Neto, J.B.; Nogueira, Y.L. A modeling environment for reinforcement learning in games. Entertain. Comput. 2022 , 43 , 100516. [ Google Scholar ] [ CrossRef ]
• Georgeon, O.L.; Casado, R.C.; Matignon, L.A. Modeling Biological Agents beyond the Reinforcement-learning Paradigm. Procedia Comput. Sci. 2015 , 71 , 17–22. [ Google Scholar ] [ CrossRef ]
• Yin, S.; Liu, H. Wind power prediction based on outlier correction, ensemble reinforcement learning, and residual correction. Energy 2022 , 250 , 123857. [ Google Scholar ] [ CrossRef ]
• Badia, A.P.; Piot, B.; Kapturowski, S.; Sprechmann, P.; Vitvitskyi, A.; Guo, D.; Blundell, C. Agent57: Outperforming the Atari Human Benchmark. arXiv 2020 , arXiv:2003.13350. [ Google Scholar ] [ CrossRef ]
• Zong, K.; Luo, C. Reinforcement learning based framework for COVID-19 resource allocation. Comput. Ind. Eng. 2022 , 167 , 107960. [ Google Scholar ] [ CrossRef ]
• Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A.A.; Veness, J.; Bellemare, M.G.; Graves, A.; Riedmiller, M.; Fidjeland, A.K.; Ostrovski, G.; et al. Human-level control through deep reinforcement learning. Nature 2015 , 518 , 529–533. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Ren, J.; Guan, F.; Li, X.; Cao, J.; Li, X. Optimization for image stereo-matching using deep reinforcement learning in rule constraints and parallax estimation. Neural Comput. Appl. 2023 , 1–11. [ Google Scholar ] [ CrossRef ]
• Morales, E.F.; Murrieta-Cid, R.; Becerra, I.; Esquivel-Basaldua, M.A. A survey on deep learning and deep reinforcement learning in robotics with a tutorial on deep reinforcement learning. Intell. Serv. Robot. 2021 , 14 , 773–805. [ Google Scholar ] [ CrossRef ]
• Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet Classification with Deep Convolutional Neural Networks. Commun. ACM 2017 , 60 , 84–90. [ Google Scholar ] [ CrossRef ]
• Krichen, M. Convolutional Neural Networks: A Survey. Computers 2023 , 12 , 151. [ Google Scholar ] [ CrossRef ]
• Song, D.; Kim, T.; Lee, Y.; Kim, J. Image-Based Artificial Intelligence Technology for Diagnosing Middle Ear Diseases: A Systematic Review. J. Clin. Med. 2023 , 12 , 5831. [ Google Scholar ] [ CrossRef ]
• Muñoz-Saavedra, L.; Escobar-Linero, E.; Civit-Masot, J.; Luna-Perejón, F.; Civit, A.; Domínguez-Morales, M. A Robust Ensemble of Convolutional Neural Networks for the Detection of Monkeypox Disease from Skin Images. Sensors 2023 , 23 , 7134. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Wang, Y.; Hargreaves, C.A. A Review Study of the Deep Learning Techniques used for the Classification of Chest Radiological Images for COVID-19 Diagnosis. Int. J. Inf. Manag. Data Insights 2022 , 2 , 100100. [ Google Scholar ] [ CrossRef ]
• Teng, Y.; Pan, D.; Zhao, W. Application of deep learning ultrasound imaging in monitoring bone healing after fracture surgery. J. Radiat. Res. Appl. Sci. 2023 , 16 , 100493. [ Google Scholar ] [ CrossRef ]
• Zaghari, N.; Fathy, M.; Jameii, S.M.; Sabokrou, M.; Shahverdy, M. Improving the learning of self-driving vehicles based on real driving behavior using deep neural network techniques. J. Supercomput. 2021 , 77 , 3752–3794. [ Google Scholar ] [ CrossRef ]
• Farag, W. Cloning Safe Driving Behavior for Self-Driving Cars using Convolutional Neural Networks. Recent Patents Comput. Sci. 2019 , 11 , 120–127. [ Google Scholar ] [ CrossRef ]
• Agyemang, I.; Zhang, X.; Acheampong, D.; Adjei-Mensah, I.; Kusi, G.; Mawuli, B.C.; Agbley, B.L. Autonomous health assessment of civil infrastructure using deep learning and smart devices. Autom. Constr. 2022 , 141 , 104396. [ Google Scholar ] [ CrossRef ]
• Zhou, S.; Canchila, C.; Song, W. Deep learning-based crack segmentation for civil infrastructure: Data types, architectures, and benchmarked performance. Autom. Constr. 2023 , 146 , 104678. [ Google Scholar ] [ CrossRef ]
• Guerrieri, M.; Parla, G. Flexible and stone pavements distress detection and measurement by deep learning and low-cost detection devices. Eng. Fail. Anal. 2022 , 141 , 106714. [ Google Scholar ] [ CrossRef ]
• Hoang, N.; Nguyen, Q. A novel method for asphalt pavement crack classification based on image processing and machine learning. Eng. Comput. 2019 , 35 , 487–498. [ Google Scholar ] [ CrossRef ]
• Tabrizi, S.E.; Xiao, K.; Van Griensven Thé, J.; Saad, M.; Farghaly, H.; Yang, S.X.; Gharabaghi, B. Hourly road pavement surface temperature forecasting using deep learning models. J. Hydrol. 2021 , 603 , 126877. [ Google Scholar ] [ CrossRef ]
• Jardim, S.V.B. Sparse and Robust Signal Reconstruction. Theory Appl. Math. Comput. Sci. 2015 , 5 , 1–19. [ Google Scholar ]
• Jackulin, C.; Murugavalli, S. A comprehensive review on detection of plant disease using machine learning and deep learning approaches. Meas. Sens. 2022 , 24 , 100441. [ Google Scholar ] [ CrossRef ]
• Keceli, A.S.; Kaya, A.; Catal, C.; Tekinerdogan, B. Deep learning-based multi-task prediction system for plant disease and species detection. Ecol. Inform. 2022 , 69 , 101679. [ Google Scholar ] [ CrossRef ]
• Kotwal, J.; Kashyap, D.; Pathan, D. Agricultural plant diseases identification: From traditional approach to deep learning. Mater. Today Proc. 2023 , 80 , 344–356. [ Google Scholar ] [ CrossRef ]
• Naik, A.; Thaker, H.; Vyas, D. A survey on various image processing techniques and machine learning models to detect, quantify and classify foliar plant disease. Proc. Indian Natl. Sci. Acad. 2021 , 87 , 191–198. [ Google Scholar ] [ CrossRef ]
• Thaiyalnayaki, K.; Joseph, C. Classification of plant disease using SVM and deep learning. Mater. Today Proc. 2021 , 47 , 468–470. [ Google Scholar ] [ CrossRef ]
• Carnegie, A.J.; Eslick, H.; Barber, P.; Nagel, M.; Stone, C. Airborne multispectral imagery and deep learning for biosecurity surveillance of invasive forest pests in urban landscapes. Urban For. Urban Green. 2023 , 81 , 127859. [ Google Scholar ] [ CrossRef ]
• Hadipour-Rokni, R.; Askari Asli-Ardeh, E.; Jahanbakhshi, A.; Esmaili paeen-Afrakoti, I.; Sabzi, S. Intelligent detection of citrus fruit pests using machine vision system and convolutional neural network through transfer learning technique. Comput. Biol. Med. 2023 , 155 , 106611. [ Google Scholar ] [ CrossRef ]
• Agrawal, P.; Chaudhary, D.; Madaan, V.; Zabrovskiy, A.; Prodan, R.; Kimovski1, D.; Timmerer, C. Automated bank cheque verification using image processing and deep learning methods. Multimed. Tools Appl. 2021 , 80 , 5319–5350. [ Google Scholar ] [ CrossRef ]
• Gordo, A.; Almazán, J.; Revaud, J.; Larlus, D. Deep Image Retrieval: Learning Global Representations for Image Search. In Proceedings of the Computer Vision—ECCV 2016, Amsterdam, The Netherlands, 11–14 October 2016; Leibe, B., Matas, J., Sebe, N., Welling, M., Eds.; Springer International Publishing: Cham, Switzerland, 2016; pp. 241–257. [ Google Scholar ]
• Jardim, S.; António, J.; Mora, C.; Almeida, A. A Novel Trademark Image Retrieval System Based on Multi-Feature Extraction and Deep Networks. J. Imaging 2022 , 8 , 238. [ Google Scholar ] [ CrossRef ]
• Lin, K.; Yang, H.F.; Hsiao, J.H.; Chen, C.S. Deep learning of binary hash codes for fast image retrieval. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Boston, MA, USA, 7–12 June 2015; pp. 27–35. [ Google Scholar ] [ CrossRef ]
• Andriasyan, V.; Yakimovich, A.; Petkidis, A.; Georgi, F.; Georgi, R.; Puntener, D.; Greber, U. Microscopy deep learning predicts virus infections and reveals mechanics of lytic-infected cells. iScience 2021 , 24 , 102543. [ Google Scholar ] [ CrossRef ]
• Lüneburg, N.; Reiss, N.; Feldmann, C.; van der Meulen, P.; van de Steeg, M.; Schmidt, T.; Wendl, R.; Jansen, S. Photographic LVAD Driveline Wound Infection Recognition Using Deep Learning. In dHealth 2019—From eHealth to dHealth ; IOS Press: Amsterdam, The Netherlands, 2019; pp. 192–199. [ Google Scholar ] [ CrossRef ]
• Fink, O.; Wang, Q.; Svensén, M.; Dersin, P.; Lee, W.J.; Ducoffe, M. Potential, challenges and future directions for deep learning in prognostics and health management applications. Eng. Appl. Artif. Intell. 2020 , 92 , 103678. [ Google Scholar ] [ CrossRef ]
• Ahmed, I.; Ahmad, M.; Jeon, G. Social distance monitoring framework using deep learning architecture to control infection transmission of COVID-19 pandemic. Sustain. Cities Soc. 2021 , 69 , 102777. [ Google Scholar ] [ CrossRef ]
• Hussain, S.; Yu, Y.; Ayoub, M.; Khan, A.; Rehman, R.; Wahid, J.A.; Hou, W. IoT and Deep Learning Based Approach for Rapid Screening and Face Mask Detection for Infection Spread Control of COVID-19. Appl. Sci. 2021 , 11 , 3495. [ Google Scholar ] [ CrossRef ]
• Kaur, J.; Kaur, P. Outbreak COVID-19 in Medical Image Processing Using Deep Learning: A State-of-the-Art Review. Arch. Comput. Methods Eng. 2022 , 29 , 2351–2382. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Groen, A.M.; Kraan, R.; Amirkhan, S.F.; Daams, J.G.; Maas, M. A systematic review on the use of explainability in deep learning systems for computer aided diagnosis in radiology: Limited use of explainable AI? Int. J. Autom. Comput. 2022 , 157 , 110592. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Hao, D.; Li, Q.; Feng, Q.X.; Qi, L.; Liu, X.S.; Arefan, D.; Zhang, Y.D.; Wu, S. SurvivalCNN: A deep learning-based method for gastric cancer survival prediction using radiological imaging data and clinicopathological variables. Artif. Intell. Med. 2022 , 134 , 102424. [ Google Scholar ] [ CrossRef ]
• Cui, X.; Zheng, S.; Heuvelmans, M.A.; Du, Y.; Sidorenkov, G.; Fan, S.; Li, Y.; Xie, Y.; Zhu, Z.; Dorrius, M.D.; et al. Performance of a deep learning-based lung nodule detection system as an alternative reader in a Chinese lung cancer screening program. Eur. J. Radiol. 2022 , 146 , 110068. [ Google Scholar ] [ CrossRef ]
• Liu, L.; Li, C. Comparative study of deep learning models on the images of biopsy specimens for diagnosis of lung cancer treatment. J. Radiat. Res. Appl. Sci. 2023 , 16 , 100555. [ Google Scholar ] [ CrossRef ]
• Muniz, F.B.; de Freitas Oliveira Baffa, M.; Garcia, S.B.; Bachmann, L.; Felipe, J.C. Histopathological diagnosis of colon cancer using micro-FTIR hyperspectral imaging and deep learning. Comput. Methods Programs Biomed. 2023 , 231 , 107388. [ Google Scholar ] [ CrossRef ]
• Gomes, S.L.; de S. Rebouças, E.; Neto, E.C.; Papa, J.P.; de Albuquerque, V.H.C.; Filho, P.P.R.; Tavares, J.M.R.S. Embedded real-time speed limit sign recognition using image processing and machine learning techniques. Neural Comput. Appl. 2017 , 28 , 573–584. [ Google Scholar ] [ CrossRef ]
• Monga, V.; Li, Y.; Eldar, Y.C. Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing. IEEE Signal Process. Mag. 2021 , 38 , 18–44. [ Google Scholar ] [ CrossRef ]
• Zhang, L.; Cheng, L.; Li, H.; Gao, J.; Yu, C.; Domel, R.; Yang, Y.; Tang, S.; Liu, W.K. Hierarchical deep-learning neural networks: Finite elements and beyond. Comput. Mech. 2021 , 67 , 207–230. [ Google Scholar ] [ CrossRef ]
• Salahzadeh, Z.; Rezaei-Hachesu, P.; Gheibi, Y.; Aghamali, A.; Pakzad, H.; Foladlou, S.; Samad-Soltani, T. A mechatronics data collection, image processing, and deep learning platform for clinical posture analysis: A technical note. Phys. Eng. Sci. Med. 2021 , 44 , 901–910. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Singh, P.; Hrisheekesha, P.; Singh, V.K. CBIR-CNN: Content-Based Image Retrieval on Celebrity Data Using Deep Convolution Neural Network. Recent Adv. Comput. Sci. Commun. 2021 , 14 , 257–272. [ Google Scholar ] [ CrossRef ]
• Varga, D.; Szirányi, T. Fast content-based image retrieval using convolutional neural network and hash function. In Proceedings of the 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Budapest, Hungary, 9–12 October 2016; pp. 2636–2640. [ Google Scholar ] [ CrossRef ]
• Latif, A.; Rasheed, A.; Sajid, U.; Ahmed, J.; Ali, N.; Ratyal, N.I.; Zafar, B.; Dar, S.H.; Sajid, M.; Khalil, T. Content-Based Image Retrieval and Feature Extraction: A Comprehensive Review. Math. Probl. Eng. 2019 , 2019 , 9658350. [ Google Scholar ] [ CrossRef ]
• Rani, P.; Kotwal, S.; Manhas, J.; Sharma, V.; Sharma, S. Machine Learning and Deep Learning Based Computational Approaches in Automatic Microorganisms Image Recognition: Methodologies, Challenges, and Developments. Arch. Comput. Methods Eng. 2022 , 29 , 1801–1837. [ Google Scholar ] [ CrossRef ]
• Jardim, S.V.B.; Figueiredo, M.A.T. Automatic Analysis of Fetal Echographic Images. Proc. Port. Conf. Pattern Recognit. 2002 , 1 , 1–6. [ Google Scholar ]
• Jardim, S.V.B.; Figueiredo, M.A.T. Automatic contour estimation in fetal ultrasound images. In Proceedings of the 2003 International Conference on Image Processing 2003, Barcelona, Spain, 14–17 September 2003; Volum 1, pp. 1065–1068. [ Google Scholar ] [ CrossRef ]
• Devunooru, S.; Alsadoon, A.; Chandana, P.W.C.; Beg, A. Deep learning neural networks for medical image segmentation of brain tumours for diagnosis: A recent review and taxonomy. J. Ambient Intell. Humaniz. Comput. 2021 , 12 , 455–483. [ Google Scholar ] [ CrossRef ]
• Anaya-Isaza, A.; Mera-Jiménez, L.; Verdugo-Alejo, L.; Sarasti, L. Optimizing MRI-based brain tumor classification and detection using AI: A comparative analysis of neural networks, transfer learning, data augmentation, and the cross-transformer network. Eur. J. Radiol. Open 2023 , 10 , 100484. [ Google Scholar ] [ CrossRef ]
• Cao, Y.; Kunaprayoon, D.; Xu, J.; Ren, L. AI-assisted clinical decision making (CDM) for dose prescription in radiosurgery of brain metastases using three-path three-dimensional CNN. Clin. Transl. Radiat. Oncol. 2023 , 39 , 100565. [ Google Scholar ] [ CrossRef ]
• Chakrabarty, N.; Mahajan, A.; Patil, V.; Noronha, V.; Prabhash, K. Imaging of brain metastasis in non-small-cell lung cancer: Indications, protocols, diagnosis, post-therapy imaging, and implications regarding management. Clin. Radiol. 2023 , 78 , 175–186. [ Google Scholar ] [ CrossRef ]
• Mehrotra, R.; Ansari, M.; Agrawal, R.; Anand, R. A Transfer Learning approach for AI-based classification of brain tumors. Mach. Learn. Appl. 2020 , 2 , 100003. [ Google Scholar ] [ CrossRef ]
• Drai, M.; Testud, B.; Brun, G.; Hak, J.F.; Scavarda, D.; Girard, N.; Stellmann, J.P. Borrowing strength from adults: Transferability of AI algorithms for paediatric brain and tumour segmentation. Eur. J. Radiol. 2022 , 151 , 110291. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Ranjbarzadeh, R.; Caputo, A.; Tirkolaee, E.B.; Jafarzadeh Ghoushchi, S.; Bendechache, M. Brain tumor segmentation of MRI images: A comprehensive review on the application of artificial intelligence tools. Comput. Biol. Med. 2023 , 152 , 106405. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Yedder, H.B.; Cardoen, B.; Hamarneh, G. Deep learning for biomedical image reconstruction: A survey. Artif. Intell. Rev. 2021 , 54 , 215–251. [ Google Scholar ] [ CrossRef ]
• Manuel Davila Delgado, J.; Oyedele, L. Robotics in construction: A critical review of the reinforcement learning and imitation learning paradigms. Adv. Eng. Inform. 2022 , 54 , 101787. [ Google Scholar ] [ CrossRef ]
• Íñigo Elguea-Aguinaco; Serrano-Muñoz, A.; Chrysostomou, D.; Inziarte-Hidalgo, I.; Bøgh, S.; Arana-Arexolaleiba, N. A review on reinforcement learning for contact-rich robotic manipulation tasks. Robot. Comput.-Integr. Manuf. 2023 , 81 , 102517. [ Google Scholar ] [ CrossRef ]
• Ahn, K.H.; Na, M.; Song, J.B. Robotic assembly strategy via reinforcement learning based on force and visual information. Robot. Auton. Syst. 2023 , 164 , 104399. [ Google Scholar ] [ CrossRef ]
• Jafari, M.; Xu, H.; Carrillo, L.R.G. A biologically-inspired reinforcement learning based intelligent distributed flocking control for Multi-Agent Systems in presence of uncertain system and dynamic environment. IFAC J. Syst. Control 2020 , 13 , 100096. [ Google Scholar ] [ CrossRef ]
• Wang, X.; Liu, S.; Yu, Y.; Yue, S.; Liu, Y.; Zhang, F.; Lin, Y. Modeling collective motion for fish schooling via multi-agent reinforcement learning. Ecol. Model. 2023 , 477 , 110259. [ Google Scholar ] [ CrossRef ]
• Jain, D.K.; Dutta, A.K.; Verdú, E.; Alsubai, S.; Sait, A.R.W. An automated hyperparameter tuned deep learning model enabled facial emotion recognition for autonomous vehicle drivers. Image Vis. Comput. 2023 , 133 , 104659. [ Google Scholar ] [ CrossRef ]
• Silver, D.; Hubert, T.; Schrittwieser, J.; Antonoglou, I.; Lai, M.; Guez, A.; Lanctot, M.; Sifre, L.; Kumaran, D.; Graepel, T.; et al. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 2018 , 362 , 1140–1144. [ Google Scholar ] [ CrossRef ]
• Ueda, M. Memory-two strategies forming symmetric mutual reinforcement learning equilibrium in repeated prisoners’ dilemma game. Appl. Math. Comput. 2023 , 444 , 127819. [ Google Scholar ] [ CrossRef ]
• Wang, X.; Liu, F.; Ma, X. Mixed distortion image enhancement method based on joint of deep residuals learning and reinforcement learning. Signal Image Video Process. 2021 , 15 , 995–1002. [ Google Scholar ] [ CrossRef ]
• Dai, Y.; Wang, G.; Muhammad, K.; Liu, S. A closed-loop healthcare processing approach based on deep reinforcement learning. Multimed. Tools Appl. 2022 , 81 , 3107–3129. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Valente, J.; António, J.; Mora, C.; Jardim, S. Developments in Image Processing Using Deep Learning and Reinforcement Learning. J. Imaging 2023 , 9 , 207. https://doi.org/10.3390/jimaging9100207

Valente J, António J, Mora C, Jardim S. Developments in Image Processing Using Deep Learning and Reinforcement Learning. Journal of Imaging . 2023; 9(10):207. https://doi.org/10.3390/jimaging9100207

Valente, Jorge, João António, Carlos Mora, and Sandra Jardim. 2023. "Developments in Image Processing Using Deep Learning and Reinforcement Learning" Journal of Imaging 9, no. 10: 207. https://doi.org/10.3390/jimaging9100207

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Subscribe to the PwC Newsletter

Join the community, search results, scikit-image: image processing in python.

1 code implementation • 23 Jul 2014

scikit-image is an image processing library that implements algorithms and utilities for use in research, education and industry applications.

Loss Functions for Neural Networks for Image Processing

2 code implementations • 28 Nov 2015

Neural networks are becoming central in several areas of computer vision and image processing and different architectures have been proposed to solve specific problems.

Image Processing GNN: Breaking Rigidity in Super-Resolution

1 code implementation • CVPR 2024

Alternatively we leverage the flexibility of graphs and propose the Image Processing GNN (IPG) model to break the rigidity that dominates previous SR methods.

Picasso: A Modular Framework for Visualizing the Learning Process of Neural Network Image Classifiers

1 code implementation • 16 May 2017

Picasso is a free open-source (Eclipse Public License) web application written in Python for rendering standard visualizations useful for analyzing convolutional neural networks.

MAXIM: Multi-Axis MLP for Image Processing

1 code implementation • CVPR 2022

In this work, we present a multi-axis MLP based architecture called MAXIM, that can serve as an efficient and flexible general-purpose vision backbone for image processing tasks.

Instruct-IPT: All-in-One Image Processing Transformer via Weight Modulation

1 code implementation • 30 Jun 2024

We have conducted experiments on Instruct-IPT to demonstrate the effectiveness of our method on manifold tasks, and we have effectively extended our method to diffusion denoisers as well.

1 code implementation • 2 Jul 2024

Fast Image Processing with Fully-Convolutional Networks

2 code implementations • ICCV 2017

Our approach uses a fully-convolutional network that is trained on input-output pairs that demonstrate the operator's action.

Simple Image Signal Processing using Global Context Guidance

1 code implementation • 17 Apr 2024

First, we propose a novel module that can be integrated into any neural ISP to capture the global context information from the full RAW images.

Pre-Trained Image Processing Transformer

6 code implementations • CVPR 2021

To maximally excavate the capability of transformer, we present to utilize the well-known ImageNet benchmark for generating a large amount of corrupted image pairs.

Recent trends in image processing and pattern recognition

• Guest Editorial
• Published: 27 October 2020
• Volume 79 , pages 34697–34699, ( 2020 )

Cite this article

• K. C. Santosh 1 &
• Sameer K. Antani 2  

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Avoid common mistakes on your manuscript.

The Call for Papers of the special issue was initially sent out to the participants of the 2018 conference (2nd International Conference on Recent Trends in Image Processing and Pattern Recognition). To attract high quality research articles, we also accepted papers for review from outside the conference event. Of 123 submissions, 22 papers were accepted. The acceptance rate, therefore, is just under 18%.

In “Multilevel Polygonal Descriptor Matching Defined by Combining Discrete Lines and Force Histogram Concepts,” authors presented a new method to describe shapes from a set of polygonal curves using a relational descriptor. In their study, relational descriptor is the main idea of the paper.

In “An Asymmetric Cryptosystem based on the Random Weighted Singular Value Decomposition and Fractional Hartley Domain,” authors proposed an encryption system for double random phase encoding based on random weighted singular value decomposition and fractional Hartley transform domain. Authors claimed that the proposed cryptosystem is efficiently compared with singular value decomposition and truncated singular value decomposition.

In “Classification of Complex Environments using Pixel Level Fusion of Satellite Data,” authors analyzed composite land features by fusing two original hyperspectral and multispectral datasets. In their study, the fusion image technique was found to be superior to the single original image.

In “Image Dehazing using Window-based Integrated Means Filter,” authors reported that the proposed technique outperforms the state-of-the-arts in single image dehazing approaches.

In “Research on Fundus Image Registration and Fusion Method based on Nonsubsampled Contourlet and Adaptive Pulse Coupled Neural Network,” authors presented a registration and fusion method of fluorescein fundus angiography image and color fundus image that combines Nonsubsampled Contourlet (NSCT) and adaptive Pulse Coupled Neural Network (PCNN). Authors claimed that the image fusion provides an effective reference for the clinical diagnosis of fundus diseases.

In “Super Resolution of Single Depth Image based on Multi-dictionary Learning with Edge Feature Regularization,” authors focused on super resolution based on multi-dictionary learning with edge regularization model. With this, the reconstructed depth images were found to be superior with respect to the state-of-art methods.

In “A Universal Foreground Segmentation Technique using Deep Neural Network,” authors presented an idea of optical-flow details to make use of temporal information in deep neural network.

In “Removal of ‘Salt & Pepper’ Noise from Color Images using Adaptive Fuzzy Technique based on Histogram Estimation,” authors focused on the use of processing window that is based on local noise densities using fuzzy based criterion.

In “Image Retrieval by Integrating Global Correlation of Color and Intensity Histograms with Local Texture Features,” authors integrated color, intensity histograms with local state-of-the-art texture features to perform content-based image retrieval.

In “Image-based Features for Speech Signal Classification,” authors analyzed speech signal with the help of image features. Authors used the idea of computer-based image features for speech analysis.

In “ Ensembling Handcrafted Features with Deep Features: An Analytical Study for Classification of Routine Colon Cancer Histopathological Nuclei Images,” authors studied deep learning models to analyze medical histopathology: classification, segmentation, and detection.

In “Non-destructive and Cost-effective 3D Plant Growth Monitoring System in Outdoor Conditions,” authors monitored plant growth precisely with the use of mobile phone.

In “Fusion based Feature Reinforcement Component for Remote Sensing Image Object Detection,” authors employed reinforcement component (FB-FRC) to improve image classification, where two fusion strategies are proposed: a hard-fusion strategy through artificially set rules; and a soft fusion strategy by learning the fusion parameters.

In “An Improved Cuckoo Search Algorithm for Multi-level Gray-scale Image Thresholding,” authors employed computationally efficient cuckoo search algorithm.

In “Image Fuzzy Enhancement Algorithm based on Contourlet Transform Domain,” authors focused on enhancing globally the texture and edge of the image.

In “Pixel Encoding for Unconstrained Face Detection,” authors employed handcrafted and visual features to detect human faces. Authors claimed an improvement when handcrafted and visual features are combined.

In “Data Augmentation for Handwritten Digit Recognition using Generative Adversarial Networks (GAN),” authors focused on the technique that does not require prior knowledge of the possible variabilities that exist across examples to create novel artificial examples.

In “Akin-based Orthogonal Space (AOS): A Subspace Learning Method for Face Recognition,” authors reported the use of subspace learning method is efficient for human face recognition.

In “A Kernel Machine for Hidden Object-Ranking Problems (HORPs),” authors proposed a kernel machine that allows retaining item-related ordinal information while avoiding emphasizing class-related information.

In “Verification of Genuine and Forged Offline Signatures using Siamese Neural Network (SNN),” authors reported one shot learning in SNN for signature verification.

In “Super-Resolution Quality Criterion (SRQC): A Super-Resolution Image Quality Assessment Metric,” authors reported the importance of SRQC in assessing image quality. In their experiments, authors found that the SRQC is more competent in modeling the features from curvelet transform that quantifies the quality score of the super-resolved image and it outperforms the formerly reported image quality assessment metrics.

In “Ensemble based Technique for the Assessment of Fetal Health using Cardiotocograph – A Case Study with Standard Feature Reduction Techniques,” authors reported the use of state-of-the-art feature reduction techniques to assess fetal health using cardiotocograph.

Within the scope of image processing pattern recognition, this special issue includes multiple applications domains, such as satellite imaging, biometrics, speech processing, medical imaging, and healthcare.

Author information

Authors and affiliations.

University of South Dakota, Vermillion, SD, 57069, USA

K. C. Santosh

U.S. National Library of Medicine, NIH, Bethesda, MD, 20894, USA

Sameer K. Antani

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to K. C. Santosh .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Santosh, K.C., Antani, S.K. Recent trends in image processing and pattern recognition. Multimed Tools Appl 79 , 34697–34699 (2020). https://doi.org/10.1007/s11042-020-10093-3

Download citation

Published : 27 October 2020

Issue Date : December 2020

DOI : https://doi.org/10.1007/s11042-020-10093-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research

Showing papers on "Image processing published in 2023"

Ieee transactions on image processing.

123  citations

Image Processing On Line

13  citations

Detection and counting of banana bunches by integrating deep learning and classic image-processing algorithms

7  citations

Impact of Artificial Intelligence on COVID-19 Pandemic: A Survey of Image Processing, Tracking of Disease, Prediction of Outcomes, and Computational Medicine

6  citations

Nuts&bolts: YOLO-v5 and image processing based component identification system

5  citations

Fragile Watermarking for Tamper Localization and Self-Recovery Based on AMBTC and VQ

4  citations

SNIS: A Signal Noise Separation-Based Network for Post-Processed Image Forgery Detection

A novel medical image enhancement algorithm for breast cancer detection on mammography images using machine learning, trends in digital image processing of isolated microalgae by incorporating classification algorithm., mri-based brain tumor image classification using cnn.

3  citations

Determining Phase Separation Dynamics with an Automated Image Processing Algorithm

An efficient automated vehicle license plate recognition system under image processing, deep learning-based real-world object detection and improved anomaly detection for surveillance videos, noncontact sensing techniques for ai-aided structural health monitoring: a systematic review, automatic seat identification system in smart transport using iot and image processing, practical application of digital image processing in measuring concrete crack widths in field studies, identification of stripe rust and leaf rust on different wheat varieties based on image processing technology, novel image processing method inspired by wavelet transform., resnet and its application to medical image processing: research progress and challenges., application of improved support vector machine for pulmonary syndrome exposure with computer vision measures, saliency map in image visual quality assessment and processing, development of static and dynamic colorimetric analysis techniques using image sensors and novel image processing software for chemical, biological and medical applications, the weight of hyperion and prisma hyperspectral sensor characteristics on image capability to retrieve urban surface materials in the city of venice, integrated diffusion image operator (idio): a pipeline for automated configuration and processing of diffusion mri data, application of swarm intelligence optimization algorithms in image processing: a comprehensive review of analysis, synthesis, and optimization, development of complete image processing system including image filtering, image compression & image security, techniques of deep learning and image processing in plant leaf disease detection: a review, traditional vs. automated computer image analysis—a comparative assessment of use for analysis of digital sem images of high-temperature ceramic material, digital image watermarking using deep learning, a hybrid deep neural network‐based automated diagnosis system using x‐ray images and clinical findings.

• Frontiers in Robotics and AI
• Robot Vision and Artificial Perception
• Research Topics

Current Trends in Image Processing and Pattern Recognition

Total Downloads

Total Views and Downloads

About this Research Topic

Special thanks to Dr. Mahendra Dashrath Shirsat , who was instrumental in the organization of this Research Topic. Recent technological advancements in computer science have opened diversified opportunities to researchers working in ...

Keywords : Computer Vision, Pattern Recognition, Artificial Intelligence

Important Note : All contributions to this Research Topic must be within the scope of the section and journal to which they are submitted, as defined in their mission statements. Frontiers reserves the right to guide an out-of-scope manuscript to a more suitable section or journal at any stage of peer review.

Topic Editors

Topic coordinators, recent articles, submission deadlines.

Submission closed.

Participating Journals

Total views.

• Demographics

No records found

total views article views downloads topic views

Top countries

Top referring sites, about frontiers research topics.

With their unique mixes of varied contributions from Original Research to Review Articles, Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author.

• Advanced Search

Coarse-to-fine underwater image enhancement with lightweight CNN and attention-based refinement

New citation alert added.

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

Information & Contributors

Bibliometrics & citations, view options, graphical abstract, recommendations, underwater image enhancement by color correction and color constancy via retinex for detail preserving.

In underwater, light attenuation causes non-uniform illumination that degrades underwater image. To enhance the degraded image, we propose an underwater image enhancement method that includes color correction, color constancy, multi-...

Display Omitted

• The formulation of color correction compensates the red and blue channels by masking.

Underwater image enhancement by combining color constancy and dehazing based on depth estimation

• Proposed an underwater image enhancement method using color constancy and dehazing.

The physical properties which are present in the underwater environment affects the images captured by the visual sensors. As a consequence of these properties, the captured image includes non-uniform illumination. This non-uniform ...

Deep retinex decomposition network for underwater image enhancement

This paper introduces a deep retinex decomposition network for underwater image enhancement to conquer the color imbalance, blurring, low contrast, etc. Specifically, we first designed a novel convolutional neural network to estimate ...

• This is Retinex-Net’s first attempt in the field of underwater image enhancement.

Information

Published in.

Academic Press, Inc.

United States

Publication History

Author tags.

• Underwater image enhancement
• Deep learning
• Image processing
• Convolutional neural network
• Attention module
• Modified histogram equalization
• Research-article

Contributors

Other metrics, bibliometrics, article metrics.

• 0 Total Citations
• 0 Total Downloads
• Downloads (Last 12 months) 0
• Downloads (Last 6 weeks) 0

View options

Login options.

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Share this publication link.

Copying failed.

Share on social media

Affiliations, export citations.

• Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
• Download citation
• Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

• All Research Labs
• 3D Deep Learning
• Applied Research
• Autonomous Vehicles
• Deep Imagination
• New and Featured
• AI Art Gallery
• AI & Machine Learning
• Computer Vision
• Academic Collaborations
• Government Collaborations
• Graduate Fellowship
• Internships
• Research Openings
• Research Scientists
• Meet the Team
• Publications

DiffiT: Diffusion Vision Transformers for Image Generation

Diffusion models with their powerful expressivity and high sample quality have achieved State-Of-The-Art (SOTA) performance in the generative domain. The pioneering Vision Transformer (ViT) has also demonstrated strong modeling capabilities and scalability, especially for recognition tasks. In this paper, we study the effectiveness of ViTs in diffusion-based generative learning and propose a new model denoted as Diffusion Vision Transformers (DiffiT). Specifically, we propose a methodology for finegrained control of the denoising process and introduce the Time-dependant Multihead Self Attention (TMSA) mechanism. DiffiT is surprisingly effective in generating high-fidelity images with significantly better parameter efficiency. We also propose latent and image space DiffiT models and show SOTA performance on a variety of class-conditional and unconditional synthesis tasks at different resolutions. The Latent DiffiT model achieves a new SOTA FID score of 1.73 on ImageNet256 dataset while having 19.85%, 16.88% less parameters than other Transformer-based diffusion models such as MDT and DiT, respectively.

Publication Date

Published in, research area, external links.

IEEE Account

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Help | Advanced Search

Computer Science > Computer Vision and Pattern Recognition

Title: transformer-based image and video inpainting: current challenges and future directions.

Abstract: Image inpainting is currently a hot topic within the field of computer vision. It offers a viable solution for various applications, including photographic restoration, video editing, and medical imaging. Deep learning advancements, notably convolutional neural networks (CNNs) and generative adversarial networks (GANs), have significantly enhanced the inpainting task with an improved capability to fill missing or damaged regions in an image or video through the incorporation of contextually appropriate details. These advancements have improved other aspects, including efficiency, information preservation, and achieving both realistic textures and structures. Recently, visual transformers have been exploited and offer some improvements to image or video inpainting. The advent of transformer-based architectures, which were initially designed for natural language processing, has also been integrated into computer vision tasks. These methods utilize self-attention mechanisms that excel in capturing long-range dependencies within data; therefore, they are particularly effective for tasks requiring a comprehensive understanding of the global context of an image or video. In this paper, we provide a comprehensive review of the current image or video inpainting approaches, with a specific focus on transformer-based techniques, with the goal to highlight the significant improvements and provide a guideline for new researchers in the field of image or video inpainting using visual transformers. We categorized the transformer-based techniques by their architectural configurations, types of damage, and performance metrics. Furthermore, we present an organized synthesis of the current challenges, and suggest directions for future research in the field of image or video inpainting.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Title: Application of digital twin virtual design and BIM technology in intelligent building image processing

Authors : Fengyi Han

Addresses : Bim School of Technology and Industry, Changchun Institute of Technology, Changchun 130012, Jilin, China

Abstract : Intelligent digital virtual technology has become an indispensable part of modern construction, but there are also some problems in its practical application. Therefore, it is necessary to strengthen the design of intelligent building image processing systems from many aspects. Starting from image digital processing methods, this paper studies the digital twin virtual design scene construction method and related algorithms, converts the original image into a colour digital image through a greyscale algorithm, and then combines morphological knowledge and feature point extraction methods to complete the construction of a three-dimensional virtual environment. Finally, through the comparison of traditional image processing effects with smart building images based on digital twins and BIM technology, the results show that the optimised image processing results have higher clarity, sharper contrast, and a sensitivity increased by 5.84%, presenting better visual effects and solving the risk of misjudgement caused by inaccurate image recognition.

Keywords : digital twins; building information modelling; BIM; intelligent buildings; electronic imaging.

DOI : 10.1504/IJDMB.2024.139481

International Journal of Data Mining and Bioinformatics, 2024 Vol.28 No.3/4, pp.257 - 271

Received: 11 Mar 2023 Accepted: 08 Sep 2023 Published online: 02 Jul 2024 *

Keep up-to-date

• Our Newsletter ( subscribe for free )
• New issue alerts
• Inderscience is a member of publishing organisations including:

Main Navigation

• Contact NeurIPS
• Code of Ethics
• Code of Conduct
• Create Profile
• Journal To Conference Track
• Diversity & Inclusion
• Proceedings
• Future Meetings
• Exhibitor Information
• Privacy Policy

NeurIPS 2024, the Thirty-eighth Annual Conference on Neural Information Processing Systems, will be held at the Vancouver Convention Center

Monday Dec 9 through Sunday Dec 15. Monday is an industry expo.

Registration

Pricing » Registration 2024 Registration Cancellation Policy » Certificate of Attendance

Our Hotel Reservation page is currently under construction and will be released shortly. NeurIPS has contracted Hotel guest rooms for the Conference at group pricing, requiring reservations only through this page. Please do not make room reservations through any other channel, as it only impedes us from putting on the best Conference for you. We thank you for your assistance in helping us protect the NeurIPS conference.

Announcements

• The call for High School Projects has been released
• The Call For Papers has been released
• See the Visa Information page for changes to the visa process for 2024.

Latest NeurIPS Blog Entries [ All Entries ]

Important Dates

If you have questions about supporting the conference, please contact us .

View NeurIPS 2024 exhibitors » Become an 2024 Exhibitor Exhibitor Info »

Organizing Committee

General chair, program chair, workshop chair, workshop chair assistant, tutorial chair, competition chair, data and benchmark chair, affinity chair, diversity, inclusion and accessibility chair, ethics review chair, communication chair, social chair, journal chair, creative ai chair, workflow manager, logistics and it, mission statement.

The Neural Information Processing Systems Foundation is a non-profit corporation whose purpose is to foster the exchange of research advances in Artificial Intelligence and Machine Learning, principally by hosting an annual interdisciplinary academic conference with the highest ethical standards for a diverse and inclusive community.

About the Conference

The conference was founded in 1987 and is now a multi-track interdisciplinary annual meeting that includes invited talks, demonstrations, symposia, and oral and poster presentations of refereed papers. Along with the conference is a professional exposition focusing on machine learning in practice, a series of tutorials, and topical workshops that provide a less formal setting for the exchange of ideas.

More about the Neural Information Processing Systems foundation »