Laboratoire Informatique, Image et Interaction (L3i)

Optical character recognition (OCR) is one of the most popular techniques used for converting printed documents into machine-readable ones. While OCR engines can do well with modern text, their performance is unfortunately significantly reduced on historical materials. Additionally, many texts have already been processed by various out-of-date digitisation techniques. As a consequence, digitised texts are noisy and need to be post-corrected. This article clarifies the importance of enhancing quality of OCR results by studying their effects on information retrieval and natural language processing applications. We then define the post-OCR processing problem, illustrate its typical pipeline, and review the state-of-the-art post-OCR processing approaches. Evaluation metrics, accessible datasets, language resources, and useful toolkits are also reported. Furthermore, the work identifies the current trend and outlines some research directions of this field.

We present in this paper some advances in color restoration of underwater images, especially with regard to the strong and non uniform color cast which is typical of underwater images. The proposed color correction method is based on ACE model, an unsupervised color equalization algorithm. ACE is a perceptual approach inspired by some adaptation mechanisms of the human visual system, in particular lightness constancy and color constancy. A perceptual approach presents a lot of advantages: it is unsupervised, robust and has local filtering properties, that lead to more effective results. The restored images give better results when displayed or processed (fish segmentation and feature extraction). The presented preliminary results are satisfying and promising.

In collaborative wireless networks with low infrastructure, the presence of misbehaving nodes can have a negative impact on network performance. In particular, we are interested in dealing with this nasty presence in road safety applications, based on vehicular ad hoc networks (VANETs). In this paper, we consider as harmful the presence of malicious nodes, which spread false and forged data, and selfish nodes, which cooperate only for their own benefit. To deal with this, we propose a distributed trust model <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="TeX">$(DTM^{2})$</tex-math></inline-formula> , which is adapted from the job market signaling model. <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="TeX">$DTM^{2} $</tex-math></inline-formula> is based on allocating <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><i>credits</i></b> to nodes and securely managing these credits. To motivate selfish nodes to cooperate more, our solution establishes the cost of reception to access data, forcing them to earn credits. Moreover, to detect and exclude malicious nodes, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="TeX">$DTM^{2} $</tex-math></inline-formula> requires the cost of sending, using signaling values inspired from economics and based on the node's behavior so that the more malicious a node is, the higher its sending cost, thus limiting their participation in the network. Similarly, rewards are given to nodes whose sent messages are considered truthful and that paid a sending cost considered correct. The latter is a guarantee for the receivers about the truthfulness of the message since, in the case of message refusal, the source node is not rewarded, despite its payment. We validated <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="TeX">$DTM^{2} $</tex-math></inline-formula> via a theoretical study using Markov chains and with a set of simulations in both urban and highway scenarios. Both theoretical and simulation results show that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="TeX">$DTM^{2} $</tex-math></inline-formula> excludes from the network 100% of malicious nodes without causing any false-positive detection. Moreover, our solution guarantees a good ratio of reception, even in the presence of selfish nodes.

Handbook of Robust Low-Rank and Sparse Matrix Decomposition: Applications in Image and Video Processing shows you how robust subspace learning and tracking by decomposition into low-rank and sparse matrices provide a suitable framework for computer vision applications. Incorporating both existing and new ideas, the book conveniently gives you one-stop access to a number of different decompositions, algorithms, implementations, and benchmarking techniques. Divided into five parts, the book begins with an overall introduction to robust principal component analysis (PCA) via decomposition into low-rank and sparse matrices. The second part addresses robust matrix factorization/completion problems while the third part focuses on robust online subspace estimation, learning, and tracking. Covering applications in image and video processing, the fourth part discusses image analysis, image denoising, motion saliency detection, video coding, key frame extraction, and hyperspectral video processing. The final part presents resources and applications in background/foreground separation for video surveillance. With contributions from leading teams around the world, this handbook provides a complete overview of the concepts, theories, algorithms, and applications related to robust low-rank and sparse matrix decompositions. It is designed for researchers, developers, and graduate students in computer vision, image and video processing, real-time architecture, machine learning, and data mining.

International audience

Background subtraction is an important task for visual surveillance systems. However, this task becomes more complex when the data size grows since the real-world scenario requires larger data to be processed in a more efficient way, and in some cases, in a continuous manner. Until now, most of background subtraction algorithms were designed for mono or trichromatic cameras within the visible spectrum or near infrared part. Recent advances in multispectral imaging technologies give the possibility to record multispectral videos for video surveillance applications. Due to the specific nature of these data, many of the bands within multispectral images are often strongly correlated. In addition, processing multispectral images with hundreds of bands can be computationally burdensome. In order to address these major difficulties of multispectral imaging for video surveillance, this paper propose an online stochastic framework for tensor decomposition of multispectral video sequences (OSTD). First, the experimental evaluations on synthetic generated data show the robustness of the OSTD with other state of the art approaches then, we apply the same idea on seven multispectral video bands to show that only RGB features are not sufficient to tackle color saturation, illumination variations and shadows problem, but the addition of six visible spectral bands together with one near infrared spectra provides a better background/foreground separation.

Post-OCR is an important processing step that follows optical character recognition (OCR) and is meant to improve the quality of OCR documents by detecting and correcting residual errors. This paper describes the results of a statistical analysis of OCR errors on four document collections. Five aspects related to general OCR errors are studied and compared with human-generated misspellings, including edit operations, length effects, erroneous character positions, real-word vs. non-word errors, and word boundaries. Based on the observations from the analysis we give several suggestions related to the design and implementation of effective OCR post-processing approaches.

International audience

Background subtraction process plays a very essential role for various computer vision tasks. The process becomes more critical when the input scene contains variation of pixels such as swaying trees, rippling of water, illumination variations, etc. Recent methods of matrix decomposition into low-rank (e.g., corresponds to the background) and sparse (e.g., constitutes the moving objects) components such as Robust Principal Component Analysis (RPCA), have been shown to be very efficient framework for background subtraction. However, when the size of the input data grows and due to the lack of sparsity-constraints, these methods cannot cope with the real-time challenges and always show a weak performance due to the erroneous foreground regions. In order to address the above mentioned issues, this paper presents a superpixel-based matrix decomposition method together with maximum norm (max-norm) regularizations and structured sparsity constraints. The low-rank component estimated from each homogeneous region is more perfect, reliable, and efficient, since each superpixel provides different characteristics with a reduced value of rank. Online max-norm based matrix decomposition is employed on each segmented superpixel to separate the low rank and initial outliers support. And then, the structured sparsity constraints such as the generalized fussed lasso (GFL) are adopted for exploiting structural information continuously as the foreground pixels are both spatially connected and sparse. We propose an online single unified optimization framework for detecting foreground and learning the background model simultaneously. Rigorous experimental evaluations on challenging datasets demonstrate the superior performance of the proposed scheme in terms of both accuracy and computational time.

The development of automated video-surveillance applications for maritime environment is a very difficult task due to the complexity of the scenes: moving water, waves, etc. The motion of the objects of interest (i.e. ships or boats) can be mixed with the dynamic behavior of the background (non-regular patterns). In this paper, a double-constrained Robust Principal Component Analysis (RPCA), named SCM-RPCA (Shape and Confidence Map-based RPCA), is proposed to improve the object foreground detection in maritime scenes. The sparse component is constrained by shape and confidence maps both extracted from spatial saliency maps. The experimental results in the UCSD and MarDT data sets indicate a better enhancement of the object foreground mask when compared with some related RPCA methods.

Skin cancer is one of the most common types of cancer all over the world. It is easily treatable when it is detected in its beginning stage. Melanoma is the most dangerous form of skin cancer. Early detection of melanoma is important in reducing the mortality rate of skin cancer. Recently, machine learning has become an efficient method in classifying skin lesions as melanoma or benign. Main features for this task include color, texture and shape. A comparative study about color, texture and shape features of melanoma is useful for future research of skin cancer classification. Inspired by this fact, our study compares the classification results of 6 classifiers in combination with 7 feature extraction methods and 4 data preprocessing steps on the two largest datasets of skin cancer. Our findings reveal that a system consisting of Linear Normalization of the input image as data preprocessing step, HSV as feature extraction method and Balanced Random Forest as classifier yields best prediction results on the HAM10000 dataset with 81.46% AUC, 74.75% accuracy, 90.09% sensitivity and 72.84 % specificity.

Digital collections are increasingly used for a variety of purposes. In Europe only, we can conservatively estimate that tens of thousands of users consult digital libraries daily. The usages are often motivated by qualitative and quantitative research. However, caution must be advised as most digitized documents are indexed through their OCRed version, which is far from perfect, especially for ancient documents. In this paper, we aim to estimate the impact of OCR errors on the use of a major online platform: The Gallica digital library from the National Library of France. It accounts for more than 100M OCRed documents and receives 80M search queries every year. In this context, we introduce two main contributions. First, an original corpus of OCRed documents composed of 12M characters along with the corresponding gold standard is presented and provided, with an equal share of English- and French-written documents. Next, statistics on OCR errors have been computed thanks to a novel alignment method introduced in this paper. Making use of all the user queries submitted to the Gallica portal over 4 months, we take advantage of our error model to propose an indicator for predicting the relative risk that queried terms mismatch targeted resources due to OCR errors, underlining the critical extent to which OCR quality impacts on digital library access.

Paper documents still represent a large amount of information supports used nowadays and may contain critical data. Even though official documents are secured with techniques such as printed patterns or artwork, paper documents suffer from a lack of security. However, the high availability of cheap scanning and printing hardware allows non-experts to easily create fake documents. As the use of a watermarking system added during the document production step is hardly possible, solutions have to be proposed to distinguish a genuine document from a forged one. In this paper, we present an automatic forgery detection method based on document's intrinsic features at character level. This method is based on the one hand on outlier character detection in a discriminant feature space and on the other hand on the detection of strictly similar characters. Therefore, a feature set is computed for all characters. Then, based on a distance between characters of the same class, the character is classified as a genuine one or a fake one.

In this paper, a new binary linear programming formulation for computing the exact Graph Edit Distance (GED) between two graphs is proposed. A fundamental strength of the formulations lies in their genericity since the GED can be computed between directed or undirected fully attributed graphs. Moreover, a continuous relaxation of the domain constraints in the formulation provides an efficient lower bound approximation of the GED. A complete experimental study that compares the proposed formulations with six state-of-the-art algorithms is provided. By considering both the accuracy of the proposed solution and the efficiency of the algorithms as performance criteria, the results show that none of the compared methods dominate the others in the Pareto sense. In general, our formulation converges faster to optimality while being able to scale up to match the largest graphs in our experiments. The relaxed formulation leads to an accurate approach that is 12% more accurate than the best approximate method of our benchmark.

Comic characters detection has been an interesting area in comic analysis as it not only allows more efficient indexation and retrieval for comic books but also yields an adequate understanding of comics so as to help better creating the digital form of comic books. In recent years, several methods that have been proposed to extract/detect characters from comics, have given reasonable performance. However, they always use their datasets to evaluate the methods without comparing with other works or experimenting on a standard dataset. In this work, we take advantage of the recent and significant development of deep learning to apply it to comic character detection. We use the latest object detection deep networks to train the comic characters detector based on our proposed dataset. By experimenting on our proposed dataset and also on available datasets from previous works, we have found that this method significantly outperforms existing methods. We believe that this state-of-the-art approach can be considered as a reliable baseline method to compare and better understand future detection techniques.

Document forgery is an increasing problem for both the public administration and private companies. It represents substantial losses in time and economical resources. Classical solutions to this problem such as watermarks or other integrated security patterns can not be applied in general for any unknown incoming document due to the large variability on types of documents. In that scenario it is important to resort to forensic techniques to seek and analyze inconsistencies on the intrinsic features of the document image. In this paper we present a classification-based approach for forgery detection. We use uniform Local Binary Patterns (LBP) to capture discriminant texture features that are common on forged regions. Besides, we combine multiple descriptors from neighboring regions to model contextual information. Results using Support Vector Machines (SVM) for patch classification show that we are able to detect several types of forgeries in a wide range of types of documents.

International audience

The quality of OCR has a direct impact on information access, and an indirect impact on the performance of natural language processing applications, making fine-grained (e.g., semantic) information access even harder. This work proposes a novel post-OCR approach based on a contextual language model and neural machine translation, aiming to improve the quality of OCRed text by detecting and rectifying erroneous tokens. This new technique obtains results comparable to the best-performing approaches on English datasets of the competition on post-OCR text correction in ICDAR 2017/2019.

Abstract The Louvain method was proposed 15 years ago as a heuristic method for the fast detection of communities in large networks. During this period, it has emerged as one of the most popular methods for community detection: the task of partitioning vertices of a network into dense groups, usually called communities or clusters. Here, after a short introduction to the method, we give an overview of the different generalizations, modifications and improvements that have been proposed in the literature, and also survey the quality functions, beyond modularity, for which it has been implemented. Finally, we conclude with a discussion on the limitations of the method and perspectives for future research.

The use of digital libraries requires an easy accessibility to documents which is strongly impacted by the quality of document indexing. Named entities are among the most important information to index digital documents. According to a recent study, 80% of the top 500 queries sent to a digital library portal contained at least one named entity. However most digitized documents are indexed through their OCRed version which includes numerous errors that may hinder the access to them. Named Entity Recognition (NER) is the task that aims to locate important names in a given text and to categorize them into a set of predefined classes (person, location, organization). This paper aims to estimate the performance of NER systems through OCRed data. It exhaustively iscusses NER errors arising from OCR errors; we studied the correlation between NER accuracy and OCR error rates and estimated the cost of character insertion, deletion and substitution in named entities. Results show that even if he OCR engine does contaminate named entities with errors, NER systems can overcome this issue and correctly recognize some of them.