Accurate semantic representation models are essential in text mining applications. For a successf... more Accurate semantic representation models are essential in text mining applications. For a successful application of the text mining process, the text representation adopted must keep the interesting patterns to be discovered. Although competitive results for automatic text classification may be achieved with traditional bag of words, such representation model cannot provide satisfactory classification performances on hard settings where richer text representations are required. In this paper, we present an approach to represent document collections based on embedded representations of words and word senses. We bring together the power of word sense disambiguation and the semantic richness of word-and wordsense embedded vectors to construct embedded representations of document collections. Our approach results in semantically enhanced and low-dimensional representations. We overcome the lack of interpretability of embedded vectors, which is a drawback of this kind of representation, with the use of word sense embedded vectors. Moreover, the experimental evaluation indicates that the use of the proposed representations provides stable classifiers with strong quantitative results, especially in semantically-complex classification scenarios.
Analysis of Domain Independent Statistical Keyword Extraction Methods for Incremental Clustering
Learning and Nonlinear Models, 2014
Analysis of Domain Independent Statistical Keyword Extraction Methods for Incremental Clustering
Incremental clustering is a very useful approach to organize dynamic text collections. Due to the... more Incremental clustering is a very useful approach to organize dynamic text collections. Due to the time/space restrictions for incremental clustering, the textual documents must be preprocessed to maintain only their most important information. Domain independent statistical keyword extraction methods are useful in this scenario, since they analyze only the content of each document individually instead of all document collection, are fast and language independent. However, different methods have different assumptions about the properties of keywords in a text, and different methods extract different set of keywords. Different ways to structure a textual document for keyword extraction can also modify the set of extracted keywords. Furthermore, extracting a small number of keywords might degrade the incremental clustering quality and a large number of keywords might increase the clustering process speed. In this article we analyze different ways to structure a textual document for key...
Sistema para extração de informações de artigos científicos-IESYSTEM
The use of frequent itemsets extracted from textual documents for the classification task
Anais do Encontro Nacional de Inteligência Artificial e Computacional (ENIAC 2020)
As representações em redes permitem modelar diferentes tipos de relações entre textos, são capaze... more As representações em redes permitem modelar diferentes tipos de relações entre textos, são capazes de capturar padrões dificilmente capturados pelo modelo espaço vetorial, e algoritmos de agrupamento em redes, como a propagação de rótulos, possuem complexidade linear. Porém, o agrupamento em redes não tem sido explorado (i) especificamente no agrupamento de textos; e (ii) com as diferentes possibilidades de representar textos em redes. Com isso, o objetivo deste artigo é a exploração e análise de técnicas de agrupamento aplicadas a diferentes tipos de representações em redes. Foram realizados experimentos utilizando 30 coleções de diferentes domínios, representadas em formato bag-of-words, redes de similaridade do tipo k-Nearest Neighbors e redes bipartidas. A abordagem de propagação de rótulos em redes de similaridade obteve os melhores resultados para a maioria das medidas de avaliação e para a maioria das coleções de textos. O algoritmo de propagação de rótulos demonstrou-se prom...
Applying transductive learning for automatic term extraction: The case of the ecology domain.
Terms are the basis for general text mining and natural language processing applications. However... more Terms are the basis for general text mining and natural language processing applications. However, the manual term extraction is unfeasible due to the huge number of words presented in a domain corpus and also the human effort required to do the extraction. For the term extraction task, machine learning techniques have been used to perform automatic term extraction (ATE). Inductive learning is commonly used, but it requires a large number of words labeled as terms and non-terms to build a classification model and classify unseen words. A better solution is the use of transductive learning, since it requires a small number of labeled examples to classify words as terms and non-terms. In this paper, we propose the use of transductive learning to ATE. The obtained results demonstrate that the application of transductive learning to ATE produces better results than the results obtained by inductive learning.
Analysis of Statistical Keyword Extraction Methods for Incremental Clustering
Incremental clustering is a very useful approach to organize dynamic text collections. Due to the... more Incremental clustering is a very useful approach to organize dynamic text collections. Due to the time/space restrictions for incremental clustering, the textual documents must be preprocessed to maintain only their most important information. Statistical keyword extraction methods from single documents are useful in this scenario. However, different statistical methods have different assumptions about the properties of keywords in a text, and different methods extract different set of keywords. In this paper we analyze the different methods for keyword extraction and the impact of the number of keywords on the quality of the incremental clustering. We also define a framework for statistical keyword extraction.
A comparative study of algorithms for recommending given names
Recommending given names is a special case of recommender system that is little explored, but has... more Recommending given names is a special case of recommender system that is little explored, but has gained a great interest recently. Indication of names related to a user's query or suggestion of names for parents in order to choose a name for their unborn child are examples of applications of name recommendation. In this paper, we present results for this task by using a baseline approach that recommends only the most frequent searched names, two state-of-the-art recommender systems, and a proposed method based on graph. The empirical evaluation shows that our method based on graph provides better results than the other methods.
Building a topic hierarchy using the bag-of-related-words representation.
A simple and intuitive way to organize a huge document collection is by a topic hierarchy. Genera... more A simple and intuitive way to organize a huge document collection is by a topic hierarchy. Generally two steps are carried out to build a topic hierarchy automatically: 1) hierarchical document clustering and 2) cluster labeling. For both steps, a good textual document representation is essential. The bag-of-words is the common way to represent text collections. In this representation, each document is represented by a vector where each word in the document collection represents a dimension (feature). This approach has well known problems as the high dimensionality and sparsity of data. Besides, most of the concepts are composed by more than one word, as "document engineering" or "text mining". In this paper an approach called bag-of-related-words is proposed to generate features compounded by a set of related words with a dimensionality smaller than the bag-of-words. The features are extracted from each textual document of a collection using association rules. Different ways to map the document into transactions in order to allow the extraction of association rules and interest measures to prune the number of features are analyzed. To evaluate how much the proposed approach can aid the topic hierarchy building, we carried out an objective evaluation for the clustering structure, and a subjective evaluation for topic hierarchies. All the results were compared with the bag-of-words. The obtained results demonstrated that the proposed representation is better than the bag-of-words for the topic hierarchy building.
Generating Features from Textual Documents Through Association Rules
The Text Mining techniques are used to organize, manage and extract knowledge from the huge amoun... more The Text Mining techniques are used to organize, manage and extract knowledge from the huge amount of textual data available in digital format. In order to use these techniques, the textual documents need to be represented in an appropriate format. The common way to represent text collections is by using the bag-of-words approach, in which each document is represented by a vector. Each word in the document collection represents a dimension of the vector. This approach has well known problems as the high dimensionality, and sparsity of data. Besides, most of the concepts are described by a set of words, such as “text mining”, “association rules”, and “machine learning”. The approaches, which generate features compounded by a set of words to solve this problem, suffer from other problems, such as the generation of features without meaning, and the need to analyze the high dimensionality of the bag-of-words in order to generate the features. An approach named bag-of-related-words is proposed to generate features compounded by a set of related words that avoids the problems as mentioned above. The features are generated from each textual document of a collection through association rules. Experiments were carried out using classification algorithms with different paradigms in order to evaluate the generated features. The obtained results demonstrated that the proposed approach is similar to the bag-of-words with much lower dimensionality and features which are easy to understand.
Several text mining techniques have been proposed to deal with the huge number of textual documen... more Several text mining techniques have been proposed to deal with the huge number of textual documents that are available and that have been published nowadays. Mainly classification techniques, which assign pre-defined labels to new documents, and clustering techniques, which separates texts into clusters. The techniques proposed in literature are usually applied to few textual collections, which are not sufficient to indicate how good a technique is or which characteristics of the collections make a technique obtain better results than others. Besides, new techniques are compared with traditional algorithms considering a small range of parameters, which make the comparison unfair. This technical report solve this lack by i) providing a characterization of 45 text collections; ii) providing classification results using traditional algorithms as Naïve Bayes, Multinomial Naïve Bayes, C4.5, k-Nearest Neighbors, and Support Vector Machines using a big range of values for the parameters; and iii) providing clustering results using traditional hierarchical algorithms as Bisecting k-Means and Unweighted Pair Group Method with Arithmetic Mean. We also make all the collections used in this technical report available in an on-line repository.
Inductive Model Generation for Text Categorization Using a Bipartite Heterogeneous Network
Usually, algorithms for categorization of numeric data have been applied for text categorization ... more Usually, algorithms for categorization of numeric data have been applied for text categorization after a preprocessing phase which assigns weights for textual terms deemed as attributes. However, due to characteristics of textual data, some algorithms for data categorization are not efficient for text categorization. Characteristics of textual data such as sparsity and high dimensionality sometimes impair the quality of general purpose classifiers. Here, we propose a text classifier based on a bipartite heterogeneous network used to represent textual document collections. Such algorithm induces a classification model assigning weights to objects that represents terms of the textual document collection. The induced weights correspond to the influence of the terms in the classification of documents they appear. The least-mean-square algorithm is used in the inductive process. Empirical evaluation using a large amount of textual document collections shows that the proposed IMBHN algorithm produces significantly better results than the k-NN, C4.5, SVM and Naïve Bayes algorithms.
Inductive Model Generation for Text Classification Using a Bipartite Heterogeneous Network
Algorithms for numeric data classification have been applied for text classification. Usually the... more Algorithms for numeric data classification have been applied for text classification. Usually the vector space model is used to represent text collections. The characteristics of this representation such as sparsity and high dimensionality sometimes impair the quality of general-purpose classifiers. Networks can be used to represent text collections, avoiding the high sparsity and allowing to model relationships among different objects that compose a text collection. Such network-based representations can improve the quality of the classification results. One of the simplest ways to represent textual collections by a network is through a bipartite heterogeneous network, which is composed of objects that represent the documents connected to objects that represent the terms. Heterogeneous bipartite networks do not require computation of similarities or relations among the objects and can be used to model any type of text collection. Due to the advantages of representing text collections through bipartite heterogeneous networks, in this article we present a text classifier which builds a classification model using the structure of a bipartite heterogeneous network. Such an algorithm, referred to as IMBHN (Inductive Model Based on Bipartite Heterogeneous Network), induces a classification model assigning weights to objects that represent the terms for each class of the text collection. An empirical evaluation using a large amount of text collections from different domains shows that the proposed IMBHN algorithm produces significantly better results than k-NN, C4.5, SVM, and Naive Bayes algorithms.
A parameter-free label propagation algorithm using bipartite heterogeneous networks for text classification
A bipartite heterogeneous network is one of the simplest ways to represent a textual document col... more A bipartite heterogeneous network is one of the simplest ways to represent a textual document collection. In such case, the network consists of two types of vertices, representing documents and terms, and links connecting terms to the documents. Transductive algorithms are usually applied to perform classification of networked objects. This type of classification is usually applied when few labeled examples are available, which may be worthwhile for practical situations. Nevertheless, for existing transductive algorithms users have to set several parameters that significantly affect the classification accuracy. In this paper, we propose a parameter-free algorithm for transductive classification of textual data, referred to as LPBHN (Label Propagation using Bipartite Heterogeneous Networks). LPBHN uses a bipartite heterogeneous network to perform the classification task. The proposed algorithm presents accuracy equivalent or higher than state-of-the-art algorithms for transductive classification in heterogeneous or homogeneous networks.
A simple and intuitive way to organize a huge document collection is by a topic hierarchy. Genera... more A simple and intuitive way to organize a huge document collection is by a topic hierarchy. Generally two steps are carried out to build a topic hierarchy automatically: 1) hierarchical document clustering and 2) cluster labeling. For both steps, a good textual document representation is essential. The bag-of-words is the common way to represent text collections. In this representation, each document is represented by a vector where each word in the document collection represents a dimension (feature). This approach has well known problems as the high dimensionality and sparsity of data. Besides, most of the concepts are composed by more than one word, as "document engineering" or "text mining". In this paper an approach called bag-of-related-words is proposed to generate features compounded by a set of related words with a dimensionality smaller than the bag-of-words. The features are extracted from each textual document of a collection using association rules. Different ways to map the document into transactions in order to allow the extraction of association rules and interest measures to prune the number of features are analyzed. To evaluate how much the proposed approach can aid the topic hierarchy building, we carried out an objective evaluation for the clustering structure, and a subjective evaluation for topic hierarchies. All the results were compared with the bag-of-words. The obtained results demonstrated that the proposed representation is better than the bag-of-words for the topic hierarchy building.
Considering the huge growing of documents in the digital universe and the possibility to obtain s... more Considering the huge growing of documents in the digital universe and the possibility to obtain some competitive advantage in processing them, this paper describes some of the difficulties to work with texts collections. More specifically, it shows some of the challenges on the step considered one of the most important of the Text Mining process -the data pre-processing -focusing on two of its main tasks: attributes generation and selection, considering not only single terms but composed terms too. In order to win the challenges imposed by these problems, this paper presents efficient unsupervised solutions to them. To evaluate these solutions, the application of them in three real data sets is presented, showing a way to treat the data step-by-step and obtaining good results at the end of the whole process.
The technological progress designing new devices and the scientific growth in the field of Human-... more The technological progress designing new devices and the scientific growth in the field of Human-Computer Interaction are enabling new interaction modalities to move from research to commercial products. However, developing multimodal interfaces is still a difficult task due to the lack of tools that consider not only code generation, but usability of such interfaces. In this paper, we present the MultiModal Web Approach and its authoring environment, whose main goal is boosting the dissemination of project knowledge for future developments benefited by the solutions to recurring problems in that multimodal context.
Uploads
Papers by Rafael Rossi