Please use this identifier to cite or link to this item: http://hdl.handle.net/2445/47608
Title: K-means web clustering amb Hadoop MapReduce
Author: Huélamo Segura, Alberto
Director: Puertas i Prats, Eloi
Keywords: Anàlisi de conglomerats
Processament distribuït de dades
Programari
Tesis
Cluster analysis
Distributed processing in electronic data processing
Computer software
Theses
Issue Date: Jun-2013
Abstract: This paper proposes a solution to the problem of clustering large amount of web documents. The Hadoop framework, implementation of MapReduce distributed programming paradigm, developed by Google, plays a very important role in this field due to its scalability and ease to parallelize software. This is the reason why it is used in this project. Meanwhile, K-means clustering algorithm is easily adaptable to MapReduce programming model and provides proper results for web documents. The documents will be represented as a frequence vectors of terms and keywords and this is what algorithm needs to work. The developed software uses Hadoop in order to perform both tasks which make up the overall process: document processing and the clustering. Web documents are in HTML, which is not suitable for K-means. It is necessary preprocess them to extract descriptors and to pass them to the clustering algorithm. This is the first part of the process. The second part, K-means on Hadoop, goes beyond typical Hadoop execution, using most of the tools which Hadoop provides to make document clusters, from descriptors obtained from first part of the process.
Note: Treballs Finals de Grau d'Enginyeria Informàtica, Facultat de Matemàtiques, Universitat de Barcelona, Any: 2013, Director: Eloi Puertas i Prats
URI: http://hdl.handle.net/2445/47608
Appears in Collections:Treballs Finals de Grau (TFG) - Enginyeria Informàtica
Programari - Treballs de l'alumnat

Files in This Item:
File Description SizeFormat 
memoria.pdfMemòria1.72 MBAdobe PDFView/Open
src.zipcodi font6.32 MBzipView/Open


This item is licensed under a Creative Commons License Creative Commons