Implementation of C4.5 Algorithm using Hadoop Map Reduce Paradigm

C4.5 is a commonly used in decision tree algorithm in data mining for classification. The existing C4.5 algorithm implementation is running in serial way. We are implementing this algorithm using Hadoop MapReduce framework which can run parallel in multiple system. In this project we are comparing our result with Weka's result where C4.5 is serially implemented with different data source of different size.

Algorithm:

CurrentNode is assumed for splitting.
Map(key, value)
{

Hadoop Installation on Ubuntu (A to Z )

Single Node Cluster setup

1) Making apt-get work and installing ssh server & client for remote login to other machines
a.i) ADDING A DEDICATED HADOOP USER
Run this command to add a dedicated user hduser and the group hadoop to your local machine
(every machine on Multi-node cluster system must have hduser otherwise you will get error while
making ssh connection with remote hosts).
% sudo addgroup hadoop
% sudo adduser --ingroup hadoop hduser

NOTE:-Login as hduser.
a.ii) If u are using any proxy server than configure /etc/apt/apt.config file by typing (as root node)
(a.ii.1) % gksudo gedit /etc/apt/apt.conf or % vi gedit /etc/apt/apt.conf (if doesn’t not exists
already than create it) on the command prompt and adding following lines.
Acquire::http::Proxy “http://name:password@proxyip:port”
Acquire::ftp::Proxy “ftp://name:password@proxyip:port”
Acquire::https::Proxy “https://name:password@proxyip:port”
(a.ii.2) Where name and password are proxy username and proxy password respectively.
Check by entering
% sudo apt-get update(it should work).
a.iii) Also uncomment the sources from /etc/apt/source.list file, by removing # from the very first
line and lines containing deb next to #.