Balanced MapReduce With Replication (BMWR) Model: An improved load balancingmodel based on fault tolerance replication factor

With the emergence of big data, there is also a growing need to process it. MapReducecurrently is the leading model in processing and analyzing big data. However, it has a majordefect. At the end of the map phase each key is assigned to only one reduce node, if the datacorresponding to a specific key accounts for most of the data, there will not be a balancebetween the loads corresponding to reduce nodes. With respect to the aforementionedproblem a new programming model called Balanced MapReduce With Replication (BMWR)is proposed which works with unbalanced keys. This paper proposes a scheduling based onthe inherent replication factor of the HDFS framework. If a reduce node is overloaded, then aportion of its remaining job is handled by the other reduce nodes that have a replica of thekey/value pairs. The testing results reveal that compared to MapReduce programming model,the proposed model achieves an improvement of 0% to 3% when the data is distributedunevenly. However, when data is distributed evenly the time cost of the proposed model is03231.

کلیدواژه ها:

MapReduce ، Scheduling ، Load Balancing ، Big Data