2024-03-28T15:37:10Z
https://meral.edu.mm/oai
oai:meral.edu.mm:recid/5341
2021-12-13T06:53:02Z
1582963342780:1596102391527
user-uit
Optimum Checkpoint Interval for MapReduce Fault-Tolerance
Naychi Nway Nway
Julia Myint
MapReduce is the efficient framework for parallel
processing of distributed big data in cluster environment.
In such a cluster, task failures can impact on performance
of applications. Although MapReduce automatically
reschedules the failed tasks, it takes long completion time
because it starts from scratch. The checkpointing
mechanism is the valuable technique to avoid reexecution
of failed tasks in MapReduce. However,
defining incorrect checkpoint interval can still decrease
the performance of MapReduce applications and job
completion time. In this paper, the optimum checkpoint
interval is proposed to reduce MapReduce job completion
time when failures occur. The proposed system defines
checkpoint interval that is based on five parameters:
expected job completion time without checkpointing,
checkpoint overhead time, rework time, down time and
restart time. Therefore, because of proposed checkpoint
interval, MapReduce does not need to re-execute the
failed tasks, so it reduces job completion time when
failures occur. The proposed system reduces job
completion time even though the number of failures
increases and the performance of this system can be
improved 4 times better than the original MapReduce.
2017-11-01
http://hdl.handle.net/20.500.12678/0000005341
https://meral.edu.mm/records/5341