Evaluating Checkpoint Interval for Fault-Tolerance in MapReduce

Naychi Nway Nway; Julia Myint; Ei Chaw Htoon

MERAL Myanmar Education Research and Learning Portal

Index Tree

RootNode
- Co-operative College, Mandalay
- Cooperative College, Phaunggyi
- Co-operative University, Sagaing
- Co-operative University, Thanlyin
- Dagon University
- Kyaukse University
- Laquarware Technological college
- Mandalay Technological University
- Mandalay University of Distance Education
- Mandalay University of Foreign Languages
- Maubin University
- Mawlamyine University
- Meiktila University
- Mohnyin University
- Myanmar Institute of Information Technology
- Myanmar Maritime University
- National Management Degree College
- Naypyitaw State Academy
- Pathein University
- Sagaing University
- Sagaing University of Education
- Taunggyi University
- Technological University, Hmawbi
- Technological University (Kyaukse)
- Technological University Mandalay
- University of Computer Studies, Mandalay
- University of Computer Studies Maubin
- University of Computer Studies, Meikhtila
- University of Computer Studies Pathein
- University of Computer Studies, Taungoo
- University of Computer Studies, Yangon
- University of Dental Medicine Mandalay
- University of Dental Medicine, Yangon
- University of Information Technology
- University of Mandalay
- University of Medicine 1
- University of Medicine 2
- University of Medicine Mandalay
- University of Myitkyina
- University of Public Health, Yangon
- University of Veterinary Science
- University of Yangon
- West Yangon University
- Yadanabon University
- Yangon Technological University
- Yangon University of Distance Education
- Yangon University of Economics
- Yangon University of Education
- Yangon University of Foreign Languages
- Yezin Agricultural University
- New Index

Item

{"_buckets": {"deposit": "4c53ce2f-9bbd-407d-a8d6-e05417de6bef"}, "_deposit": {"created_by": 45, "id": "5342", "owner": "45", "owners": [45], "owners_ext": {"displayname": "", "username": ""}, "pid": {"revision_id": 0, "type": "recid", "value": "5342"}, "status": "published"}, "_oai": {"id": "oai:meral.edu.mm:recid/5342", "sets": ["user-uit"]}, "communities": ["uit"], "item_1583103067471": {"attribute_name": "Title", "attribute_value_mlt": [{"subitem_1551255647225": "Evaluating Checkpoint Interval for Fault-Tolerance in MapReduce", "subitem_1551255648112": "en"}]}, "item_1583103085720": {"attribute_name": "Description", "attribute_value_mlt": [{"interim": "MapReduce is the efficient framework for parallel\nprocessing of distributed big data in cluster environment. In\nsuch a cluster, task failures can impact on performance of\napplications. Although MapReduce automatically reschedules\nthe failed tasks, it takes long completion time because it starts\nfrom scratch. The checkpointing mechanism is the valuable\ntechnique to avoid re-execution of finished tasks in\nMapReduce. However, defining incorrect checkpoint interval\ncan still decrease the performance of MapReduce applications\nand job completion time. So, in this paper, checkpoint interval\nis proposed to avoid re-execution of whole tasks in case of task\nfailures and save job completion time. The proposed\ncheckpoint interval is based on five parameters: expected job\ncompletion time without checkpointing, checkpoint overhead\ntime, rework time, down time and restart time. The\nexperiments show that the proposed checkpoint interval takes\nthe advantage of less checkpoints overhead and reduce\ncompletion time at failure time."}]}, "item_1583103108160": {"attribute_name": "Keywords", "attribute_value_mlt": [{"interim": "MapReduce"}, {"interim": "Big data"}, {"interim": "Task failures"}, {"interim": "Completion time"}, {"interim": "Checkpoint interval"}]}, "item_1583103120197": {"attribute_name": "Files", "attribute_type": "file", "attribute_value_mlt": [{"accessrole": "open_no", "date": [{"dateType": "Available", "dateValue": "2020-09-14"}], "displaytype": "preview", "filename": "Evaluating Checkpoint Interval for Fault-Tolerance in MapReduce.pdf", "filesize": [{"value": "286 Kb"}], "format": "application/pdf", "licensefree": "©2018 IEEE", "licensetype": "license_free", "url": {"url": "https://meral.edu.mm/record/5342/files/Evaluating Checkpoint Interval for Fault-Tolerance in MapReduce.pdf"}, "version_id": "b5a0a9fd-da1d-4f42-b9a7-1c3a072c7ded"}]}, "item_1583103147082": {"attribute_name": "Conference papers", "attribute_value_mlt": [{"subitem_acronym": "Cyber C", "subitem_c_date": "18 October, 2018", "subitem_conference_title": "2018 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery", "subitem_place": "Zhengzhou, China, China"}]}, "item_1583105942107": {"attribute_name": "Authors", "attribute_value_mlt": [{"subitem_authors": [{"subitem_authors_fullname": "Naychi Nway Nway"}, {"subitem_authors_fullname": "Julia Myint"}, {"subitem_authors_fullname": "Ei Chaw Htoon"}]}]}, "item_1583108359239": {"attribute_name": "Upload type", "attribute_value_mlt": [{"interim": "Publication"}]}, "item_1583108428133": {"attribute_name": "Publication type", "attribute_value_mlt": [{"interim": "Conference paper"}]}, "item_1583159729339": {"attribute_name": "Publication date", "attribute_value": "2018-10-18"}, "item_1583159847033": {"attribute_name": "Identifier", "attribute_value": "10.1109/CyberC.2018.00046"}, "item_title": "Evaluating Checkpoint Interval for Fault-Tolerance in MapReduce", "item_type_id": "21", "owner": "45", "path": ["1596102391527"], "permalink_uri": "http://hdl.handle.net/20.500.12678/0000005342", "pubdate": {"attribute_name": "Deposited date", "attribute_value": "2020-09-14"}, "publish_date": "2020-09-14", "publish_status": "0", "recid": "5342", "relation": {}, "relation_version_is_last": true, "title": ["Evaluating Checkpoint Interval for Fault-Tolerance in MapReduce"], "weko_shared_id": -1}

Evaluating Checkpoint Interval for Fault-Tolerance in MapReduce

http://hdl.handle.net/20.500.12678/0000005342

Publication type
		Conference paper
Upload type
		Publication
Title
	Title	Evaluating Checkpoint Interval for Fault-Tolerance in MapReduce
	Language	en
Publication date		2018-10-18
Authors
		Naychi Nway Nway
		Julia Myint
		Ei Chaw Htoon
Description
		MapReduce is the efficient framework for parallel processing of distributed big data in cluster environment. In such a cluster, task failures can impact on performance of applications. Although MapReduce automatically reschedules the failed tasks, it takes long completion time because it starts from scratch. The checkpointing mechanism is the valuable technique to avoid re-execution of finished tasks in MapReduce. However, defining incorrect checkpoint interval can still decrease the performance of MapReduce applications and job completion time. So, in this paper, checkpoint interval is proposed to avoid re-execution of whole tasks in case of task failures and save job completion time. The proposed checkpoint interval is based on five parameters: expected job completion time without checkpointing, checkpoint overhead time, rework time, down time and restart time. The experiments show that the proposed checkpoint interval takes the advantage of less checkpoints overhead and reduce completion time at failure time.
Keywords
		MapReduce, Big data, Task failures, Completion time, Checkpoint interval
Identifier		10.1109/CyberC.2018.00046
Conference papers
		Cyber C
		18 October, 2018
		2018 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery
		Zhengzhou, China, China