MERAL Myanmar Education Research and Learning Portal
Item
{"_buckets": {"deposit": "f2839568-b077-4848-b0b1-17a0b73c04df"}, "_deposit": {"created_by": 45, "id": "5341", "owner": "45", "owners": [45], "owners_ext": {"displayname": "", "username": ""}, "pid": {"revision_id": 0, "type": "recid", "value": "5341"}, "status": "published"}, "_oai": {"id": "oai:meral.edu.mm:recid/5341", "sets": ["user-uit"]}, "communities": ["uit"], "item_1583103067471": {"attribute_name": "Title", "attribute_value_mlt": [{"subitem_1551255647225": "Optimum Checkpoint Interval for MapReduce Fault-Tolerance", "subitem_1551255648112": "en"}]}, "item_1583103085720": {"attribute_name": "Description", "attribute_value_mlt": [{"interim": "MapReduce is the efficient framework for parallel\nprocessing of distributed big data in cluster environment.\nIn such a cluster, task failures can impact on performance\nof applications. Although MapReduce automatically\nreschedules the failed tasks, it takes long completion time\nbecause it starts from scratch. The checkpointing\nmechanism is the valuable technique to avoid reexecution\nof failed tasks in MapReduce. However,\ndefining incorrect checkpoint interval can still decrease\nthe performance of MapReduce applications and job\ncompletion time. In this paper, the optimum checkpoint\ninterval is proposed to reduce MapReduce job completion\ntime when failures occur. The proposed system defines\ncheckpoint interval that is based on five parameters:\nexpected job completion time without checkpointing,\ncheckpoint overhead time, rework time, down time and\nrestart time. Therefore, because of proposed checkpoint\ninterval, MapReduce does not need to re-execute the\nfailed tasks, so it reduces job completion time when\nfailures occur. The proposed system reduces job\ncompletion time even though the number of failures\nincreases and the performance of this system can be\nimproved 4 times better than the original MapReduce."}]}, "item_1583103108160": {"attribute_name": "Keywords", "attribute_value_mlt": [{"interim": "MapReduce"}, {"interim": "Big data"}, {"interim": "Task failures"}, {"interim": "Completion time"}, {"interim": "Checkpoint interval"}]}, "item_1583103120197": {"attribute_name": "Files", "attribute_type": "file", "attribute_value_mlt": [{"accessrole": "open_access", "date": [{"dateType": "Available", "dateValue": "2020-09-14"}], "displaytype": "preview", "download_preview_message": "", "file_order": 0, "filename": "Optimum Checkpoint Interval for MapReduce Fault-Tolerance.pdf", "filesize": [{"value": "471 Kb"}], "format": "application/pdf", "future_date_message": "", "is_thumbnail": false, "licensetype": "license_0", "mimetype": "application/pdf", "size": 471000.0, "url": {"url": "https://meral.edu.mm/record/5341/files/Optimum Checkpoint Interval for MapReduce Fault-Tolerance.pdf"}, "version_id": "72ca6625-206c-430e-9f24-2098f6fd26f2"}]}, "item_1583103147082": {"attribute_name": "Conference papers", "attribute_value_mlt": [{"subitem_acronym": "ICAIT", "subitem_c_date": "1 November, 2017", "subitem_conference_title": "International Conference on Advanced Information Technologies", "subitem_place": "Yangon, Myanamar", "subitem_website": "https://www.uit.edu.mm/icait-2017/"}]}, "item_1583105942107": {"attribute_name": "Authors", "attribute_value_mlt": [{"subitem_authors": [{"subitem_authors_fullname": "Naychi Nway Nway"}, {"subitem_authors_fullname": "Julia Myint"}]}]}, "item_1583108359239": {"attribute_name": "Upload type", "attribute_value_mlt": [{"interim": "Publication"}]}, "item_1583108428133": {"attribute_name": "Publication type", "attribute_value_mlt": [{"interim": "Conference paper"}]}, "item_1583159729339": {"attribute_name": "Publication date", "attribute_value": "2017-11-01"}, "item_title": "Optimum Checkpoint Interval for MapReduce Fault-Tolerance", "item_type_id": "21", "owner": "45", "path": ["1596102391527"], "permalink_uri": "http://hdl.handle.net/20.500.12678/0000005341", "pubdate": {"attribute_name": "Deposited date", "attribute_value": "2020-09-14"}, "publish_date": "2020-09-14", "publish_status": "0", "recid": "5341", "relation": {}, "relation_version_is_last": true, "title": ["Optimum Checkpoint Interval for MapReduce Fault-Tolerance"], "weko_shared_id": -1}
Optimum Checkpoint Interval for MapReduce Fault-Tolerance
http://hdl.handle.net/20.500.12678/0000005341
http://hdl.handle.net/20.500.12678/000000534173ee94e5-489d-46fe-a857-8f0ab43085b2
f2839568-b077-4848-b0b1-17a0b73c04df
Name / File | License | Actions |
---|---|---|
Optimum Checkpoint Interval for MapReduce Fault-Tolerance.pdf (471 Kb)
|
Publication type | ||||||
---|---|---|---|---|---|---|
Conference paper | ||||||
Upload type | ||||||
Publication | ||||||
Title | ||||||
Title | Optimum Checkpoint Interval for MapReduce Fault-Tolerance | |||||
Language | en | |||||
Publication date | 2017-11-01 | |||||
Authors | ||||||
Naychi Nway Nway | ||||||
Julia Myint | ||||||
Description | ||||||
MapReduce is the efficient framework for parallel processing of distributed big data in cluster environment. In such a cluster, task failures can impact on performance of applications. Although MapReduce automatically reschedules the failed tasks, it takes long completion time because it starts from scratch. The checkpointing mechanism is the valuable technique to avoid reexecution of failed tasks in MapReduce. However, defining incorrect checkpoint interval can still decrease the performance of MapReduce applications and job completion time. In this paper, the optimum checkpoint interval is proposed to reduce MapReduce job completion time when failures occur. The proposed system defines checkpoint interval that is based on five parameters: expected job completion time without checkpointing, checkpoint overhead time, rework time, down time and restart time. Therefore, because of proposed checkpoint interval, MapReduce does not need to re-execute the failed tasks, so it reduces job completion time when failures occur. The proposed system reduces job completion time even though the number of failures increases and the performance of this system can be improved 4 times better than the original MapReduce. |
||||||
Keywords | ||||||
MapReduce, Big data, Task failures, Completion time, Checkpoint interval | ||||||
Conference papers | ||||||
ICAIT | ||||||
1 November, 2017 | ||||||
International Conference on Advanced Information Technologies | ||||||
Yangon, Myanamar | ||||||
https://www.uit.edu.mm/icait-2017/ |