MERAL Myanmar Education Research and Learning Portal
Item
{"_buckets": {"deposit": "b263d667-5a33-4150-aa37-3475753356c3"}, "_deposit": {"created_by": 45, "id": "2931", "owner": "45", "owners": [45], "owners_ext": {"displayname": "", "username": ""}, "pid": {"revision_id": 0, "type": "recid", "value": "2931"}, "status": "published"}, "_oai": {"id": "oai:meral.edu.mm:recid/2931", "sets": ["1596102355557", "user-uit"]}, "communities": ["uit"], "item_1583103067471": {"attribute_name": "Title", "attribute_value_mlt": [{"subitem_1551255647225": "Merging Small Files Based on Agglomerative Hierarchical Clustering on HDFS for Cloud Storage", "subitem_1551255648112": "en"}]}, "item_1583103085720": {"attribute_name": "Description", "attribute_value_mlt": [{"interim": "Hadoop distributed file system (HDFS) was\noriginally designed for large files. HDFS stores each\nsmall file as one separate block although the size of\nseveral small files is lesser than the size of block size.\nTherefore, a large number of blocks are created with\nmassive small files. When the large number of small\nfiles is accessed, NameNode often becomes the\nbottleneck. The problem of storing and accessing\nlarge number of small files is named as small file\nproblem. In order to solve this issue in HDFS, an\napproach of merging small files on HDFS is\nproposed. In this paper, small files are merged into a\nlarger file based on the agglomerative hierarchical\nclustering mechanism to reduce NameNode memory\nconsumption. This approach will provide small files\nfor cloud storage."}]}, "item_1583103108160": {"attribute_name": "Keywords", "attribute_value_mlt": [{"interim": "HDFS"}, {"interim": "Small Files"}]}, "item_1583103120197": {"attribute_name": "Files", "attribute_type": "file", "attribute_value_mlt": [{"accessrole": "open_access", "date": [{"dateType": "Available", "dateValue": "2020-08-06"}], "displaytype": "preview", "download_preview_message": "", "file_order": 0, "filename": "Merging Small Files Based on Agglomerative Hierarchical Clustering on HDFS for Cloud Storage.pdf", "filesize": [{"value": "239 Kb"}], "format": "application/pdf", "future_date_message": "", "is_thumbnail": false, "licensetype": "license_0", "mimetype": "application/pdf", "size": 239000.0, "url": {"url": "https://meral.edu.mm/record/2931/files/Merging Small Files Based on Agglomerative Hierarchical Clustering on HDFS for Cloud Storage.pdf"}, "version_id": "9fe1a53b-01f3-40f7-b4c2-5ccafa8b7d64"}]}, "item_1583103147082": {"attribute_name": "Conference papers", "attribute_value_mlt": [{"subitem_acronym": "ICCA 2018", "subitem_c_date": "22-23 February, 2018", "subitem_conference_title": "16th International Conference on Computer Applications", "subitem_place": "Sedona Hotel, Yangon, Myanmar", "subitem_website": "https://www.ucsy.edu.mm/page228.do"}]}, "item_1583105942107": {"attribute_name": "Authors", "attribute_value_mlt": [{"subitem_authors": [{"subitem_authors_fullname": "Khin Su Su Wai"}, {"subitem_authors_fullname": "Julia Myint"}, {"subitem_authors_fullname": "Tin Tin Yee"}]}]}, "item_1583108359239": {"attribute_name": "Upload type", "attribute_value_mlt": [{"interim": "Publication"}]}, "item_1583108428133": {"attribute_name": "Publication type", "attribute_value_mlt": [{"interim": "Conference paper"}]}, "item_1583159729339": {"attribute_name": "Publication date", "attribute_value": "2018-02-23"}, "item_title": "Merging Small Files Based on Agglomerative Hierarchical Clustering on HDFS for Cloud Storage", "item_type_id": "21", "owner": "45", "path": ["1596102355557"], "permalink_uri": "http://hdl.handle.net/20.500.12678/0000002931", "pubdate": {"attribute_name": "Deposited date", "attribute_value": "2020-08-06"}, "publish_date": "2020-08-06", "publish_status": "0", "recid": "2931", "relation": {}, "relation_version_is_last": true, "title": ["Merging Small Files Based on Agglomerative Hierarchical Clustering on HDFS for Cloud Storage"], "weko_shared_id": -1}
Merging Small Files Based on Agglomerative Hierarchical Clustering on HDFS for Cloud Storage
http://hdl.handle.net/20.500.12678/0000002931
http://hdl.handle.net/20.500.12678/0000002931d7235ba7-688c-4731-bd85-4e6fd15b1d20
b263d667-5a33-4150-aa37-3475753356c3
Name / File | License | Actions |
---|---|---|
![]() |
Publication type | ||||||
---|---|---|---|---|---|---|
Conference paper | ||||||
Upload type | ||||||
Publication | ||||||
Title | ||||||
Title | Merging Small Files Based on Agglomerative Hierarchical Clustering on HDFS for Cloud Storage | |||||
Language | en | |||||
Publication date | 2018-02-23 | |||||
Authors | ||||||
Khin Su Su Wai | ||||||
Julia Myint | ||||||
Tin Tin Yee | ||||||
Description | ||||||
Hadoop distributed file system (HDFS) was originally designed for large files. HDFS stores each small file as one separate block although the size of several small files is lesser than the size of block size. Therefore, a large number of blocks are created with massive small files. When the large number of small files is accessed, NameNode often becomes the bottleneck. The problem of storing and accessing large number of small files is named as small file problem. In order to solve this issue in HDFS, an approach of merging small files on HDFS is proposed. In this paper, small files are merged into a larger file based on the agglomerative hierarchical clustering mechanism to reduce NameNode memory consumption. This approach will provide small files for cloud storage. |
||||||
Keywords | ||||||
HDFS, Small Files | ||||||
Conference papers | ||||||
ICCA 2018 | ||||||
22-23 February, 2018 | ||||||
16th International Conference on Computer Applications | ||||||
Sedona Hotel, Yangon, Myanmar | ||||||
https://www.ucsy.edu.mm/page228.do |