Index Link

  • RootNode
    • Co-operative College, Mandalay
    • Cooperative College, Phaunggyi
    • Co-operative University, Sagaing
    • Co-operative University, Thanlyin
    • Dagon University
    • Kyaukse University
    • Laquarware Technological college
    • Mandalay Technological University
    • Mandalay University of Distance Education
    • Mandalay University of Foreign Languages
    • Maubin University
    • Mawlamyine University
    • Meiktila University
    • Mohnyin University
    • Myanmar Institute of Information Technology
    • Myanmar Maritime University
    • National Management Degree College
    • Naypyitaw State Academy
    • Pathein University
    • Sagaing University
    • Sagaing University of Education
    • Taunggyi University
    • Technological University, Hmawbi
    • Technological University (Kyaukse)
    • Technological University Mandalay
    • University of Computer Studies, Mandalay
    • University of Computer Studies Maubin
    • University of Computer Studies, Meikhtila
    • University of Computer Studies Pathein
    • University of Computer Studies, Taungoo
    • University of Computer Studies, Yangon
    • University of Dental Medicine Mandalay
    • University of Dental Medicine, Yangon
    • University of Information Technology
    • University of Mandalay
    • University of Medicine 1
    • University of Medicine 2
    • University of Medicine Mandalay
    • University of Myitkyina
    • University of Public Health, Yangon
    • University of Veterinary Science
    • University of Yangon
    • West Yangon University
    • Yadanabon University
    • Yangon Technological University
    • Yangon University of Distance Education
    • Yangon University of Economics
    • Yangon University of Education
    • Yangon University of Foreign Languages
    • Yezin Agricultural University
    • New Index

Item

{"_buckets": {"deposit": "b826e53f-1b2a-45ec-8832-878d3e7b2617"}, "_deposit": {"created_by": 45, "id": "6300", "owner": "45", "owners": [45], "owners_ext": {"displayname": "", "username": ""}, "pid": {"revision_id": 0, "type": "recid", "value": "6300"}, "status": "published"}, "_oai": {"id": "oai:meral.edu.mm:recid/6300", "sets": ["user-uit"]}, "communities": ["uit"], "item_1583103067471": {"attribute_name": "Title", "attribute_value_mlt": [{"subitem_1551255647225": "Evaluation of Apache Kafka in Real-Time Big Data Pipeline Architecture", "subitem_1551255648112": "en"}]}, "item_1583103085720": {"attribute_name": "Description", "attribute_value_mlt": [{"interim": "Today, many applications based on real-time analytics\nneed to enable time-critical decision with real-time\nrequirements and process with reliability requirements.\nModern distributed systems are growing exponentially as\nfar as performance and scale. The main purpose of Big\nData real-time processing is to realize an entire system\nthat can process such mesh data in a short time. And the\nperformance of processing time can be guaranteed in a\nsatisfactory range. To develop a distributed data pipeline,\nthe system proposes real-time big data pipeline by using\nApache Kafka and Apache Storm. Apache Kafka is\ncurrently the most popular framework used to ingest the\ndata streams into the processing platforms. However,\nthere are many challenges how to send reliable messages\non many servers. This paper focuses on the comparison\nof the processing time between successful processes and\nfailed processes on many servers. The experimental\nresults show the performance impact on both producer\nand consumer of the Apache Kafka framework."}]}, "item_1583103108160": {"attribute_name": "Keywords", "attribute_value_mlt": [{"interim": "Apache Kafka"}, {"interim": "Apache storm"}, {"interim": "Asynchronous replication"}, {"interim": "Real time processing"}]}, "item_1583103120197": {"attribute_name": "Files", "attribute_type": "file", "attribute_value_mlt": [{"accessrole": "open_access", "date": [{"dateType": "Available", "dateValue": "2020-11-19"}], "displaytype": "preview", "download_preview_message": "", "file_order": 0, "filename": "Evaluation of Apache Kafka in Real-Time Big Data Pipeline Architecture.pdf", "filesize": [{"value": "1.4 Mb"}], "format": "application/pdf", "future_date_message": "", "is_thumbnail": false, "licensefree": "© 2018 ICAIT", "licensetype": "license_free", "mimetype": "application/pdf", "size": 1400000.0, "url": {"url": "https://meral.edu.mm/record/6300/files/Evaluation of Apache Kafka in Real-Time Big Data Pipeline Architecture.pdf"}, "version_id": "b55ec1a3-4d88-4acd-a1a7-eff8a1f2139a"}]}, "item_1583103147082": {"attribute_name": "Conference papers", "attribute_value_mlt": [{"subitem_acronym": "ICAIT-2018", "subitem_c_date": "1-2 November, 2018", "subitem_conference_title": "2nd International Conference on Advanced Information Technologies", "subitem_place": "Yangon, Myanmar", "subitem_session": "Database and Big Data Analytics", "subitem_website": "https://www.uit.edu.mm/icait-2018/"}]}, "item_1583105942107": {"attribute_name": "Authors", "attribute_value_mlt": [{"subitem_authors": [{"subitem_authors_fullname": "Thandar Aung"}, {"subitem_authors_fullname": "Hla Yin Min"}, {"subitem_authors_fullname": "Aung Htein Maw"}]}]}, "item_1583108359239": {"attribute_name": "Upload type", "attribute_value_mlt": [{"interim": "Publication"}]}, "item_1583108428133": {"attribute_name": "Publication type", "attribute_value_mlt": [{"interim": "Conference paper"}]}, "item_1583159729339": {"attribute_name": "Publication date", "attribute_value": "2018-11-02"}, "item_title": "Evaluation of Apache Kafka in Real-Time Big Data Pipeline Architecture", "item_type_id": "21", "owner": "45", "path": ["1605779935331"], "permalink_uri": "http://hdl.handle.net/20.500.12678/0000006300", "pubdate": {"attribute_name": "Deposited date", "attribute_value": "2020-11-19"}, "publish_date": "2020-11-19", "publish_status": "0", "recid": "6300", "relation": {}, "relation_version_is_last": true, "title": ["Evaluation of Apache Kafka in Real-Time Big Data Pipeline Architecture"], "weko_shared_id": -1}

Evaluation of Apache Kafka in Real-Time Big Data Pipeline Architecture

http://hdl.handle.net/20.500.12678/0000006300
b430ea6e-c289-4a62-a5cb-3a185d7978ba
b826e53f-1b2a-45ec-8832-878d3e7b2617
None
Name / File License Actions
Evaluation Evaluation of Apache Kafka in Real-Time Big Data Pipeline Architecture.pdf (1.4 Mb)
© 2018 ICAIT
Publication type
Conference paper
Upload type
Publication
Title
Title Evaluation of Apache Kafka in Real-Time Big Data Pipeline Architecture
Language en
Publication date 2018-11-02
Authors
Thandar Aung
Hla Yin Min
Aung Htein Maw
Description
Today, many applications based on real-time analytics
need to enable time-critical decision with real-time
requirements and process with reliability requirements.
Modern distributed systems are growing exponentially as
far as performance and scale. The main purpose of Big
Data real-time processing is to realize an entire system
that can process such mesh data in a short time. And the
performance of processing time can be guaranteed in a
satisfactory range. To develop a distributed data pipeline,
the system proposes real-time big data pipeline by using
Apache Kafka and Apache Storm. Apache Kafka is
currently the most popular framework used to ingest the
data streams into the processing platforms. However,
there are many challenges how to send reliable messages
on many servers. This paper focuses on the comparison
of the processing time between successful processes and
failed processes on many servers. The experimental
results show the performance impact on both producer
and consumer of the Apache Kafka framework.
Keywords
Apache Kafka, Apache storm, Asynchronous replication, Real time processing
Conference papers
ICAIT-2018
1-2 November, 2018
2nd International Conference on Advanced Information Technologies
Yangon, Myanmar
Database and Big Data Analytics
https://www.uit.edu.mm/icait-2018/
512
256
views
downloads
Views Downloads
BE15
CA2
CN9
DE24
FR2
GB5
HN1
IR1
MM2
NZ74
RS1
RU2
UNKNOWN1
US373

Export

OAI-PMH
  • OAI-PMH DublinCore
Other Formats