Log in
Language:

MERAL Myanmar Education Research and Learning Portal

  • Top
  • Universities
  • Ranking
To
lat lon distance
To

Field does not validate



Index Link

Index Tree

Please input email address.

WEKO

One fine body…

WEKO

One fine body…

Item

{"_buckets": {"deposit": "b826e53f-1b2a-45ec-8832-878d3e7b2617"}, "_deposit": {"created_by": 45, "id": "6300", "owner": "45", "owners": [45], "owners_ext": {"displayname": "", "username": ""}, "pid": {"revision_id": 0, "type": "recid", "value": "6300"}, "status": "published"}, "_oai": {"id": "oai:meral.edu.mm:recid/6300", "sets": ["user-uit"]}, "communities": ["uit"], "item_1583103067471": {"attribute_name": "Title", "attribute_value_mlt": [{"subitem_1551255647225": "Evaluation of Apache Kafka in Real-Time Big Data Pipeline Architecture", "subitem_1551255648112": "en"}]}, "item_1583103085720": {"attribute_name": "Description", "attribute_value_mlt": [{"interim": "Today, many applications based on real-time analytics\nneed to enable time-critical decision with real-time\nrequirements and process with reliability requirements.\nModern distributed systems are growing exponentially as\nfar as performance and scale. The main purpose of Big\nData real-time processing is to realize an entire system\nthat can process such mesh data in a short time. And the\nperformance of processing time can be guaranteed in a\nsatisfactory range. To develop a distributed data pipeline,\nthe system proposes real-time big data pipeline by using\nApache Kafka and Apache Storm. Apache Kafka is\ncurrently the most popular framework used to ingest the\ndata streams into the processing platforms. However,\nthere are many challenges how to send reliable messages\non many servers. This paper focuses on the comparison\nof the processing time between successful processes and\nfailed processes on many servers. The experimental\nresults show the performance impact on both producer\nand consumer of the Apache Kafka framework."}]}, "item_1583103108160": {"attribute_name": "Keywords", "attribute_value_mlt": [{"interim": "Apache Kafka"}, {"interim": "Apache storm"}, {"interim": "Asynchronous replication"}, {"interim": "Real time processing"}]}, "item_1583103120197": {"attribute_name": "Files", "attribute_type": "file", "attribute_value_mlt": [{"accessrole": "open_access", "date": [{"dateType": "Available", "dateValue": "2020-11-19"}], "displaytype": "preview", "download_preview_message": "", "file_order": 0, "filename": "Evaluation of Apache Kafka in Real-Time Big Data Pipeline Architecture.pdf", "filesize": [{"value": "1.4 Mb"}], "format": "application/pdf", "future_date_message": "", "is_thumbnail": false, "licensefree": "© 2018 ICAIT", "licensetype": "license_free", "mimetype": "application/pdf", "size": 1400000.0, "url": {"url": "https://meral.edu.mm/record/6300/files/Evaluation of Apache Kafka in Real-Time Big Data Pipeline Architecture.pdf"}, "version_id": "b55ec1a3-4d88-4acd-a1a7-eff8a1f2139a"}]}, "item_1583103147082": {"attribute_name": "Conference papers", "attribute_value_mlt": [{"subitem_acronym": "ICAIT-2018", "subitem_c_date": "1-2 November, 2018", "subitem_conference_title": "2nd International Conference on Advanced Information Technologies", "subitem_place": "Yangon, Myanmar", "subitem_session": "Database and Big Data Analytics", "subitem_website": "https://www.uit.edu.mm/icait-2018/"}]}, "item_1583105942107": {"attribute_name": "Authors", "attribute_value_mlt": [{"subitem_authors": [{"subitem_authors_fullname": "Thandar Aung"}, {"subitem_authors_fullname": "Hla Yin Min"}, {"subitem_authors_fullname": "Aung Htein Maw"}]}]}, "item_1583108359239": {"attribute_name": "Upload type", "attribute_value_mlt": [{"interim": "Publication"}]}, "item_1583108428133": {"attribute_name": "Publication type", "attribute_value_mlt": [{"interim": "Conference paper"}]}, "item_1583159729339": {"attribute_name": "Publication date", "attribute_value": "2018-11-02"}, "item_title": "Evaluation of Apache Kafka in Real-Time Big Data Pipeline Architecture", "item_type_id": "21", "owner": "45", "path": ["1605779935331"], "permalink_uri": "http://hdl.handle.net/20.500.12678/0000006300", "pubdate": {"attribute_name": "Deposited date", "attribute_value": "2020-11-19"}, "publish_date": "2020-11-19", "publish_status": "0", "recid": "6300", "relation": {}, "relation_version_is_last": true, "title": ["Evaluation of Apache Kafka in Real-Time Big Data Pipeline Architecture"], "weko_shared_id": -1}
  1. University of Information Technology
  2. International Conference on Advanced Information Technologies

Evaluation of Apache Kafka in Real-Time Big Data Pipeline Architecture

http://hdl.handle.net/20.500.12678/0000006300
http://hdl.handle.net/20.500.12678/0000006300
b430ea6e-c289-4a62-a5cb-3a185d7978ba
b826e53f-1b2a-45ec-8832-878d3e7b2617
None
Preview
Name / File License Actions
Evaluation Evaluation of Apache Kafka in Real-Time Big Data Pipeline Architecture.pdf (1.4 Mb)
© 2018 ICAIT
Publication type
Conference paper
Upload type
Publication
Title
Title Evaluation of Apache Kafka in Real-Time Big Data Pipeline Architecture
Language en
Publication date 2018-11-02
Authors
Thandar Aung
Hla Yin Min
Aung Htein Maw
Description
Today, many applications based on real-time analytics
need to enable time-critical decision with real-time
requirements and process with reliability requirements.
Modern distributed systems are growing exponentially as
far as performance and scale. The main purpose of Big
Data real-time processing is to realize an entire system
that can process such mesh data in a short time. And the
performance of processing time can be guaranteed in a
satisfactory range. To develop a distributed data pipeline,
the system proposes real-time big data pipeline by using
Apache Kafka and Apache Storm. Apache Kafka is
currently the most popular framework used to ingest the
data streams into the processing platforms. However,
there are many challenges how to send reliable messages
on many servers. This paper focuses on the comparison
of the processing time between successful processes and
failed processes on many servers. The experimental
results show the performance impact on both producer
and consumer of the Apache Kafka framework.
Keywords
Apache Kafka, Apache storm, Asynchronous replication, Real time processing
Conference papers
ICAIT-2018
1-2 November, 2018
2nd International Conference on Advanced Information Technologies
Yangon, Myanmar
Database and Big Data Analytics
https://www.uit.edu.mm/icait-2018/
Back
0
0
views
downloads
See details
Views Downloads

Versions

Ver.1 2020-11-19 17:25:31.288305
Show All versions

Share

Mendeley Twitter Facebook Print Addthis

Export

OAI-PMH
  • OAI-PMH DublinCore
Other Formats
  • JSON

Confirm


Back to MERAL


Back to MERAL