Catalogue Search | MBRL
Are you sure you want to remove the book from the shelf?
{{itemTitle}}
13,201
result(s) for
"Big data Data processing."
Sort by:
Big Data: Big Data Analysis, Issues and Challenges and Technologies
The data generated at an exponential rate has resulted in Big Data. This data has many characteristics and consists of structured, unstructured, and semi-structured data formats. It contains valuable information for the different types of stakeholders based on their need however it is not possible to meet them with the help of traditional tools and techniques. Here the big data technologies play a crucial role to handle, store, and process this tremendous amount of data in real-time. Big data analytics is used to extract meaningful information or patterns from the voluminous data. It can be further divided into three types i.e. text analytics, audio analytics, video analytics, and social media analytics. Big data analytics if followed by big data analysis process plays a significant role in generating meaningful information from big data. Big data analysis process consists of data acquisition, data storage, data management, data analytics, and finally data visualization. However, it is not simple and brings many challenges that need to be resolved. This paper presents the issues and challenges related to big data, prominent characteristics of big data, big data analytics, big data analysis process, and technologies used for processing the massive data.
Journal Article
Marine big data
\"As the volume of marine big data has increased dramatically, one of the main concerns is how to fully exploit the value of such data in the development of marine economy and marine science and technology. The book covers data acquisition, feature classification, processing and applications of marine big data in evaluation and decision-making, using case studies such as storm surge and marine oil spill disaster\"-- Provided by publisher.
Book
Performance Analysis of IoT-Based Sensor, Big Data Processing, and Machine Learning Model for Real-Time Monitoring System in Automotive Manufacturing
With the increase in the amount of data captured during the manufacturing process, monitoring systems are becoming important factors in decision making for management. Current technologies such as Internet of Things (IoT)-based sensors can be considered a solution to provide efficient monitoring of the manufacturing process. In this study, a real-time monitoring system that utilizes IoT-based sensors, big data processing, and a hybrid prediction model is proposed. Firstly, an IoT-based sensor that collects temperature, humidity, accelerometer, and gyroscope data was developed. The characteristics of IoT-generated sensor data from the manufacturing process are: real-time, large amounts, and unstructured type. The proposed big data processing platform utilizes Apache Kafka as a message queue, Apache Storm as a real-time processing engine and MongoDB to store the sensor data from the manufacturing process. Secondly, for the proposed hybrid prediction model, Density-Based Spatial Clustering of Applications with Noise (DBSCAN)-based outlier detection and Random Forest classification were used to remove outlier sensor data and provide fault detection during the manufacturing process, respectively. The proposed model was evaluated and tested at an automotive manufacturing assembly line in Korea. The results showed that IoT-based sensors and the proposed big data processing system are sufficiently efficient to monitor the manufacturing process. Furthermore, the proposed hybrid prediction model has better fault prediction accuracy than other models given the sensor data as input. The proposed system is expected to support management by improving decision-making and will help prevent unexpected losses caused by faults during the manufacturing process.
Journal Article
A review on COVID-19 forecasting models
The novel coronavirus (COVID-19) has spread to more than 200 countries worldwide, leading to more than 36 million confirmed cases as of October 10, 2020. As such, several machine learning models that can forecast the outbreak globally have been released. This work presents a review and brief analysis of the most important machine learning forecasting models against COVID-19. The work presented in this study possesses two parts. In the first section, a detailed scientometric analysis presents an influential tool for bibliometric analyses, which were performed on COVID-19 data from the Scopus and Web of Science databases. For the above-mentioned analysis, keywords and subject areas are addressed, while the classification of machine learning forecasting models, criteria evaluation, and comparison of solution approaches are discussed in the second section of the work. The conclusion and discussion are provided as the final sections of this study.
Journal Article
DolphinNext: a distributed data processing platform for high throughput genomics
Background
The emergence of high throughput technologies that produce vast amounts of genomic data, such as next-generation sequencing (NGS) is transforming biological research. The dramatic increase in the volume of data, the variety and continuous change of data processing tools, algorithms and databases make analysis the main bottleneck for scientific discovery. The processing of high throughput datasets typically involves many different computational programs, each of which performs a specific step in a pipeline. Given the wide range of applications and organizational infrastructures, there is a great need for highly parallel, flexible, portable, and reproducible data processing frameworks.
Several platforms currently exist for the design and execution of complex pipelines. Unfortunately, current platforms lack the necessary combination of parallelism, portability, flexibility and/or reproducibility that are required by the current research environment. To address these shortcomings, workflow frameworks that provide a platform to develop and share portable pipelines have recently arisen. We complement these new platforms by providing a graphical user interface to create, maintain, and execute complex pipelines. Such a platform will simplify robust and reproducible workflow creation for non-technical users as well as provide a robust platform to maintain pipelines for large organizations.
Results
To simplify development, maintenance, and execution of complex pipelines we created DolphinNext. DolphinNext facilitates building and deployment of complex pipelines using a modular approach implemented in a graphical interface that relies on the powerful Nextflow workflow framework by providing 1. A drag and drop user interface that visualizes pipelines and allows users to create pipelines without familiarity in underlying programming languages. 2. Modules to execute and monitor pipelines in distributed computing environments such as high-performance clusters and/or cloud 3. Reproducible pipelines with version tracking and stand-alone versions that can be run independently. 4. Modular process design with process revisioning support to increase reusability and pipeline development efficiency. 5. Pipeline sharing with GitHub and automated testing 6. Extensive reports with R-markdown and shiny support for interactive data visualization and analysis.
Conclusion
DolphinNext is a flexible, intuitive, web-based data processing and analysis platform that enables creating, deploying, sharing, and executing complex Nextflow pipelines with extensive revisioning and interactive reporting to enhance reproducible results.
Journal Article
Data, now bigger and better!
\"Data is too big to be left to the data analysts! Here, Prickly Paradigm brings together five researchers whose work is deeply informed by anthropology, understood as more than a basket of ethnographic methods like participants observation and interviewing. The value of anthropology lies also in its conceptual frameworks, frameworks that are comparative as well as field-based. Kinship! Gifts! Everything old is new when the anthropological archive washes over 'big data'. Bringing together anthropology's classic debates and contemporary interventions, this book counters the future-oriented hype and speculation so characteristic of discussions regarding big data. By drawing as well on long experience in industry contexts, the contributors provide analytical provocations that can help reframe what may prove to be some of the most important shifts in technology and society in the first half of the twenty-first century\"--Back cover.
Book
An efficient architecture for processing real-time traffic data streams using apache flink
Big Data technologies emerging day by day and are making drastic changes in various real-world applications. Traditional data mining tools adequate to process volumes of data but from past decades the rapid growth in data becomes difficult for processing. Due to continuous flow of data, data streams require additional computational processing than the traditional one. Big data stream processing considers different features of the data streams heterogeneity, scalability, fault tolerance and query optimization. Efficient implementation of these features in real-world applications using big data analytics is a challenging job during data storage, processing, and analysis phases. Therefore, the proposed model FRTSPS is a generic architecture which is influenced by popular big data processing Lambda architecture, based on distributed computing platform. The architecture using open-source platform Apache Flink for doing data processing. Flink is a popular platform for processing historical and stream data flows at once parallelly. Its stateful streaming can obtain more scalability and flexibility along with high throughput and low latency than the remaining stream processing programming models.
Journal Article