search
Only on Eduzaurus

Hadoop MapReduce

Download essay Need help with essay?
Need help with writing assignment?
74
writers online
to help you with essay
Download PDF

With advancement in Recent technology, gradually mobile devices are replacing traditional personal computers. With gigabytes of memory and multi-core processors nowadays new generations of mobile devices are very powerful. These devices are high-end computing hardware and complex software applications that generate large amounts of data on the order of hundreds of megabytes. This data can vary from application raw data to images, audio, video or text files. With the fast hike in the number of mobile devices, big data processing on mobile devices has become a key emerging necessity for providing capabilities similar to those provided by traditional servers.

Recent mobile applications that can do massive computing tasks (big data processing) offload data and tasks to data centers or powerful servers in the cloud. For processing large datasets, there are numerous cloud services that provides computing infrastructure to end users. Hadoop MapReduce is one of the popular open source programming framework deployed in cloud for cloud computing applications. The framework splits the user job into smaller tasks and runs these tasks in parallel on different nodes, thus decreasing the overall execution time when compared with a sequential execution on a single node. In the case of military or disaster response operations this architecture however, fails in the absence of external network connectivity. This architecture is also not used in emergency response scenarios where there is limited connectivity to cloud, which leads to expensive data upload and download operations. In such scenarios, wireless mobile ad-hoc networks are typically deployed. The drawbacks of the traditional cloud computing motivate us to study the data processing problem in an infrastructure less and mobile environment. In which the internet is unavailable and all jobs are performed on mobile devices. The mobile devices in the vicinity are willing to share each other’s computational resources.

Essay due? We'll write it for you!

Any subject

Min. 3-hour delivery

Pay if satisfied

Get your price

There are lots of challenges in bringing big data capabilities to the mobile environment:

  1. mobile devices are resource-constrained in terms of memory, processing power and energy. However, energy consumption during job execution must be minimized, since most mobile devices are powered with a battery. Energy utilization depends on the complete nodes selected for the job execution. The nodes are selected based on each node’s remaining energy, job retrieval time, and energy profile. As the jobs are retrieved wirelessly, shorter job retrieval time shows lower transmission energy and shorter job completion time. In comparison with the traditional cloud computing, transmission time is the bottleneck for the job make span and wireless transmission is the major source of the energy consumption;
  2. reliability of data is a major challenge in dynamic networks with unpredictable topology changes. Connection failures causes mobile devices to go out of the network reach after limited participation. Device failures can also happens because of energy depletion or application specific failures. Hence, a reliability model stronger than those used by traditional static networks is essential;
  3. security is also a major concern as the stored data often contains sensitive user information. previous security mechanisms tailored for static networks are not fully secure for dynamic networks. If necessary security measures are not provided chances of data leakage and Devices can be captured by unauthorized users. To address these challenges of energy efficiency, reliability and security of dynamic network topologies, the k-out-of-n computing framework was introduced.

Hadoop MapReduce framework over MDFS and evaluate its performance on a general heterogeneous cluster of devices. implementing the generic file system interface of Hadoop for MDFS. which makes system practical similar with other Hadoop frameworks like HBase. Since not requirement of any changes for existing HDFS applications to be deployed over MDFS. Hadoop MapReduce framework for mobile cloud that truly addresses the challenges of the dynamic network environment. The System provides a distributed computing model for processing of large datasets such as unstructured data like media files, text and sensor data in mobile environment while ensuring strong guarantees for energy efficiency, data reliability and security. Thus, system is a viable solution to meet the growing demands of data processing in mobile environment.

Disclaimer

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers. You can order our professional work here.

We use cookies to offer you the best experience. By continuing to use this website, you consent to our Cookies policy.

background

Want to get a custom essay from scratch?

Do not miss your deadline waiting for inspiration!

Our writers will handle essay of any difficulty in no time.