Hadoop Distributed File System - Hadoop Online Training

Outline

1. Introduction
2. Goals for this Module
3. Distributed File System Basics
4. Configuring HDFS
5. Interacting With HDFS
1. Common Example Operations
2. HDFS Command Reference
3. DFSAdmin Command Reference
6. Using HDFS in MapReduce
7. Using HDFS Programmatically
8. HDFS Permissions and Security
9. Additional HDFS Tasks
1. Rebalancing Blocks
2. Copying Large Sets of Files
3. Decommissioning Nodes
4. Verifying File System Health
5. Rack Awareness
10. HDFS Web Interface
11. References

Introduction

HDFS, the Hadoop Distributed File System, is a distributed file system designed to hold very large amounts of data (terabytes or even petabytes), and provide high-throughput access to this information. Files are stored in a redundant fashion across multiple machines to ensure their durability to failure and high availability to very parallel applications. This module introduces the design of this distributed file system and instructions on how to operate it.

Goals for this Module:

• Understand the basic design of HDFS and how it relates to basic distributed file system concepts
• Learn how to set up and use HDFS from the command line
• Learn how to use HDFS in your applications