HDFS Create Directories: hadoop fs -mkdir

Creating directories is an integral part of managing your data within the Hadoop Distributed File System (HDFS). The hadoop fs -mkdir command provides you with the ability to organize your data by establishing new directories.

In this blog post, we will explore the hadoop fs -mkdir command’s usage and its most commonly used flags with examples to help you become proficient in creating directories in HDFS.

Command Syntax:

hadoop fs -mkdir [options] <directory_path>

• directory_path: The path to the directory to be created.
• options: The following options are available:
  • -p: Recursively create the directory, including all its parent directories.
    • Example: hadoop fs -mkdir -p /user/hadoop/mydir

Basic Directory Creation

This basic usage creates a new directory at the specified path in HDFS.

hadoop fs -mkdir <directory_path>

Example:

In this example, a new directory named new_folder is created in the /user/hadoop/ directory within HDFS.

hadoop fs -mkdir /user/hadoop/new_folder

Create Parent Directories (-p)

The -p flag ensures the creation of not only the new directory but also any necessary parent directories along the given path.

hadoop fs -mkdir -p <parent_directory_path>/<new_directory_name>

Example:

Using the -p flag, the command creates a new directory named subfolder within the documents directory. If the documents directory does not exist, the command also creates it.

hadoop fs -mkdir -p /user/hadoop/documents/subfolder