Manage files in Python at file system level

How to manage files in Python like the create, delete, rename, or move operations? Python programming is a versatile language that can be used for a wide range of tasks. File management is the process of organizing, storing, and manipulating files on a computer or any remote file system. Using Python methods, you can easily automate repetitive file management tasks, such as creating, renaming, deleting, moving, and searching for files.

1. List files in Python

To list all the files in a directory in Python, you can use the os library and its listdir() function. The listdir() function takes the directory path as an argument and returns a list of all the files in that directory. A demonstration of the simplicity in utilizing this function to retrieve a list of all the files in the current directory is illustrated below.

Let’s consider the Windows folder below, it contains 6 files, including csv and txt files.

import os

files = os.listdir('C:\data')
for file in files:
    print(file)

The result of the script is the display of the list of 6 file names.

Indeed, whether it is a Windows or Linux file system, one of the biggest strengths of Python programming is its ability to easily manage files in scripts when it comes to managing and analysing data.

2. Read file attributes in Python

To read file attributes in Python, you can use the os library and its stat() function. The stat() function takes the file path as an argument and returns an object containing the file’s attributes such as creation time, modification time, size, etc.

Here, is this second script example, check how to use this function to read the creation date and of all the text files within a folder.

import os
import time

path = "C:\data"
files = os.listdir(path)
for file in files:

    file_path = os.path.join(path, file)
    file_stats = os.stat(file_path)
    create_time = time.ctime(file_stats.st_ctime)

    print(file)
    print(f'File creation time: {create_time}')

We use the same folder as the previous example, i.e., C:\data. We also use os.path.join() to combine the folder path and the file name. Also, you need to use os.stat(file_path) to get the statistics of the text file. Also, the folder path should be enclosed in double quotes (“).

3. List files recursively in Python

To list all the files in a directory and its subdirectories recursively in Python, you can use the os library and its walk() function. The walk() function takes the directory path as an argument and generates the file names in a directory tree by walking the tree either top-down or bottom-up.

This script uses the walk function to recursively list all the files in a directory and its subdirectories.

import os

for root, dirs, files in os.walk("path/to/directory"):
    for file in files:
        print(os.path.join(root, file))

To list all files using the windows command line, use the dir command and its recursive options.

List files recursively with cmd in folders and subfolders

4. Create files with Python

One of the most useful features when you manage files in Python is of course the file creation. To create a new file in Python, you can use the open() function. This function takes the file name and the mode as arguments. Here’s an example of how you can use this function to create a new file.

with open("example.txt", "w") as file:
    file.write("Example content")

By the way, the open method is well explained with much more details in this tutorial about text files management in Python.

Manage text files in Python using read and write methods

5. Create temporary files in Python

To create a temporary file in Python, you can use the tempfile library and its NamedTemporaryFile() function. This function creates a temporary file with a unique name and returns a file object. Here’s one more script example, this time to use the tempfile function to create a temporary file.

import tempfile
temp = tempfile.NamedTemporaryFile(delete=False)
print(temp.name)

# result: C:\Users\user1\AppData\Local\Temp\tmp7chtz52e

6. Rename files in Python

Renaming files in Python can be done using the os library and its rename() function. This function takes the current file name and the new file name as arguments. Use the following script to easily rename a file, in this case the file does not have any extension.

import os

os.rename("path/to/current/file", "path/to/new/file")

Another way to rename a file is by using the shutil library and its move() function. This function also takes the current file name and the new file name as arguments.

import shutil

shutil.move("path/to/current/file", "path/to/new/file")

Renaming files with Python scripts can be a powerful way to organize and keep your files in order. It’s a simple task but it can save you a lot of time in the long run. With just a few lines of code, you can easily rename multiple files at once, making it a breeze to keep your files organized and easy to find.

Whether you’re working on a personal project or running a business, renaming files is a must-have skill for anyone working with a large amount of data stored in numerous files.

7. Delete files

Deleting files in Python can be done using the os library and its remove() function. This function takes the file name as an argument.

import os

os.remove("path/to/file")

Another way to delete a file is by using the shutil library and its rmtree() function. This function can be used to delete a directory and all its contents. Python deletion operations can be used to remove unnecessary files, like old versions of files, or files that are taking up too much space.

import shutil

shutil.rmtree("path/to/directory")

8. Copy files

Copying files in Python can be done using the shutil library and its copy() function. This function takes the source file name and the destination file name as arguments. Here’s an example of to use this function in a script to copy a file:

import shutil

shutil.copy("path/to/source/file", "path/to/destination/file")

Another way to copy a file is by using the os library and its system() function. This function can be used to run command line commands.

import os

os.system("cp path/to/source/file path/to/destination/file")

Copying files is a crucial task that not only keeps your data safe but also enables you to easily share it across different systems. Using Python methods, you can quickly copy multiple files at once, making it a breeze to keep your files organized and easily accessible.

Indeed, the ability to manage files in Python is an important skill for anyone working with file systems. So go ahead and master the art of copying files with Python and take control of your data.

Another very common way to copy many files at once, is to use MS scripting tools, like MS-DOS or PowerShell. This other tutorial explains how to copy files and folders recursively.

Copy files recursively with cmd and the xcopy command

9. Move files

Moving files in Python can be done using the shutil library and its move() function. This function takes the source file name and the destination file name as arguments. An example of using the function to move a file can be seen here:

import shutil
shutil.move("path/to/source/file", "path/to/destination/file")

Another way to move a file is by using the os library and its rename() function. This function also takes the source file name and the destination file name as arguments.

import os
os.rename("path/to/source/file", "path/to/destination/file")

Moving files is an essential task in Python file management. It allows you to reorganize files, move them to different directories, or even rename them.

10. Search files with Python

Searching for files in Python can be done using the glob library and its glob() function. This function takes a file pattern as an argument and returns a list of file names that match the pattern.

Use for example the function below to search for all the files having a .txt extension in the directory passed as parameter:

import glob

txt_files = glob.glob("path/to/directory/*.txt")
print(txt_files)

Another way to search for files is by using the os library and its scandir() function. This function returns an iterator that yields the files and directories in the specified directory.

import os

for file in os.scandir("path/to/directory"):
    if file.name.endswith(".txt"):
        print(file.name)

Searching for files is an important aspect of file management, it enables you to quickly locate the files you require, at the time you need them. With a minimal amount of code, you can search for multiple files at once, based on various criteria.

For both all IT projects and business operations, the capability to search for files is useful for those working with data. Empower yourself and master the art of managing files using Python.

Conclusion on file management in Python

In conclusion, using Python for managing files is a powerful tool for anyone working with data. Whether you need to list, read, create, rename, delete, copy, move or search for files, Python offers various libraries and functions that make it easy to accomplish these tasks.

From listing all the files in a directory recursively and also in its subdirectories, to creating temporary files, renaming multiple files at once, or searching for files based on different criteria, Python offers a wide range of possibilities for file management. With simple Python scripts, accomplish all these automated tasks and keep files organized and easily accessible.

Once the file management at OS level is acquired, you can start using the file management to read and write data from and into text or csv files.

Manage text files in Python using read and write methods