Python File Operations- A Comprehensive Guide with Examples

Python, one of the world's most popular programming languages, offers a powerful set of tools for file handling. These tools allow developers to open, read, write, and append to files, providing the ability to interact with stored data in an easy and efficient manner. In this guide, we'll dive into Python's file operations, providing plenty of code examples to illustrate these essential processes.

Python File Opening Modes

In Python, you can open a file using the open() function. This function requires at least one argument: the path to the file. You can also specify the mode in which you want to open the file. Here are the various modes you can use:

'r': Read-only mode. The file must exist.
'w': Write mode. If the file exists, it will be overwritten. If it does not exist, it will be created.
'a': Append mode. The file is opened for writing, but data will be appended to the end of the file rather than overwriting existing content.
'x': Exclusive creation mode. The function will fail if the file already exists.
'b': Binary mode.
't': Text mode.
'+': Reading and writing mode.

You can combine these modes, for example 'rb' for reading in binary mode or 'w+' for reading and writing.

# Open a file for reading
file = open("example.txt", 'r')

# Open a file for writing
file = open("example.txt", 'w')

# Open a file for appending
file = open("example.txt", 'a')

# Open a file for reading and writing
file = open("example.txt", 'r+')

After you're done with a file, it's important to close it using the close() function. This frees up resources tied to the file.

file.close()

File Operations: Reading and Writing

In Python, you can read from a file using the read(), readline(), or readlines() methods, and you can write to a file using the write() or writelines() methods.

# Reading from a file
file = open("example.txt", 'r')
content = file.read()
print(content)
file.close()

# Writing to a file
file = open("example.txt", 'w')
file.write("Hello, Python!")
file.close()

In many scenarios, using the with keyword is recommended while working with files. This ensures the file is properly closed after its suite finishes, even if an exception is raised.

# Reading from a file
with open("example.txt", 'r') as file:
    print(file.read())

# Writing to a file
with open("example.txt", 'w') as file:
    file.write("Hello, Python!")

Exercises for Reading and Writing txt files in Python

Here is the code under each exercise:

Exercise 1: Write Text to a File

Create a Python script that writes the following lines to a file named "hello.txt":

"Hello, World!"
"Python is great for handling files."
"Let's learn Python."

with open('hello.txt', 'w') as f:
    f.write("Hello, World!\nPython is great for handling files.\nLet's learn Python.\n")

Exercise 2: Read Text from a File

Now, create a Python script that reads the "hello.txt" file created in Exercise 1 and prints its contents to the console.

with open('hello.txt', 'r') as f:
    print(f.read())

Exercise 3: Count the Lines in a File

Write a Python script that counts the number of lines in "hello.txt".

with open('hello.txt', 'r') as f:
    print(len(f.readlines()))

Exercise 4: Append Text to a File

Write a Python script that appends the following line to "hello.txt":

"This line was appended."

with open('hello.txt', 'a') as f:
    f.write("This line was appended.\n")

Exercise 5: Read and Modify a File

Write a Python script that reads "hello.txt", replaces the word "Python" with "Programming", and writes the modified content back to "hello.txt".

with open('hello.txt', 'r') as f:
    content = f.read()

content = content.replace("Python", "Programming")

with open('hello.txt', 'w') as f:
    f.write(content)

Exercise 6: Copy the Contents from One File to Another

Write a Python script that copies the contents of "hello.txt" to a new file named "copy.txt".

with open('hello.txt', 'r') as f:
    content = f.read()

with open('copy.txt', 'w') as f:
    f.write(content)

These exercises will give you hands-on experience with some of the most common operations you can perform on text files in Python.

Advanced File Operations: CSV and JSON

Python provides libraries to handle CSV and JSON files, two commonly used file formats for data storage.

CSV Operations in Python

You can use the csv module in Python to read and write CSV files.

import csv

# Reading from a CSV file
with open('example.csv', 'r') as file:
    reader = csv.reader(file)
    for row in reader:
        print(row)

# Writing to a CSV file
with open('example.csv', 'w', newline='') as file:
    writer = csv.writer(file)
    writer.writerow(["id", "name", "age"])
    writer.writerow([1, "John Doe", 23])

JSON Operations in Python

You can use the json module in Python to parse JSON files.

import json

# Reading from a JSON file
with open('example.json', 'r') as file:
    data = json.load(file)
    print(data)

# Writing to a JSON file
with open('example.json', 'w') as file:
    json.dump({"name": "John Doe", "age": 23}, file)

CSV and JSON exercises

Here are some exercises for CSV and JSON file operations:

Exercise 1: Write to a CSV File

Create a Python script that writes the following data into a CSV file named "students.csv":

Name, Age, Grade
John, 15, 10
Jane, 16, 11
Bob, 15, 10

import csv

header = ['Name', 'Age', 'Grade']
data = [['John', 15, 10], ['Jane', 16, 11], ['Bob', 15, 10]]

with open('students.csv', 'w', newline='') as f:
    writer = csv.writer(f)
    writer.writerow(header)
    writer.writerows(data)

Exercise 2: Read from a CSV File

Now, create a Python script that reads the "students.csv" file created in Exercise 1 and prints its contents to the console.

import csv

with open('students.csv', 'r') as f:
    reader = csv.reader(f)
    for row in reader:
        print(', '.join(row))

Exercise 3: Convert CSV to JSON

Write a Python script that reads the "students.csv" file and converts it to a JSON file named "students.json".

import csv
import json

with open('students.csv', 'r') as f:
    reader = csv.DictReader(f)
    data = list(reader)

with open('students.json', 'w') as f:
    json.dump(data, f)

Exercise 4: Write to a JSON File

Write a Python script that writes the following data to a JSON file named "data.json":

data = {
    "name": "John",
    "age": 30,
    "city": "New York"
}

import json

data = {
    "name": "John",
    "age": 30,
    "city": "New York"
}

with open('data.json', 'w') as f:
    json.dump(data, f)

Exercise 5: Read from a JSON File

Create a Python script that reads the "data.json" file created in Exercise 4 and prints the data to the console.

import json

with open('data.json', 'r') as f:
    data = json.load(f)

print(data)

These exercises will provide you hands-on experience with reading, writing, and converting data between CSV and JSON formats using Python.

Working with Excel and Pandas

Python, with libraries such as pandas and openpyxl, can interact with Excel files. pandas makes it straightforward to read from and write to Excel files.

import pandas as pd

# Reading from an Excel file
df = pd.read_excel('example.xlsx')

# Writing to an Excel file
df.to_excel('example.xlsx', index=False)

Excel and Pandas exercise

Exercises on using the Pandas library for Excel file operations:

Exercise 1: Write Data to Excel File

Create a Python script using pandas that writes the following data into an Excel file named "employees.xlsx":

data = {
    "Name": ["John", "Jane", "Bob"],
    "Age": [30, 25, 45],
    "Salary": [50000, 60000, 70000]
}

Solution:

import pandas as pd

data = {
    "Name": ["John", "Jane", "Bob"],
    "Age": [30, 25, 45],
    "Salary": [50000, 60000, 70000]
}

df = pd.DataFrame(data)

df.to_excel("employees.xlsx", index=False)

Exercise 2: Read Data from an Excel File

Create a Python script that reads the "employees.xlsx" file created in Exercise 1 using pandas and prints its contents to the console.

Solution:

import pandas as pd

df = pd.read_excel("employees.xlsx")

print(df)

Exercise 3: Add a New Column to the Excel Data

Write a Python script that reads the "employees.xlsx" file, adds a new column "Experience" with values [5, 3, 10] respectively, and writes the updated data back to "employees.xlsx".

Solution:

import pandas as pd

df = pd.read_excel("employees.xlsx")

df["Experience"] = [5, 3, 10]

df.to_excel("employees.xlsx", index=False)

Exercise 4: Filter Data Based on a Condition

Write a Python script that reads "employees.xlsx" and prints only those rows where "Salary" is greater than 55000.

Solution:

import pandas as pd

df = pd.read_excel("employees.xlsx")

filtered_df = df[df["Salary"] > 55000]

print(filtered_df)

These exercises and their corresponding solutions should provide you with a good understanding of how to use the pandas library to manipulate Excel data in Python.

File Handling in Practice:

Python's file handling capabilities make it suitable for building data-intensive applications.

Student Management System

Let's build simple student management system. In such a system, you could use file operations to read student data, append new student data, search for specific student data, and more.

class StudentManagementSystem:

    def __init__(self, filename):
        self.filename = filename

    def add_student(self, student_data):
        with open(self.filename, 'a') as file:
            file.write(f"{student_data}\n")

    def search_student(self, student_name):
        with open(self.filename, 'r') as file:
            for line in file:
                if student_name in line:
                    return line
        return None

# Usage
sms = StudentManagementSystem('students.txt')
sms.add_student("John Doe, 23, Computer Science")
print(sms.search_student("John Doe"))

Combine the contents of text files in the folder into a single file

Let's consider an example where you have a directory of text files and you want to combine the contents of these text files into a single file.

Firstly, you'll need the os module to interact with the operating system, enabling you to list all files in the directory. Then, you'll use the open function in a loop to read from each file and append its contents to the final output file.

Here's a script that accomplishes this task:

import os

# Directory containing the text files
directory = "text_files"

# Name of the output file
output_file_name = "combined.txt"

# Make sure the output file is clear
with open(output_file_name, 'w') as output_file:
    output_file.write('')

# Get a list of all files in the directory
files = os.listdir(directory)

# Open the output file in append mode
with open(output_file_name, 'a') as output_file:
    # Loop over all files
    for file_name in files:
        # Build the full input path and open the file
        input_path = os.path.join(directory, file_name)
        with open(input_path, 'r') as input_file:
            # Append this file's contents to the output file
            output_file.write(input_file.read())
            output_file.write("\n\n")

print(f"Combined files from {directory} into {output_file_name}")

In this script, we first clear the output file to ensure it's empty. We then get a list of all files in the specified directory. For each file, we build the full path to the file, open the file, read its contents, and append those contents to the output file. We also append two newlines ("\n\n") to ensure that there's a blank line between each file's contents in the output file.

In conclusion, Python offers a robust set of tools for file handling, making it an ideal language for data manipulation and processing tasks. Its simplicity and power make it a top choice among developers for all sorts of file operations.

Published 15 Jul 2023