by Did you ever encounter a situation where you needed to just make a small change in a file? Perhaps even quite often, to the point that you want to automate this task? A basic script written in the Python programming language does the trick. You just need to know how to read a text file line by line, using the Python programming language. This Python tutorial explains exactly that: How to read a text file line by line.
Background
File manipulation is a skill every software developer should master sooner or later. Data storage and data exchange often takes place through text files. Think of XML, YML and INI files for storing configurations and settings. Or JSON files for data exchange in web applications. Plenty of libraries exist for handling these common file formats.
However, you’ll probably encounter custom and proprietary file formats throughout your career as well. For manipulating these types of file formats in Python, you’ll need to know how to read, modify and write the file contents line by line. In this tutorial you’ll learn how to use Python to both read and write a file, line by line. Basically copying a file. Once you know how to access individual lines in a file, you can modify text file contents to your heart’s desire.
What do you need
To follow along with the presented examples, you just need a PC with the Python interpreter installed. Since the Python interpreter runs on pretty much all operating systems, you can use Windows, macOS or Linux throughout this article. Because this blog primarily focuses on Linux, that’s what I’ll use. My trusty openSUSE Tumbleweed machine to be exact. Curious about trying out openSUSE Tumbleweed? This article gets you started:
All popular Linux distributions come with Python installed. So theoretically, you just need a text editor for writing your Python script. I personally prefer a bit of a beefier development environment: the PyCharm IDE from JetBrains. They offer a community edition, which you can use free or charge. To install the PyCharm Community edition, I recommend its Flatpak edition as explained in this article:
We’ll need a empty Python script as a starting point for this tutorial. Here’s one to get you started. Save it in a file called myfilecopy.py:
#!/usr/bin/env python3
def main():
# TODO Enter code here..
pass
if __name__ == "__main__":
main()
Creating our source file for testing purposes
Together we’ll build a Python script that copies a text file. For starters, we’ll need an actual text file that we can test our code with. You can use whatever text file you prefer. In the presented examples, I’ll use a text file creatively named testfile.txt. Instead of the standard lorem ipsum placeholder text, I decided to spice it up and use ASCII art generated from the PragmaticLinux logo. Go ahead and create a new text file called testfile.txt with the following contents in it:
⣿⣿⡿⣿⢿⡿⡿⣿⢿⢿⡿⡿⣿⢿⢿⡿⡿⣿⢿⢿⡿⡿⣿⢿⣿⢿⣿⢿⣿⢿⣿⢿⣿⢿⣿⢿⣿⢿⣿⢿⣿⢿⣿⣿⣿ ⣿⣿⠀⠠⠀⠠⠀⠄⠠⠀⠄⠠⠀⠄⠠⠀⠄⠠⠀⠄⠠⠀⢢⠱⡰⡑⡢⢣⠪⡘⡔⡱⡰⢱⢨⢊⢆⢣⠢⡣⢪⠸⣐⣿⣿ ⣿⣿⠀⠂⢈⠀⠂⠐⠀⢂⠐⡀⠂⠐⠀⠂⠐⠀⠂⠐⠀⠂⡘⢌⢢⠣⡱⡑⢕⢅⢣⢪⠸⡨⡢⠣⡊⡆⡣⡱⡡⡣⡒⣿⣿ ⣿⣿⠀⡁⢀⢸⣯⣿⣿⣿⣿⣿⣿⣿⣿⣾⣶⣌⡀⢁⠈⡀⢸⠨⡢⢣⢑⢽⣿⣿⣿⣇⢇⢕⠬⡱⡡⢣⠱⡘⡔⢜⢌⣿⣿ ⣿⣿⠀⠄⢀⢸⣿⣿⣿⡿⠛⠛⠛⠻⠿⣿⣿⣿⣿⡤⠀⠄⢢⠱⡘⡌⢎⢽⣿⣿⣿⡗⡌⡆⢇⢪⠸⡨⡊⡎⢜⢌⢆⣿⣿ ⣿⣿⠀⠂⡀⢸⣿⣿⣿⡧⠁⡀⢁⠐⠀⢹⣿⣿⣿⣯⠀⠂⡘⢌⢪⠸⡨⣻⣿⣿⣿⣏⢆⢣⢱⠡⡣⡱⡘⡌⢎⢢⠱⣿⣿ ⣿⣿⠀⡁⠀⢼⣿⣿⣿⡧⢁⠀⠄⡀⣡⣮⣿⣿⣿⡗⠀⡁⢸⢐⢅⢇⠕⣽⣿⣿⣿⣇⠇⡕⢜⢸⠨⡢⢣⠱⡑⡅⢇⣿⣿ ⣿⣿⠀⠄⠂⢸⣿⣿⣿⣿⣷⣿⣿⣿⣿⣿⣿⡿⡛⠀⠂⡀⢢⢑⢌⢆⠣⣻⣿⣿⣿⡧⡣⡱⡑⡅⢇⢎⠪⡪⡘⡌⢎⣿⣿ ⣿⣿⠀⠂⠐⢸⣿⣿⣿⡿⠻⠛⡛⠛⠙⢉⠁⠄⠠⠈⠀⠄⡘⢄⢇⢪⢊⢾⣿⣿⣿⡧⡃⡎⡢⢣⠱⡘⡜⢔⠱⣑⠱⣿⣿ ⣿⣿⠀⡁⠐⢸⣿⣿⣿⡗⡀⠄⠠⠈⠠⠀⠄⠂⠐⠈⠠⠀⢸⢐⠥⡑⡅⣿⣿⣿⣿⣗⡕⣜⣌⢎⣪⡪⣸⣨⠪⡢⡩⣿⣿ ⣿⣿⠀⠄⠂⢸⣿⣿⣿⡗⡀⠐⠀⠂⠐⠀⠂⢈⠀⡁⠐⠈⢰⠡⡪⡊⢆⢿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⢱⢘⠬⣿⣿ ⣿⣿⠀⠂⡀⠉⠋⠋⠙⠑⢀⠈⡀⢁⠈⡀⢁⠠⠀⠄⢈⠠⢘⠔⡱⡘⡔⢝⢫⢛⢝⢫⢛⢝⢫⢛⢝⢫⢛⢝⠪⡢⢣⣿⣿ ⣿⣿⠀⡁⠠⠈⢀⠁⡈⠠⠀⠄⠠⠀⠄⠠⠀⠄⠂⠠⠀⠄⠸⡈⡆⡣⡪⡊⡎⢜⢌⢆⢣⠱⡡⢣⢑⢅⢣⢑⠕⡅⡕⣿⣿ ⣿⣿⠀⠄⠐⠈⠀⠄⠠⠐⠀⠂⠐⠀⠂⠐⠀⠂⠈⠠⠐⠀⢱⠨⡊⢆⠕⡜⢌⢆⠣⡪⠢⡃⡇⢕⢅⢣⠱⡡⢣⠱⡘⣿⣿ ⣿⣿⣿⣾⣿⣿⣷⣿⣾⣿⣾⣿⣿⣻⣿⣻⣿⣻⣿⣾⣿⣷⣷⣿⣿⣿⣿⣿⣿⣾⣿⣿⣿⣿⣾⣿⣷⣿⣿⣿⣿⣿⣿⣿⣿
Store testfile.txt in the same directory as the myfilecopy.py Python script.
Python code to read a text file line by line
Reading a text file one line at a time in Python, consists of three steps:
- Opening the file for reading.
- Looping through all lines in the file.
- Closing the file again.
Opening a file for reading
Let’s start with how to open our testfile.txt file for reading. In the myfilecopy.py Python script, replace the pass line in function main() with:
source_file = open('testfile.txt', 'r')
This opens our file for reading and stores its resulting file object in a variable that I named source_file. You can change this variable name to whatever you prefer. Note that the 'r' parameter informs the open() function that we just want to read from the file.
Looping through all the lines in the file
In Python, the file object returned by the open() function is actually a so-called iterable object. Meaning that we can easily use it in a for-loop to loop (also called iterate) over its contents. To following Python example code, loops over each line in the file and prints the line back on the display:
for line in source_file:
print(line.rstrip())
During each iteration, the loop stores the next line from the file, in a string variable that I named line. Again, you can rename this variable to something else, if you want to. Inside the loop’s body, we print the value currently stored in variable line on the standard output. Each line read from the file ends with a newline character code. I remove this one from the string using its rstrip() function. Simply because the print() function already adds a newline character at the end. Otherwise we would see an empty line after each line we print on the display.
Closing the file
Once you no longer need the file object, you should close it:
source_file.close()
Putting it all together
If you put all these code snippets together and add a few comments, the resulting function main() looks like:
def main():
# Opening the source file in read-only mode
source_file = open('testfile.txt', 'r')
# Loop through the file one line at a time
for line in source_file:
# Display the line on the standard output
print(line.rstrip())
# Close the source file
source_file.close()
Our mytestcopy.py Python script now handles the file opening and closing, reads the file line by line and prints each line on the standard output. The output of the Python script:
In a way you just created a basic version of the Linux cat command, a terminal program to concatenate a file’s contents to the standard output.
Python code to copy a text file line by line
The example in the previous section explained how to read a text file one line at a time. With a little expansion, you can turn it into a Python script that can copy the text file to another text file. This code expansion consists of the following steps:
- Open the destination file for writing.
- Write each line from the source file to the destination file.
- Close the destination file, when no longer needed.
Opening a file for writing
Opening a file for writing resembles opening a file for reading. You just need to replace the 'r' parameter with 'w':
destination_file = open('newfile.txt', 'w')
This creates a new file for writing and stores its resulting file object in a variable that I named destination_file. You can change this variable name to whatever you prefer. Note that if the file already exists, it will be overwritten. If instead you want to append to the existing file, you would use 'a' as the parameter, instead of 'w'.
Write each line to the destination file
Currently, our for-loop reads each line from the source file and prints it on the display. Instead of printing it on the display, we now want to write it to the destination file. To do so, update the for-loop:
for line in source_file:
destination_file.write(line)
As opposed to the print() function, the file object’s write() function does not add an extra newline character at the end of each line. Therefore we do not need to explicitly remove it first with the string’s rstrip() function.
Closing the file
Just like the source file, we should close the destination file, when we no longer need it:
destination_file.close()
Putting it all together
If you put all these code snippets together and add a few comments, the resulting function main() looks like:
def main():
# Opening the source file in read-only mode
source_file = open('testfile.txt', 'r')
# Opening the destination for in write mode
destination_file = open('newfile.txt', 'w')
# Loop through the file one line at a time
for line in source_file:
# Store the line in the destination file
destination_file.write(line)
# Close the destination file
destination_file.close()
# Close the source file
source_file.close()
Our mytestcopy.py Python script can now copy the contents of testfile.txtto a new file called newfile.txt. The following screen offers proof of this:
In a way you just created a basic version of the Linux cp command, a terminal program to copy a file.
Wrap up
In this tutorial you learned how to both read and write a file line by line in Python. A must-have skill for when you need to manipulate contents of a text file. To do the actual manipulation you could use Python’s build-in string functionality to adjust a specific line, before you write it to the destination file.
Follow this link for more Python related tutorials:
