How do read the contents of a file in Python and insert these lines into a list?
Using the built-in function
open()
and the asterisk operator
*
a file’s contents can easily be translated into a list with the following one-liner:
[*open('my_file.txt')]
.
What Does
open()
Function Do?
The built-in
open()
function has one required parameter and several optional parameters. The required parameter is the location of the file. If the file is located in the current folder where the Python script is run you can insert the name of the file, for example,
test.txt
.
If the file is located in sub-folders then you’ll need to provide a full path to access the file. You can do relative to where the Python script is (e.g.
folder_name/another_folder/test.txt
), or you can provide an absolute path from the root of your server or computer (e.g.
/usr/ryan/scripts/folder_name/another_folder/test.txt
).
The other parameters that are most notable for this example which can be included in the
open()
function include:
-
mode='r'This is the way in which the file will be opened. If no value for this parameter is provided the file will be opened by default in read-only mode . -
buffering=-1Sets the size of the buffer in bytes. -
encoding=NoneSets the encoding type of the text entered in the file. -
newline=NoneSets the type of newline character to be found in the file.
Here’s how you can use this handy function to extract data from the file into a list.
Get Data From File To List
Using the
open()
function here’s an example of the
basic code
needed to read a file and extract its contents to a list.
The file throughout these examples is labelled
test.txt
and its current contents are:
Hello world
It's me!
Here’s the output from the one line of code that gets the above text into a list, split according to the new line character
\n
:
>>> [*open('test.txt', 'r')]
['Hello world\n', "It's me!"]
Notice with the result above how each line has successfully been inserted as its own list item.
How To Remove Leading And Trailing Spaces On Each Line
You’ll also notice that each line from the text import will contain the trailing newline character
\n
. If you want to remove this from your file import you can expand the initial one-liner a little more into a list comprehension.
Here’s how this code would work if expanded to be a list comprehension, first I’ll break it up to make it easier to understand then I will compress it to one line:
>>> my_lines = [*open('text.txt', 'r')]
>>> my_list = [x.strip() for x in my_lines]
>>> print(my_list)
['Hello world', "It's me!"]
With the code above the initial capture of the lines from the file are stored in a variable labelled
my_lines
. From here the next line is the list comprehension that loops through each element in the initial lines list and applies the string method
.strip()
which removes all the leading and trailing spaces from a string.
Compressing this into one line would look as follows:
[x.strip() for x in [*open('text.txt', 'r')]]
Convert File Into 2D List
The same one liner can be further expanded to convert the contents of the lines into a file into a two dimensional list.
This is most common whenever you’re operating on a csv file.
For the next series of examples the contents of the
test.txt
file are changed to the following:
A,B,C
1,2,3
To get these simple CSV contents into a two-dimensional list you can apply another
minor change
to the working one-liner code to change the contents of each line so that they are
split
individually into cells: and this string method that does this operation is also aptly called
.split()
!
Breaking up each line into its own to explain what happens, would look like this:
>>> my_lines = [*open('test.txt', 'r')]
>>> print(my_lines)
['A,B,C\n', '1,2,3']
>>> my_list = [x.strip() for x in my_lines]
>>> print(my_list)
['A,B,C', '1,2,3']
>>> [x.strip().split(',') for x in my_lines]
[['A', 'B', 'C'], ['1', '2', '3']]
As you can see from the last line of code and its subsequent result the file’s contents have successfully been imported as a two dimensional list.
In one line of code this would look like this:
[x.strip().split(',') for x in [*open('test.txt', 'r')]]
Obviously, it’s quite rudimentary, but it does the job quite nicely.
While there are a couple of issues with this code in that the file object created is never expressly closed or the fact that when you import CSV text containing a comma where that comma is not to be used as a column or field separator it will not work.
For example, if the file had the following valid CSV contents:
A,B,C
"1,000","2,000","3,000"
Using the simple CSV import code above would result in the following output:
[['A', 'B', 'C'], ['"1', '000"', '"2', '000"', '"3', '000"']]
But if the needs are simple then this small code could be sufficient to do the job.
How To Close File
One last important matter is making sure you can close the file you have opened, especially if you’ll be operating on it further.
To enable Python to perform a closure of the opened file once it is used you can perform the same technique as above but enclose it in a with clause as follows:
with open('test.txt', 'r') as f:
my_list = [x.strip().split(',') for x in [*f]]
Or another alternative way that is a little more explicit would be:
>>> f = open('test.txt, 'r')
>>> my_list = [x.strip().split(',') for x in [*f]]
>>> f.close()
Closing files is a good habit to get into with Python and both approaches above would serve you well if added to your Python code.
However, Python will automatically close opened files during its process when the process exits.
Summary
To get the contents of a file into a list using Python use the built-in
open()
function to open the file and then apply the asterisk operator to get the contents of the file, line-by-line parsed. To finish simply wrap it all in the list square brackets and you will have all the contents of the file as a nice string list.
If you further want to perform operations on the individual lines being inserted into the newly created list use a list comprehension and perform any necessary of filter tasks for any basic needs.
