Concatenating multiple files through multiple folders

Question

Im trying to create a single file out of multiple text files I have across multiple folders. This is my code for concatenating. It works only if the program file is placed in each folder:

        import os

        file_list = [each for each in cur_folder if each.endswith(".txt")]
        print file_list

        align_file = open("all_the_files.txt","w")

        seq_list = []

        for each_file in file_list:
                f_o = open(file_path,"r")
                seq = (f_o.read().replace("\n",""))
                lnth = len(seq)
                wholeseq = ">"+each_file+" | "+str(lnth)+" nt\n"+seq+"\n"
                align_file.write(wholeseq)
                print "done"

Now I tried to edit to make sure that it automatically runs through the entire Data folder and then enters the subdirectories and concatenates all the files without me having to paste the program file in each folder. This is the edit.

    import os

    dir_folder = os.listdir("C:\Users\GAMER\Desktop\Data")

    for each in dir_folder:
            cur_folder = os.listdir("C:\\Users\\GAMER\\Desktop\\Data\\"+each)
            file_list = []

            file_list = [each for each in cur_folder if each.endswith(".txt")]
            print file_list

            align_file = open("all_the_files.txt","w")

            seq_list = []

            for each_file in file_list:

                f_o = open(file_path,"r")
                seq = (f_o.read().replace("\n",""))
                lnth = len(seq)
                wholeseq = ">"+each_file+" | "+str(lnth)+" nt\n"+seq+"\n"
                align_file.write(wholeseq)
                print "done" , cur_folder

However when I run this , I get an error on the first file of the folder saying no such file exists. I can seem to understand why, specifically since it names the file which is not "hardcoded". Any help will be appreciated.

If the code looks ugly to you feel free to suggested better ways to do it.

@goncalopp cat is unix - his filepaths suggest he's on Windows. — furkle
– furkle, Commented Oct 29, 2014 at 17:06

furkle · Accepted Answer · 2014-10-29 17:16:46Z

2

Jamie is correct - os.walk is most likely the function you need.

An example based on your use case:

for root, dirs, files in os.walk(r"C:\Users\GAMER\Desktop\Data"):
    for f in files:
        if f.endswith('.txt'):
            print(f)

This will print the name of every single file within every folder within the root directory passed in os.walk, as long as the filename ends in .txt.

Python's documentation is here: https://docs.python.org/2/library/os.html#os.walk

edited Oct 29, 2014 at 17:16

answered Oct 29, 2014 at 17:08

furkle

5,0591 gold badge18 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Concatenating multiple files through multiple folders

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related