looping object as inputs in BASH, SHELL script

Question

multiple files with the same name grid_200001.grb2.nc with only one components changing and that is the month grid_200010.grb2.nc. I would like to use them all simultaneously as input files

An example of what I am hoping to achieve is

fname="grid_"
year="2010"
month="01 02 03 04 05 06 07 08 09 10"
ext="grb2"
end="nc"

for((y=$year;y<=$year;y++));
do
    for m in $month
    do

    ifile=$fname$y$m.$ext.$end

    >>merge $ifile $ifile ... ofile

example of the desired command i would like to have is

 >> merge grid_200001.grb2.nc grid_200002.grb2.nc ....grid_200012.grb2.nc ofile

I would like all the files i have available as input files inputs at the same time

It would be better if you give more details about how you want it done as I see many possible intentions from your explanation. It's better not to guess. A pseudocode perhaps would do. — konsolebox
– konsolebox, Commented Jul 5, 2014 at 9:46
@konsolebox Thank you for the advice, i re-wrote with an example and hopefully now it is clearer what i want to achieve — George
– George, Commented Jul 5, 2014 at 17:52
ofile would be overwritten in every loop. Is that intended? Also, please indent your merging command properly so we can tell which loop body you want to run it. Why as well is cat not applicable? cat ifile1 ifile2 > ofile — konsolebox
– konsolebox, Commented Jul 5, 2014 at 18:00
@konsolebox I am using a specific toolbox which merges the kind of files that i have, cat unfortunately is not something i can use. the objective is to use all the files i have from previous step as simultaneous inputs and produce one final output (ofile) — George
– George, Commented Jul 5, 2014 at 18:06
I understand. Still it's confusing how you get the sources for merging and where you want to merge them. Can you modify your loop body to make it complete (ended with done) and -indent- it well so we'd know more how you intend to merge it? As you see your assignment with ifile and your merge command is aligned with the inner for loop. — konsolebox
– konsolebox, Commented Jul 5, 2014 at 18:09

konsolebox · Accepted Answer · 2014-07-10 09:52:19Z

1

fname="grid_"
years=({2000..2010})
months=(01 02 03 04 05 06 07 08 09 10)  ## ({01..10}) should be the same
ext="grb2"
end="nc"

shopt -s extglob
IFS='|' eval 'pattern="${fname}@(${years[*]})@(${months[*]}).${ext}.${end}"'
echo "pattern = ${pattern}"  ## for curiosity.
files=($pattern)  ## pathname expansion is sorted in Bash.

merge "${files[@]}" ofile

edited Jul 10, 2014 at 9:52

answered Jul 5, 2014 at 19:08

konsolebox

76.3k13 gold badges110 silver badges114 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

George Over a year ago

To understand the process you use the pattern as "reader" for all the files and you use the wildcard(*) to make sure all the files with the fname are included as inputs (correct??) then in the command of merge i have i can name pattern = file and use them all as simultaneous inputs (?) what does the @ means?

konsolebox Over a year ago

Yes it's a wildcard pattern but an extended form. "{files[@]}" expands properly to multiple arguments. You can do echo merge "${files[@]}" ofile to know how it goes with your command.

Tiago Lopo · Accepted Answer · 2014-07-04 11:12:12Z

0

You can try this:

cat grid_2010[01][0-9].grb2.nc >> ofile

answered Jul 4, 2014 at 11:12

Tiago Lopo

8,0291 gold badge39 silver badges53 bronze badges

3 Comments

George Over a year ago

I dont want to use cat, I am using a toolbox that requires all the same files to be used as simultaneous inputs. The whole thing is a part of a larger script and this is a step, but I am confused on how can i use the multiple files as input at the same time

Tiago Lopo Over a year ago

@George I don't know what your purposes are, but in your script you're are not taking ".grb2." into consideration, have you noticed that?

George Over a year ago

thanks for the notice, i didnt noticed that, i will correct it now. I tried to give the code in simple example and not use my inputs

Emilien · Accepted Answer · 2014-07-04 12:13:10Z

0

I'm not sure I understand the question exactly (edited thanks to your comments)

filenames=""

for year in $(seq 2000 2010)
do 
    for month in $(seq -w 0 10)
    do 
        filename="grid_"$year$month".grb2.nc"
        filenames=$filenames" "$filename
    done
done
your_command $filenames

which may be what you need? cat and wildcards is nice but would it keep the same order?

edited again: But then, the wildcard choice is much simpler your_command grid_20[01][0-9][01][0-9].grb2.nc if you need them only in ascending order (will that always be the case with the wildcards?)

edited Jul 4, 2014 at 12:13

answered Jul 4, 2014 at 11:26

Emilien

2,47517 silver badges24 bronze badges

4 Comments

George Over a year ago

As i mention i have the file produced previously in the code. The name of the files is usually long but I gave the example of the code that i want to create to make it easier to read. Unfortunately if i use wildcards at this point i endanger the process of the process, since usually i have form 20-50 files with similar names.

Tiago Lopo Over a year ago

@Emillien You may use seq -w 0 10 for padding zeros

Emilien Over a year ago

Thanks for the -w tip, it's much nicer! @George I still don't get what you want, do you need to put all these names in a string and in this order such that you can apply on command to all of them? In your question the merge $ifile ... ofile is not clear

George Over a year ago

@Emilien Yes, i would like to take the names of all the files and put them in ascending order after that the command i use will take care of the rest .

Collectives™ on Stack Overflow

looping object as inputs in BASH, SHELL script

3 Answers 3

2 Comments

3 Comments

4 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

3 Comments

4 Comments

Linked

Related