I need to iterate over two text files and a shell script

Question

My first file screen.txt contains single letters, such as:

d
m
a
o

The second file beta.txt contains a lot of lines:

cvvbbe
etgjiua
qwrfggo

The third file gama.sh is a shell script

I need to loop over the beta.txt file in the following way:

• replace the last letter of every line in beta.txt with the first letter from screen.txt
• save beta.txt
• run gama.sh
• replace the last letter of every line in beta.txt with the second letter from screen.txt
• save beta.txt
• run gama.sh
• and so on

Answer 1

Using any awk:

$ cat gama.sh
#!/usr/bin/env sh

awk '
    NR==FNR {
        beta[++nr] = substr($0,1,length($0)-1)
        next
    }
    {
        for ( i=1; i<=nr; i++ ) {
            print beta[i] $0 > ARGV[1]
        }
        print "" > ARGV[1]
    }
' "$@"

$ ./gama.sh beta.txt screen.txt

$ cat beta.txt
cvvbbd
etgjiud
qwrfggd

cvvbbm
etgjium
qwrfggm

cvvbba
etgjiua
qwrfgga

cvvbbo
etgjiuo
qwrfggo

The above assumes the contents of beta.txt aren't too huge to fit in memory, otherwise if the contents of screen.txt fit in memory then:

$ cat gama.sh
#!/usr/bin/env sh

tmp=$(mktemp) &&
awk '
    BEGIN { OFS="\t" }
    NR==FNR {
        screen[++nr] = $0
        next
    }
    {
        $0 = substr($0,1,length($0)-1)
        for ( i=1; i<=nr; i++ ) {
            print i, FNR, $0 screen[i]
        }
    }
    END {
        for ( i=1; i<=nr; i++ ) {
            print i, FNR+1
        }
    }
' "$@" |
sort -k1,2n |
cut -f3- > "$tmp" &&
mv -- "$tmp" "$2"

$ ./gama.sh screen.txt  beta.txt

$ cat beta.txt
cvvbbd
etgjiud
qwrfggd

cvvbbm
etgjium
qwrfggm

cvvbba
etgjiua
qwrfgga

cvvbbo
etgjiuo
qwrfggo

That second script applies the DSU (Decorate/Sort/Undecorate) idiom to produce output lines in the desired order, see https://stackoverflow.com/questions/71691113/how-to-sort-data-based-on-the-value-of-a-column-for-part-multiple-lines-of-a-f/71694367#71694367 for more details on that.

Answer 2

You can run sed in a loop, like this:

 $ while read letter; do 
    printf '%s\n\n' "$(sed "s/.$/$letter/" beta.txt)";
   done < screen.txt > tmpFile && mv tmpFile beta.txt

$ cat beta.txt
cvvbbd
etgjiud
qwrfggd

cvvbbm
etgjium
qwrfggm

cvvbba
etgjiua
qwrfgga

cvvbbo
etgjiuo
qwrfggo

If you want it as a script, you can just save this as gamma.sh:

#!/bin/sh

tmpFile=$(mktemp)
while read letter; do 
    printf '%s\n\n' "$(sed "s/.$/$letter/" beta.txt)";
done < screen.txt > "$tmpFile" && mv tmpFile 

Note that this will be extremely slow and impractical for large files. It's fine for the dummy example you gave, but if you need to deal with a few thousand or even hundreds of lines, you should use another language. For example, in Perl:

#!/usr/bin/perl
use strict;
use warnings;

my @letters;
open(my $letterFile, '<', "$ARGV[0]") or
    die("Failed to open letter file '$ARGV[0]':$!\n");
while (my $line = <$letterFile>) {
  chomp($line);
  $line =~ /(.)\s*$/;
  push @letters, $1;
}
close($letterFile);

open(my $dataFile, '<', "$ARGV[1]") or
    die("Failed to open data file '$ARGV[1]':$!\n");

my @data = <$dataFile>;
my $c = 0;
foreach my $letter (@letters) {
  print "\n" if $c;
  foreach my $line (@data) {
    $line =~ s/.$/$letter/;
    print "$line";
  }
  $c++;
}
close($dataFile);

If you save the script above as foo.pl, you can do:

$ perl foo.pl screen.txt beta.txt
cvvbbd
etgjiud
qwrfggd

cvvbbm
etgjium
qwrfggm

cvvbba
etgjiua
qwrfgga

cvvbbo
etgjiuo
qwrfggo

Note that this will store all of beta.txt in memory which might be an issue for huge files. If that is a problem, then you will need a different approach but that's beyond the scope of the current question.