Linux bash read lines from file to script

Question

I would like to create a bash script, but there are some parameters I want to be configurable. I have a main bash.sh script where I want to load in some commands. So far this is what the the original file looks like:

#!/bin/bash

source values.txt

for i in $(ls 3_TRIMMED/*_R1.* | sort -u); do echo STAR --genomeDir $ref_genome \
--readFilesIn ${i} ${i/_R1./_R2.} \
--runThreadN $runThread --outFileNamePrefix 5_ALIGNED/${i} \
--sjdbGTFfile $gtf_file \
--readFilesCommand gunzip -c ; done

Where $ref_genome $runThread $gtf_file is coming from a config file, with the help of source. So it turned out that I need more config options and with that way its not possible. So I worked around and made another file containing the command line I want to run in the loop:

>STAR
STAR --genomeDir $ref_genome --readFilesIn ${i} ${i/_R1./_R2.} --runThreadN $runThread --outFileNamePrefix 5_ALIGNED/${i} --sjdbGTFfile $gtf_file --readFilesCommand gunzip -c
STAR>

and a script that reads that line of command from the file:

star="$( awk '/>STAR/{flag=1; next} /STAR>/{flag=0} flag' test.sh)"

This reads the command in between the STAR tags and saves it as a string.

My problem is that when I use that string in the for loop, like:

for i in $(ls 3_TRIMMED/*_R1.* | sort -u); do echo $star; done

it creates the loop but the values what supposed to be coming from the source are not replaced:

STAR --genomeDir $ref_genome --readFilesIn ${i} ${i/_R1./_R2.} --runThreadN $runThread --outFileNamePrefix 5_ALIGNED/${i} --sjdbGTFfile $gtf_file --readFilesCommand gunzip -c
STAR --genomeDir $ref_genome --readFilesIn ${i} ${i/_R1./_R2.} --runThreadN $runThread --outFileNamePrefix 5_ALIGNED/${i} --sjdbGTFfile $gtf_file --readFilesCommand gunzip -c
STAR --genomeDir $ref_genome --readFilesIn ${i} ${i/_R1./_R2.} --runThreadN $runThread --outFileNamePrefix 5_ALIGNED/${i} --sjdbGTFfile $gtf_file --readFilesCommand gunzip -c

Is there a way to load in the values like in my original script?

Thank you!

There are way to many errors and bad practices in your scrpt to list them all here. Missing double quotes around variable expansion, parsing the output of ls is invalid... Check your shell scripts with shellcheck.net It is a valuable tool to navigate the pitfalls and complexity of writing shell scripts and having these scripts do what they were intended to do, and only that. — Léa Gris
– Léa Gris, Commented Aug 3, 2022 at 8:58
Your whole approach sounds pretty confusing to me, but - after following the suggestion by Léa Gris -, one approach you can do is to make the variables you are going to replace enviroment variables (i.e. export them) and then use the ecommand envsubst to trigger the substitiution inside your $star string. But perhaps an even better solution would be to take some time and design your complete script from scratch.... — user1934428
– user1934428, Commented Aug 3, 2022 at 9:21

Léa Gris · Accepted Answer · 2022-08-03 09:27:46Z

I'd suggest you cleanup your script with proper quoting and using intermediate variables to process file names and paths:

#!/bin/bash

source values.txt

trimmed_dir='./3_TRIMMED'
aligned_dir='./5_ALIGNED'

# Iterates _R1 file paths in trimmed_dir
for r1_path in "${trimmed_dir}/"*_R1.*; do

  # Trims leading path to get file name only
  r1_file="${r1_path##*/}"

  # Trims trailing .extension to get base name
  r1_base_name="${r1_file%.*}"

  # Trims leading base name to get extension 
  r1_ext="${r1_file#*.}"

  # Gets base name of R2 by replacing
  # trailing _R1 from r1 base name with _R2
  r2_base_name="${r1_base_name%_R1}_R2"

  # Adds .extension to r2 file
  r2_file="${r2_base_name}.${r1_ext}"

  # Concatenates base trimmed_dir and r2 file name to get r2 path
  r2_path="${trimmed_dir}/${r2_file}"

  # Creates out-file prefix from r1 base name
  out_file_name_prefix="${aligned_dir}/${r1_base_name}"

  # Do STAR stuffs
  echo STAR \
    --genomeDir "$ref_genome" \
    --readFilesIn "${r1_path}" "${r2_path}" \
    --runThreadN "$runThread" \
    --outFileNamePrefix "${out_file_name_prefix}" \
    --sjdbGTFfile "$gtf_file" \
    --readFilesCommand gunzip -c
done

Collectives™ on Stack Overflow

Linux bash read lines from file to script

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related