awk multiple csv file skip headers

Question

I am concatenating multiple csv files with awk and modifying the rows but i do not want the headers to print i am taking care of the headers in another method i just want awk to skip the headers

here is my code

OutFileName="Final.csv"                       # Fix the output name
i=0                                       # Reset a counter
for filename in ./*.csv; do 
 if [ "$filename"  != "$OutFileName" ] ;      # Avoid recursion 
 then 
   if [[ $i -eq 0 ]] ; then 
      head -1  $filename >   $OutFileName # Copy header if it is the first file
   fi
   text=$(echo $filename | cut -d '_' -f 2-) #spilliting the string to get the date 
   text=$(echo $text | cut -d '.' -f -1)
   awk -F',' -v txt="$text" 'FNR>1||$0=txt" "$1 FS $0' OFS=',' $filename | cat >> $OutFileName # cocatinating with awk 
   i=$(( $i + 1 )) #increse number
 fi
done

it gives an output like this

Trialtime   type    Track   time    Notes   Athlete 
Trialtime   type    Track   time    Notes   Athlete 
20170101 07:15:00   07:15:00    Warmup  ABC 85.2    Initial warmup  Jon
20170101 07:45:00   07:45:00    Sprint1 ABC 59.44   First Sprint    Jon
20170101 08:30:00   08:30:00    TRIAL   ABC 57.21   Final attempt   Jon
20170101 08:00:00   08:00:00    Warmup  ABC 120.51  Initial warmup  Bill
20170101 08:40:05   08:40:05    Sprint1 ABC 61.35   First Sprint    Bill
20170101 09:15:00   09:15:00    Sprint2 ABC 60.08   Second Sprint   Bill
20170101 10:30:00   10:30:00    TRIAL   ABC 60.37   Final attempt   Bill
20170101 07:15:00   07:15:00    Warmup  ABC 85.2    Initial warmup  Jon
20170101 07:45:00   07:45:00    Sprint1 ABC 59.44   First Sprint    Jon
20170101 08:30:00   08:30:00    TRIAL   ABC 57.21   Final attempt   Jon
20170101 08:00:00   08:00:00    Warmup  ABC 120.51  Initial warmup  Bill
20170101 08:40:05   08:40:05    Sprint1 ABC 61.35   First Sprint    Bill
20170101 09:15:00   09:15:00    Sprint2 ABC 60.08   Second Sprint   Bill
20170101 10:30:00   10:30:00    TRIAL   ABC 60.37   Final attempt   Bill
Trialtime   type    Track   time    Notes   Athlete 
201701023 07:15:00  07:15:00    Warmup  ABC 85.2    Initial warmup  Jon
201701023 07:45:00  07:45:00    Sprint1 ABC 59.44   First Sprint    Jon
201701023 08:30:00  08:30:00    TRIAL   ABC 57.21   Final attempt   Jon
201701023 08:00:00  08:00:00    Warmup  ABC 120.51  Initial warmup  Bill
201701023 08:40:05  08:40:05    Sprint1 ABC 61.35   First Sprint    Bill
201701023 09:15:00  09:15:00    Sprint2 ABC 60.08   Second Sprint   Bill
201701023 10:30:00  10:30:00    TRIAL   ABC 60.37   Final attempt   Bill
201701023 07:15:00  07:15:00    Warmup  ABC 85.2    Initial warmup  Jon
201701023 07:45:00  07:45:00    Sprint1 ABC 59.44   First Sprint    Jon
201701023 08:30:00  08:30:00    TRIAL   ABC 57.21   Final attempt   Jon
201701023 08:00:00  08:00:00    Warmup  ABC 120.51  Initial warmup  Bill
201701023 08:40:05  08:40:05    Sprint1 ABC 61.35   First Sprint    Bill
201701023 09:15:00  09:15:00    Sprint2 ABC 60.08   Second Sprint   Bill
201701023 10:30:00  10:30:00    TRIAL   ABC 60.37   Final attempt   Bill

you can see awk is taking the headers each time for each csv file is there a way to ignore the headers ?

Nahuel Fouilleul · Accepted Answer · 2017-08-17 13:02:40Z

2

changing || with {} and using explicit print

awk -F',' -v txt="$text" 'FNR>1 { print txt" "$1 FS $0 }'

answered Aug 17, 2017 at 13:02

Nahuel Fouilleul

19.5k2 gold badges36 silver badges37 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Marcos Oliveira · Accepted Answer · 2017-08-17 13:00:58Z

In your code, just add NR!=1 &&:

OutFileName="Final.csv"                       # Fix the output name
i=0                                       # Reset a counter
for filename in ./*.csv; do 
 if [ "$filename"  != "$OutFileName" ] ;      # Avoid recursion 
 then 
   if [[ $i -eq 0 ]] ; then 
      head -1  $filename >   $OutFileName # Copy header if it is the first file
   fi
   text=$(echo $filename | cut -d '_' -f 2-) #spilliting the string to get the date 
   text=$(echo $text | cut -d '.' -f -1)
   awk -F',' -v txt="$text" 'FNR>1||NR!=1 && $0=txt" "$1 FS $0' OFS=',' $filename | cat >> $OutFileName # cocatinating with awk 
   i=$(( $i + 1 )) #increse number
 fi
done

This will skip the first line.

Collectives™ on Stack Overflow

awk multiple csv file skip headers

2 Answers 2

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Related