Extracting lines to new files

Question

Say I have a large CSV file with a header and several columns. For the purpose of this question I will consider a small file with just two columns. We can call it use_rep.

user_id,rep
885,500K+
22565,200K+
7453,200K+
86440,100K+
116858,100K+
22222,100K+
38906,100K+
10762,<100K
70524,<100K

I'd like to send each row to a file corresponding to the value on the second column. For example, I'd like there to be a file whose name is 200K+ and whose content is

user_id,rep
22565,200K+
7453,200K+

The contents of use_rep should not be assumed to be ordered in anyway. The pattern to be used would ideally accept regular expressions.

No sed or perl is preferred.

Answer 1

Ignoring the header (which you can tack on later):

awk -F, 'NR > 1 {print > $2}' use_rep

which will print each line to a file named by the second column:

~ head *[0-9]*
==> 100K+ <==
86440,100K+
116858,100K+
22222,100K+
38906,100K+

==> 200K+ <==
22565,200K+
7453,200K+

==> 500K+ <==
885,500K+

==> <100K <==
10762,<100K

To put the header, maybe something like:

awk -F, 'NR == 1 {header = $0; next} # save header, skip this line
  !a[$2]++ { print header > $2 } # print if second field hasnt been seen before 
  { print > $2 }' use_rep

Result:

~ head *[0-9]*
==> 100K+ <==
user_id,rep
86440,100K+
116858,100K+
22222,100K+
38906,100K+

==> 200K+ <==
user_id,rep
22565,200K+
7453,200K+

==> 500K+ <==
user_id,rep
885,500K+

==> <100K <==
user_id,rep
10762,<100K
70524,<100K

Answer 2

Tried with below command and it worked fine

command

for i in `awk -F "," '{print $2}' l.txt | sort | uniq`; do sed -n '/'$i'$/p' l.txt > $i.txt ; done