Read File word by word

Question

My problem is to read a text file word by word and recovers every word in a variable. I tried to do:

while read ligne; 
do {
for ( i=1; i<= length ($ligne); i++); do
 { var=$(awk '{print $i}' test.txt)}

}done < test.txt

But it doesn't work and I have this error:

Couldn't parse this for loop

Answer 1

It depends on how you define words.

If words are separated by one or more spaces, you can do:

tr -s '[:blank:]' '[\n*]' < file |
  while IFS= read -r word; do
    : echo "$word" here
  done

If words are sequences of characters contains A-Z, a-z and _:

tr -cs 'A-Za-z_' '[\n*]' < file | ...

On historical System V systems, you need to use square brackets [A-Za-z_].

Answer 2

Just

while read -ra line; 
do
    for word in "${line[@]}";
    do
        echo "$word";
    done;
done < test.txt

will split up the file word by word. Change the echo to whatever you want to do with the words.

semicolons are added so this can be put into a one-liner.

Answer 3

You will receive list of "words" which was been separated by spaces but with punctuation marks enclosed:

while read -a tmp_var; 
do
    for i in "${tmp_var[@]}"
    do
        var[${#var[*]}=$i
    done
done < test.txt

But, as usual, test.txt has been transformed by tr or sed or etc. in "1 word in line" list by tr or sed or etc. and read line by line.

Answer 4

A simpler way, using whitespace as separator and all else as a word,

set -o noglob
words=($(cat text_file)) # use split+glob operator with glob disabled
                         # above. Splits on space tab and newline with
                         # the default value of $IFS.

If that words contain punctuactions and punctuations are words don't cause much trouble you could try this way.

Answer 5

read -a WORDS -d "" < file.txt
for word in "${WORDS[@]}"
do
        echo $word
done

Option -a stores the words in an array.

Option -d specifies the delim string. This tells read where to stop processing the file. I've specified an empty string which causes read to continue until it gets an EOF. That is, the whole file is processed regardless of line endings.

Answer 6

Considering that you used awk in your question, here's another alternative to tr and fmt:

awk '{ for ( i = 1; i < NF; ++i ) print $(i); }' test.txt |
while IFS= read -r var 
do 
    echo processing: "$var" 
done

Note that, as with fmt and unlike tr, awk accepts input filenames as arguments.

Answer 7

I have written the following sample script which reads file /etc/passwd word by word.

#!/bin/bash

COUNT=1

FCOUNT=`cat /etc/passwd|wc -l`

while [ $COUNT -le $FCOUNT ]
do
    FTCOUNT=`awk -F ":" '{print NF,$0}' /etc/passwd|awk '{print $1}'|head -$COUNT|tail -1`
    TCOUNT=1
    while [ $TCOUNT -le $FTCOUNT ]
    do
            if [ $TCOUNT -gt $FTCOUNT ]
            then
                    FTCOUNT=""
                    break
            else
                    OUTPUT=`head -$COUNT /etc/passwd|cut -d ":" -f $TCOUNT|tail -1`
                    echo -n "${OUTPUT} "
                    sleep 2
                    TCOUNT=$(( TCOUNT + 1 ))
            fi
    done
    echo ""
    COUNT=$(( COUNT + 1 ))
done