Split input file into X parts of size total lines / X

Question

Currently, my bash script splits by number of lines. However, I'd like to split a file into X pieces, each of those having total lines equal to the file length divided by X. The script is run as follows:

./script.sh input_file.tsv

So far, in the script, I have this:

INPUT_FILE=$1
SPLIT_NUM_THREADS=15
TOTAL_LINES=$(wc -l < $INPUT_FILE)
SPLIT_NUM=$( echo "scale=6; $TOTAL_LINES / $SPLIT_NUM_THREADS" | bc)

The following issues exist:

• Using $INPUT_FILE to get TOTAL_LINES gets me the error "ambiguous redirect", but using simply "input.tsv" does not. What's wrong there?
• SPLIT_NUM is a float, how do I convert it to an int so it can split by lines?

How can I resolve these issues and split a file by number of pieces?

Answer 1

Each part gets the integer divide ($((a/b))). If the line number modulo the number of parts ($((a%b))) is not zero then you have to distribute the spare modulo number over the parts. One solution is to give the modulo value number of parts an additional line.

SPLIT_NUM_THREADS=15
TOTAL_LINES=52
for((i=0;i<$((TOTAL_LINES%SPLIT_NUM_THREADS));i++)); do
  echo $((TOTAL_LINES/SPLIT_NUM_THREADS+1))
done
4
4
4
4
4
4
4
for((i=$((TOTAL_LINES%SPLIT_NUM_THREADS));i<SPLIT_NUM_THREADS;i++)); do
  echo $((TOTAL_LINES/SPLIT_NUM_THREADS))
done
3
3
3
3
3
3
3
3