Don't use a loop at all. Instead, list all files and use a command like grep or akw to filter out the files between start and stop. printf won't have the problem Argument list too long as it is a bash built-in.
count the total number [of files] that exists between $start and $stop
Count the files
printf '%s\0' /home/me/*/file_*.txt |
grep -cEzf <(seq "$start" "$stop" | sed 's/.*/_&\.txt$/')
Here we assume you have the GNU versions of grep which can handle null bytes. This is important to process paths safely. The null byte is the only character which cannot be part of a path so we can use it to separate paths.
If you don't have the GNU version you can create something that delimits paths by newlines under the assumption that no path contains a newline character. Replace \0 with \n and remove the -z flag.
If you change …_*.txt to something else you may also have to update the sed command.