There is a cross-platform set of command line utilities called Dateutils which includes a datediff command. If you are on Linux, there is a good chance you can install dateutils using your package manager.
You haven't stated whether you want this for Linux, Mac, a BSD or some other Unix, so the following might need tweaking for your needs, but it works for me on Arch Linux:
$ paste start.txt end.txt | sed 's/ /T/g' | while read dates; do datediff $dates; done
51s
5s
Explanation
You can think of the pipeline as consisting of two parts: the while read dates; do datediff $dates; done which feeds any input to datediff for calculating the difference; and the paste start.txt end.txt | sed 's/ /T/g' part which preprocesses the raw data into a form suitable for consumption by datediff.
In particular, given the example inputs you've provided, you need to worry about escaping spaces; if the data were left in its raw form, datediff would think each space in the file indicates a separate argument. For example, this would not be suitable input:
$ paste start.txt end.txt
2019-01-08 04:14:59 2019-01-08 04:15:50
2019-01-08 04:16:57 2019-01-08 04:17:02
Therefore I use sed to replace the spaces (not tabs) with T, to match some examples from the datediff manual (man datediff):
$ paste start.txt end.txt | sed 's/ /T/g'
2019-01-08T04:14:59 2019-01-08T04:15:50
2019-01-08T04:16:57 2019-01-08T04:17:02
This data now only has spaces in between the intended arguments, and each argument is in a form that matches examples provided in the manual.