Bash: Append command output and timestamp to file

Question

Normally in my bash scripts I'm used to do some_command >> log.log. This works fine, however how can I append more data like time and command name?

My goal is to have a log like this

2012-01-01 00:00:01 [some_command] => some command output...
2012-01-01 00:01:01 [other_command] => other command output...

The processes should running and writing to the file concurrently.

The final solution, pointed by William Pursell in my case would be:

some_command 2>&1 | perl -ne '$|=1; print localtime . ": [somme_command] $_"' >> /log.log &

I also added 2>&1 to redirect the STDOUTand STDERR to the file and an & on the end to keep the program on background.

Thank you!

William Pursell · Accepted Answer · 2012-05-31 18:39:05Z

3

Given your comments, it seems that you want multiple processes to be writing to the file concurrently, and have a timestamp on each individual line. Something like this might suffice:

some_cmd | perl -ne '$|=1; print localtime . ": [some_cmd] $_"' >> logfile

If you want to massage the format of the date, use POSIX::strftime

some_cmd | perl -MPOSIX -ne 'BEGIN{ $|=1 }
   print strftime( "%Y-%m-%d %H:%M:%S", localtime ) . " [some_cmd] $_"' >> logfile

edited May 31, 2012 at 18:39

answered May 31, 2012 at 18:21

William Pursell

214k48 gold badges278 silver badges317 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

TCB13 Over a year ago

Actually it only works fine once, if I do a second command it won't execute at all. Tips? :S

William Pursell Over a year ago

Are you running it asynchronously? (eg, with &) How do you determine that it does not work? Possibly a buffering issue, but this should work ok.

TCB13 Over a year ago

I'm running like this: some_command | perl -ne '$|=1; print localtime . ": [somme_command] $_"' >> /log.log 2>&1 & I know they are not working because I need to killall the first one to get the second to do it's job (and also logging). The command causing the issue is a python script (I can deal with that later). But there's a problem... if I start dhcpd in the second place it will output to the console instead of the log file.

William Pursell Over a year ago

Simple cases of some_command work okay for me. Have you tried with different commands? Perhaps they are contending for a resource somehow. But it works for you if you eliminate the perl? What if you replace the entire perl command with cat?

William Pursell Over a year ago

The stderr of your command is not being redirectd. Try some_command 2>&1 | perl ... >> log.log &

|

estan · Accepted Answer · 2013-05-31 07:36:41Z

3

An alternative solution using sed would be:

some_command 2>&1 | sed "s/^/`date '+%Y-%m-%d %H:%M:%S'`: [some_command] /" >> log.log &

It works by replacing the beginning of line "character" (^). Might come in handy if you don't want to depend on Perl.

answered May 31, 2013 at 7:36

estan

1,4943 gold badges18 silver badges29 bronze badges

2 Comments

Baruch Even Over a year ago

I just tried this and it appears that sed only runs the command once and thus all the lines go with the same time

Paul Wheeler Over a year ago

Yeah, that shell command interpolation is only going to be run once before sed starts. You can use the e command in GNU sed to actually execute the date command for every line of the file. See my answer: stackoverflow.com/a/73723738/229247

Cookie · Accepted Answer · 2014-03-26 08:43:37Z

2

On Ubuntu:

sudo apt-get install moreutils
echo "cat" | ts
Mar 26 09:43:00 cat

answered Mar 26, 2014 at 8:43

Cookie

12.8k14 gold badges57 silver badges86 bronze badges

Comments

Dennis Williamson · Accepted Answer · 2012-05-31 19:49:45Z

1

something like this:

(echo -n $(date); echo  -n " ls => ";  ls) >> /tmp/log

however, your command output is multiple lines and it will not have the format above you are showing. you may want to replace the newline in output with some other character with a command like tr or sed in that case.

edited May 31, 2012 at 19:49

Dennis Williamson

364k95 gold badges386 silver badges446 bronze badges

answered May 31, 2012 at 18:17

johnshen64

3,8841 gold badge23 silver badges17 bronze badges

Comments

DigitalRoss · Accepted Answer · 2012-05-31 18:19:41Z

0

One approach is to use logger(1).

Another might be something like this:

stamp () {
  ( echo -n "`date +%T` "
    "$@" ) >> logfile
}

stamp echo how now brown cow

answered May 31, 2012 at 18:19

DigitalRoss

147k25 gold badges254 silver badges336 bronze badges

Comments

Paul Wheeler · Accepted Answer · 2022-09-14 22:18:01Z

A better alternative using GNU sed would be:

some_command 2>&1 | sed 'h; s/.*/date "+%Y-%m-%d %H:%M:%S"/e; G; s/\n/ [some_command]: /'

Breaking down how this sed program works:

# store the current line (a.k.a. "pattern space") in the "hold space"
h;
# replace the current line with the date command and execute it
# note the e command is available in GNU sed, but not in some other version
s/.*/date "+%Y-%m-%d %H:%M:%S"/e;
# Append a newline and the contents of the "hold space" to the "pattern space"
G;
# Replace that newline inserted by the G command with whatever text you want
s/\n/ [some_command]: /

I have made this answer a community wiki answer in case somebody wants to code 🏌️‍♂️ my sed command. Ideally the command can be easily understood without extraneous steps.

Collectives™ on Stack Overflow

Bash: Append command output and timestamp to file

6 Answers 6

6 Comments

2 Comments

Comments

Comments

Comments

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

6 Comments

2 Comments

Comments

Comments

Comments

1 Comment

Related