Skip to main content
added 216 characters in body
Source Link
Stéphane Chazelas
  • 585.2k
  • 96
  • 1.1k
  • 1.7k
  • one MUST NOT use files with fixed names in world-writable directories such as /tmp (hence the switch to ~/tmp here, or use a dedicated area in /var or ~/var/~/.local/$XDG_RUNTIME_DIR...).

  • there's nothing bash-specific in that code, so no need to add that bash dependency.

  • with -n, extra arguments to perl are the input to the script

  • as Chris already said, you had issues with your quoting.

  • you have a AUTOSERV / AUTOSERVE discrepancy.

  • . is a regex operator that matches any single character. Use \. or [.] to match a literal dot.

  • beware that usage of date is GNU-specific. Not all date implementations support a -d option, and among those that do, it can be for something totally unrelated like on BSDs or they don't recognise yesterday (like with the date of busybox or toybox). perl can also do date manipulation¹ if you need that script to be portable to non-GNU systems.

  • You could easily change that to use a single regexp such as:

    /STATUS:\s+(?:SUCCESS|FAILURE|TERMINATED)\b.+MACHINE:\s+(\w+.\w+.\w+)$/
    
  • replace keys %count with sort cmp, keys %count if you want the list of machines to be sorted lexically.

  • exec, common in wrapper scripts like this one is just to save a process. It tells the shell to run perl in the same process rather than in a child process and wait for it. cmd does fork()+exec(cmd)&wait(child), while exec cmd (which maybe should have been called nofork cmd) is just exec(cmd) so even though it's longer to type, it's simpler/shorter to run for the system and uses less resource.

  • %m%d%Y is not a very good choice of timestamp format. It's ambiguous and its lexical order (like in the output of ls) does not match the chronological order. %Y-%m-%d or %F for short is much better as it's universally recognised, and sorts lexically chronologically (at least for years 0001 to 9999).

  • cat is the command to concatenate files, it makes little sense to use it for one file. Using cmd < input > output (or <input cmd >output, but not cmd > output < input) also has the benefit that if input can't be opened for reading, then cmd won't be run and output won't be clobbered.

¹ For instance here by adding -MPOSIX and a BEGIN{@t = localtime; $t[3]--; $dt = strftime "%m%d%Y", @t} or even as a hack just -M'POSIX;@t = localtime; $t[3]--; $dt = strftime "%m%d%Y", @t'.

  • one MUST NOT use files with fixed names in world-writable directories such as /tmp (hence the switch to ~/tmp here, or use a dedicated area in /var or ~/var/~/.local/$XDG_RUNTIME_DIR...).

  • there's nothing bash-specific in that code, so no need to add that bash dependency.

  • with -n, extra arguments to perl are the input to the script

  • as Chris already said, you had issues with your quoting.

  • you have a AUTOSERV / AUTOSERVE discrepancy.

  • . is a regex operator that matches any single character. Use \. or [.] to match a literal dot.

  • beware that usage of date is GNU-specific. Not all date implementations support a -d option, and among those that do, it can be for something totally unrelated like on BSDs or they don't recognise yesterday (like with the date of busybox or toybox). perl can also do date manipulation¹ if you need that script to be portable to non-GNU systems.

  • You could easily change that to use a single regexp such as:

    /STATUS:\s+(?:SUCCESS|FAILURE|TERMINATED)\b.+MACHINE:\s+(\w+.\w+.\w+)$/
    
  • replace keys %count with sort cmp, keys %count if you want the list of machines to be sorted lexically.

  • %m%d%Y is not a very good choice of timestamp format. It's ambiguous and its lexical order (like in the output of ls) does not match the chronological order. %Y-%m-%d or %F for short is much better as it's universally recognised, and sorts lexically chronologically (at least for years 0001 to 9999).

  • cat is the command to concatenate files, it makes little sense to use it for one file. Using cmd < input > output (or <input cmd >output, but not cmd > output < input) also has the benefit that if input can't be opened for reading, then cmd won't be run and output won't be clobbered.

¹ For instance here by adding -MPOSIX and a BEGIN{@t = localtime; $t[3]--; $dt = strftime "%m%d%Y", @t} or even as a hack just -M'POSIX;@t = localtime; $t[3]--; $dt = strftime "%m%d%Y", @t'

  • one MUST NOT use files with fixed names in world-writable directories such as /tmp (hence the switch to ~/tmp here, or use a dedicated area in /var or ~/var/~/.local/$XDG_RUNTIME_DIR...).

  • there's nothing bash-specific in that code, so no need to add that bash dependency.

  • with -n, extra arguments to perl are the input to the script

  • as Chris already said, you had issues with your quoting.

  • you have a AUTOSERV / AUTOSERVE discrepancy.

  • . is a regex operator that matches any single character. Use \. or [.] to match a literal dot.

  • beware that usage of date is GNU-specific. Not all date implementations support a -d option, and among those that do, it can be for something totally unrelated like on BSDs or they don't recognise yesterday (like with the date of busybox or toybox). perl can also do date manipulation¹ if you need that script to be portable to non-GNU systems.

  • You could easily change that to use a single regexp such as:

    /STATUS:\s+(?:SUCCESS|FAILURE|TERMINATED)\b.+MACHINE:\s+(\w+.\w+.\w+)$/
    
  • replace keys %count with sort cmp, keys %count if you want the list of machines to be sorted lexically.

  • exec, common in wrapper scripts like this one is just to save a process. It tells the shell to run perl in the same process rather than in a child process and wait for it. cmd does fork()+exec(cmd)&wait(child), while exec cmd (which maybe should have been called nofork cmd) is just exec(cmd) so even though it's longer to type, it's simpler/shorter to run for the system and uses less resource.

  • %m%d%Y is not a very good choice of timestamp format. It's ambiguous and its lexical order (like in the output of ls) does not match the chronological order. %Y-%m-%d or %F for short is much better as it's universally recognised, and sorts lexically chronologically (at least for years 0001 to 9999).

  • cat is the command to concatenate files, it makes little sense to use it for one file. Using cmd < input > output (or <input cmd >output, but not cmd > output < input) also has the benefit that if input can't be opened for reading, then cmd won't be run and output won't be clobbered.

¹ For instance here by adding -MPOSIX and a BEGIN{@t = localtime; $t[3]--; $dt = strftime "%m%d%Y", @t} or even as a hack just -M'POSIX;@t = localtime; $t[3]--; $dt = strftime "%m%d%Y", @t'.

added 216 characters in body
Source Link
Stéphane Chazelas
  • 585.2k
  • 96
  • 1.1k
  • 1.7k
  • one MUST NOT use files with fixed names in world-writable directories such as /tmp (hence the switch to ~/tmp here, or use a dedicated area in /var or ~/var/~/.local/$XDG_RUNTIME_DIR...).

  • there's nothing bash-specific in that code, so no need to add that bash dependency.

  • with -n, extra arguments to perl are the input to the script

  • as Chris already said, you had issues with your quoting.

  • you have a AUTOSERV / AUTOSERVE discrepancy.

  • . is a regex operator that matches any single character. Use \. or [.] to match a literal dot.

  • beware that usage of date is GNU-specific. Not all date implementations support a -d option, and among those that do, it can be for something totally unrelated like on BSDs or they don't recognise yesterday (like with the date of busybox or toybox). perl can also do date manipulationmanipulation¹ if you need that script to be portable to non-GNU systems.

  • You could easily change that to use a single regexp such as:

    /STATUS:\s+(?:SUCCESS|FAILURE|TERMINATED)\b.+MACHINE:\s+(\w+.\w+.\w+)$/
    
  • replace keys %count with sort cmp, keys %count if you want the list of machines to be sorted lexically.

  • %m%d%Y is not a very good choice of timestamp format. It's ambiguous and its lexical order (like in the output of ls) does not match the chronological order. %Y-%m-%d or %F for short is much better as it's universally recognised, and sorts lexically chronologically (at least for years 0001 to 9999).

  • cat is the command to concatenate files, it makes little sense to use it for one file. Using cmd < input > output (or <input cmd >output, but not cmd > output < input) also has the benefit that if input can't be opened for reading, then cmd won't be run and output won't be clobbered.


¹ For instance here by adding -MPOSIX and a BEGIN{@t = localtime; $t[3]--; $dt = strftime "%m%d%Y", @t} or even as a hack just -M'POSIX;@t = localtime; $t[3]--; $dt = strftime "%m%d%Y", @t'

  • one MUST NOT use files with fixed names in world-writable directories such as /tmp (hence the switch to ~/tmp here, or use a dedicated area in /var or ~/var/~/.local/$XDG_RUNTIME_DIR...).

  • there's nothing bash-specific in that code, so no need to add that bash dependency.

  • with -n, extra arguments to perl are the input to the script

  • as Chris already said, you had issues with your quoting.

  • you have a AUTOSERV / AUTOSERVE discrepancy.

  • . is a regex operator that matches any single character. Use \. or [.] to match a literal dot.

  • beware that usage of date is GNU-specific. Not all date implementations support a -d option, and among those that do, it can be for something totally unrelated like on BSDs or they don't recognise yesterday (like with the date of busybox or toybox). perl can also do date manipulation if you need that script to be portable to non-GNU systems.

  • You could easily change that to use a single regexp such as:

    /STATUS:\s+(?:SUCCESS|FAILURE|TERMINATED)\b.+MACHINE:\s+(\w+.\w+.\w+)$/
    
  • replace keys %count with sort cmp, keys %count if you want the list of machines to be sorted lexically.

  • %m%d%Y is not a very good choice of timestamp format. It's ambiguous and its lexical order (like in the output of ls) does not match the chronological order. %Y-%m-%d or %F for short is much better as it's universally recognised, and sorts lexically chronologically (at least for years 0001 to 9999).

  • cat is the command to concatenate files, it makes little sense to use it for one file. Using cmd < input > output (or <input cmd >output, but not cmd > output < input) also has the benefit that if input can't be opened for reading, then cmd won't be run and output won't be clobbered.

  • one MUST NOT use files with fixed names in world-writable directories such as /tmp (hence the switch to ~/tmp here, or use a dedicated area in /var or ~/var/~/.local/$XDG_RUNTIME_DIR...).

  • there's nothing bash-specific in that code, so no need to add that bash dependency.

  • with -n, extra arguments to perl are the input to the script

  • as Chris already said, you had issues with your quoting.

  • you have a AUTOSERV / AUTOSERVE discrepancy.

  • . is a regex operator that matches any single character. Use \. or [.] to match a literal dot.

  • beware that usage of date is GNU-specific. Not all date implementations support a -d option, and among those that do, it can be for something totally unrelated like on BSDs or they don't recognise yesterday (like with the date of busybox or toybox). perl can also do date manipulation¹ if you need that script to be portable to non-GNU systems.

  • You could easily change that to use a single regexp such as:

    /STATUS:\s+(?:SUCCESS|FAILURE|TERMINATED)\b.+MACHINE:\s+(\w+.\w+.\w+)$/
    
  • replace keys %count with sort cmp, keys %count if you want the list of machines to be sorted lexically.

  • %m%d%Y is not a very good choice of timestamp format. It's ambiguous and its lexical order (like in the output of ls) does not match the chronological order. %Y-%m-%d or %F for short is much better as it's universally recognised, and sorts lexically chronologically (at least for years 0001 to 9999).

  • cat is the command to concatenate files, it makes little sense to use it for one file. Using cmd < input > output (or <input cmd >output, but not cmd > output < input) also has the benefit that if input can't be opened for reading, then cmd won't be run and output won't be clobbered.


¹ For instance here by adding -MPOSIX and a BEGIN{@t = localtime; $t[3]--; $dt = strftime "%m%d%Y", @t} or even as a hack just -M'POSIX;@t = localtime; $t[3]--; $dt = strftime "%m%d%Y", @t'

added 244 characters in body
Source Link
Stéphane Chazelas
  • 585.2k
  • 96
  • 1.1k
  • 1.7k

Sounds like you want something like:

#! /bin/sh -
DT=$(date -d yesterday +%m%d%Y) || exit
export DT
exec perl -lne '
  if (
    ($status, $machine) = /STATUS:\s+(\w+).+MACHINE:\s+(\w+\.\w+\.\w+)$/ and
    $status =~ /^(SUCCESS|FAILURE|TERMINATED)\z/
  ) {$count{$machine}++}
  END {
    for (keys %count) {
      print join "\t", $ENV{DT}, $ENV{AUTOSERVE}, $_, $count{$_};
    }
  }' < ~/tmp/log."$AUTOSERVE.$dt" > ~/tmp/output.txt

Note:

  • one MUST NOT use files with fixed names in world-writable directories such as /tmp (hence the switch to ~/tmp here, or use a dedicated area in /var or ~/var/~/.local/$XDG_RUNTIME_DIR...).

  • there's nothing bash-specific in that code, so no need to add that bash dependency.

  • with -n, extra arguments to perl are the input to the script

  • as Chris already said, you had issues with your quoting.

  • you have a AUTOSERV / AUTOSERVE discrepancy.

  • . is a regex operator that matches any single character. Use \. or [.] to match a literal dot.

  • beware that usage of date is GNU-specific. Not all date implementations support a -d option, and among those that do, it can be for something totally unrelated like on BSDs or they don't recognise yesterday (like with the date of busybox or toybox). perl can also do date manipulation if you need that script to be portable to non-GNU systems.

  • You could easily change that to use a single regexp such as:

    /STATUS:\s+(?:SUCCESS|FAILURE|TERMINATED)\b.+MACHINE:\s+(\w+.\w+.\w+)$/
    
  • replace keys %count with sort cmp, keys %count if you want the list of machines to be sorted lexically.

  • %m%d%Y is not a very good choice of timestamp format. It's ambiguous and its lexical order (like in the output of ls) does not match the chronological order. %Y-%m-%d or %F for short is much better as it's universally recognised, and sorts lexically chronologically (at least for years 0001 to 9999).

  • cat is the command to concatenate files, it makes little sense to use it for one file. Using cmd < input > output (or <input cmd >output, but not cmd > output < input) also has the benefit that if input can't be opened for reading, then cmd won't be run and output won't be clobbered.

Sounds like you want something like:

#! /bin/sh -
DT=$(date -d yesterday +%m%d%Y) || exit
export DT
exec perl -lne '
  if (
    ($status, $machine) = /STATUS:\s+(\w+).+MACHINE:\s+(\w+\.\w+\.\w+)$/ and
    $status =~ /^(SUCCESS|FAILURE|TERMINATED)\z/
  ) {$count{$machine}++}
  END {
    for (keys %count) {
      print join "\t", $ENV{DT}, $ENV{AUTOSERVE}, $_, $count{$_};
    }
  }' < ~/tmp/log."$AUTOSERVE.$dt" > ~/tmp/output.txt

Note:

  • one MUST NOT use files with fixed names in world-writable directories such as /tmp (hence the switch to ~/tmp here, or use a dedicated area in /var or ~/var/~/.local/$XDG_RUNTIME_DIR...).

  • there's nothing bash-specific in that code, so no need to add that bash dependency.

  • with -n, extra arguments to perl are the input to the script

  • as Chris already said, you had issues with your quoting.

  • you have a AUTOSERV / AUTOSERVE discrepancy.

  • . is a regex operator that matches any single character. Use \. or [.] to match a literal dot.

  • beware that usage of date is GNU-specific. Not all date implementations support a -d option, and among those that do, it can be for something totally unrelated like on BSDs or they don't recognise yesterday (like with the date of busybox or toybox). perl can also do date manipulation if you need that script to be portable to non-GNU systems.

  • You could easily change that to use a single regexp such as:

    /STATUS:\s+(?:SUCCESS|FAILURE|TERMINATED)\b.+MACHINE:\s+(\w+.\w+.\w+)$/
    
  • replace keys %count with sort cmp, keys %count if you want the list of machines to be sorted lexically.

  • %m%d%Y is not a very good choice of timestamp format. It's ambiguous and its lexical order (like in the output of ls) does not match the chronological order. %Y-%m-%d or %F for short is much better as it's universally recognised, and sorts lexically chronologically (at least for years 0001 to 9999).

Sounds like you want something like:

#! /bin/sh -
DT=$(date -d yesterday +%m%d%Y) || exit
export DT
exec perl -lne '
  if (
    ($status, $machine) = /STATUS:\s+(\w+).+MACHINE:\s+(\w+\.\w+\.\w+)$/ and
    $status =~ /^(SUCCESS|FAILURE|TERMINATED)\z/
  ) {$count{$machine}++}
  END {
    for (keys %count) {
      print join "\t", $ENV{DT}, $ENV{AUTOSERVE}, $_, $count{$_};
    }
  }' < ~/tmp/log."$AUTOSERVE.$dt" > ~/tmp/output.txt

Note:

  • one MUST NOT use files with fixed names in world-writable directories such as /tmp (hence the switch to ~/tmp here, or use a dedicated area in /var or ~/var/~/.local/$XDG_RUNTIME_DIR...).

  • there's nothing bash-specific in that code, so no need to add that bash dependency.

  • with -n, extra arguments to perl are the input to the script

  • as Chris already said, you had issues with your quoting.

  • you have a AUTOSERV / AUTOSERVE discrepancy.

  • . is a regex operator that matches any single character. Use \. or [.] to match a literal dot.

  • beware that usage of date is GNU-specific. Not all date implementations support a -d option, and among those that do, it can be for something totally unrelated like on BSDs or they don't recognise yesterday (like with the date of busybox or toybox). perl can also do date manipulation if you need that script to be portable to non-GNU systems.

  • You could easily change that to use a single regexp such as:

    /STATUS:\s+(?:SUCCESS|FAILURE|TERMINATED)\b.+MACHINE:\s+(\w+.\w+.\w+)$/
    
  • replace keys %count with sort cmp, keys %count if you want the list of machines to be sorted lexically.

  • %m%d%Y is not a very good choice of timestamp format. It's ambiguous and its lexical order (like in the output of ls) does not match the chronological order. %Y-%m-%d or %F for short is much better as it's universally recognised, and sorts lexically chronologically (at least for years 0001 to 9999).

  • cat is the command to concatenate files, it makes little sense to use it for one file. Using cmd < input > output (or <input cmd >output, but not cmd > output < input) also has the benefit that if input can't be opened for reading, then cmd won't be run and output won't be clobbered.

added 312 characters in body
Source Link
Stéphane Chazelas
  • 585.2k
  • 96
  • 1.1k
  • 1.7k
Loading
added 312 characters in body
Source Link
Stéphane Chazelas
  • 585.2k
  • 96
  • 1.1k
  • 1.7k
Loading
added 128 characters in body
Source Link
Stéphane Chazelas
  • 585.2k
  • 96
  • 1.1k
  • 1.7k
Loading
added 53 characters in body
Source Link
Stéphane Chazelas
  • 585.2k
  • 96
  • 1.1k
  • 1.7k
Loading
added 1 character in body
Source Link
Stéphane Chazelas
  • 585.2k
  • 96
  • 1.1k
  • 1.7k
Loading
Source Link
Stéphane Chazelas
  • 585.2k
  • 96
  • 1.1k
  • 1.7k
Loading