3

I'm processing a log file and I need reorder each line (not sort). The log file looks like this:

11-06-2014 - 10:49:06PM lat = 41.858657; lon = -91.345142
11-06-2014 - 10:49:49PM lat = 42.864653; lon = -92.349914
11-06-2014 - 10:50:35PM lat = 43.874808; lon = -93.350364
11-06-2014 - 10:51:21PM lat = 44.885047; lon = -94.350058
11-06-2014 - 10:52:08PM lat = 45.895078; lon = -95.349920
11-06-2014 - 10:53:30PM lat = 46.905178; lon = -96.349837
11-06-2014 - 10:54:50PM lat = 47.910303; lon = -97.350606

and I want to move the date/time to the end of each line:

lat = 41.858657; lon = -91.345142 11-06-2014 - 10:49:06PM
lat = 42.864653; lon = -92.349914 11-06-2014 - 10:49:49PM
lat = 43.874808; lon = -93.350364 11-06-2014 - 10:50:35PM
lat = 44.885047; lon = -94.350058 11-06-2014 - 10:51:21PM
lat = 45.895078; lon = -95.349920 11-06-2014 - 10:52:08PM
lat = 46.905178; lon = -96.349837 11-06-2014 - 10:53:30PM
lat = 47.910303; lon = -97.350606 11-06-2014 - 10:54:50PM

sed? awk? how?

Bonus question: my end goal is to turn this into gpx/xml, and it's probably just as easy to add the intermediate text as each line is processed, so that the out looks like this:

<wpt lat="41.858657" lon="-91.345142">
    <time>11-06-2014 - 10:49:06PM</time>
</wpt>

<wpt lat"="42.864653" lon="-92.349914">
     <time>11-06-2014 - 10:49:49PM</time>
</wpt>

<wpt lat"="43.874808" lon="-93.350364">
     <time>11-06-2014 - 10:50:35PM</time>
</wpt>

<wpt lat"="44.885047" lon="-94.350058">
     <time>11-06-2014 - 10:51:21PM</time>
</wpt>

<wpt lat"="45.895078" lon="-95.349920">
     <time>11-06-2014 - 10:52:08PM</time>
</wpt>

<wpt lat"="46.905178" lon="-96.349837">
     <time>11-06-2014 - 10:53:30PM</time>
</wpt>

<wpt lat"="47.910303" lon="-97.350606">
     <time>11-06-2014 - 10:54:50PM</time>
</wpt>

7 Answers 7

5

using awk :

 awk '{print $4,$5,$6,$7,$8,$9,$1,$2,$3}' log_file

you can do it directly from you log file like this:

awk '{printf("<wpt %s%s\"%s\" %s%s\"%s\">\n<time>%s %s %s</time>\n</wpt>\n",$4,$5,substr($6,0,length($6)),$7,$8,$9,$1,$2,$3)}' log_file

output:

<wpt lat="41.858657" lon="-91.345142">
<time>11-06-2014 - 10:49:06PM</time>
</wpt>
<wpt lat="42.864653" lon="-92.349914">
<time>11-06-2014 - 10:49:49PM</time>
</wpt>
<wpt lat="43.874808" lon="-93.350364">
<time>11-06-2014 - 10:50:35PM</time>
</wpt>
<wpt lat="44.885047" lon="-94.350058">
<time>11-06-2014 - 10:51:21PM</time>
</wpt>
<wpt lat="45.895078" lon="-95.349920">
<time>11-06-2014 - 10:52:08PM</time>
</wpt>
<wpt lat="46.905178" lon="-96.349837">
<time>11-06-2014 - 10:53:30PM</time>
</wpt>
<wpt lat="47.910303" lon="-97.350606">
<time>11-06-2014 - 10:54:50PM</time>
</wpt>
2
  • @Robert Altman check the for your bonus question Commented Nov 7, 2014 at 17:58
  • Well answered. One tweak is to replace substr($6,0,length($6)) with substr($6,0,length($6 - 1)) to drop the semicolon. Commented Nov 10, 2014 at 22:40
4

You can use

perl -lane 'print "@F[2..$#F] $F[0] $F[1]"' log_file

to flip the log lines.

2
  • 1
    Neat, I didn't know about perl's -l and -a. Very handy, and -lane is easy to remember. Commented Nov 7, 2014 at 23:14
  • The code above appears to only move the date to the end of the line. To move the time to the end of the line as well, you need to do perl -lane 'print "@F[3..$#F] @F[0..2]"' . Commented Jul 16, 2021 at 1:27
2
sed '   s/ *[=;]"* */="/g;s//" /2         #first sub-out spaces; for "
        s|\(.*M\) \(.*\)|\                #then do the replace
<wpt \2">\
        <time>\1</time>\
</wpt>|
' <<\INPUT
11-06-2014 - 10:49:06PM lat = 41.858657; lon = -91.345142
11-06-2014 - 10:49:49PM lat = 42.864653; lon = -92.349914
11-06-2014 - 10:50:35PM lat = 43.874808; lon = -93.350364
11-06-2014 - 10:51:21PM lat = 44.885047; lon = -94.350058
11-06-2014 - 10:52:08PM lat = 45.895078; lon = -95.349920
11-06-2014 - 10:53:30PM lat = 46.905178; lon = -96.349837
11-06-2014 - 10:54:50PM lat = 47.910303; lon = -97.350606
INPUT

This is what I came up with. It's fairly simple in sed. I use literal characters here because it is easier for me to read and because not all sed's will handle backslash escapes in the right hand substitution field. If yours does, then feel free. Anyway, definitely those inline comments will muck up any sed so do remove those before testing.

OUTPUT

<wpt lat="41.858657" lon="-91.345142">
        <time>11-06-2014 - 10:49:06PM</time>
</wpt>

<wpt lat="42.864653" lon="-92.349914">
        <time>11-06-2014 - 10:49:49PM</time>
</wpt>

<wpt lat="43.874808" lon="-93.350364">
        <time>11-06-2014 - 10:50:35PM</time>
</wpt>

<wpt lat="44.885047" lon="-94.350058">
        <time>11-06-2014 - 10:51:21PM</time>
</wpt>

<wpt lat="45.895078" lon="-95.349920">
        <time>11-06-2014 - 10:52:08PM</time>
</wpt>

<wpt lat="46.905178" lon="-96.349837">
        <time>11-06-2014 - 10:53:30PM</time>
</wpt>

<wpt lat="47.910303" lon="-97.350606">
        <time>11-06-2014 - 10:54:50PM</time>
</wpt>
1

the first part of your rearrangement can be done pretty easily, as long as the composition of the data is the same and doesn't change.

cat myfile | while read line
do
  latlong=$(echo ${line} | cut -d " " -f 4-)
  datetime=$(echo ${line} | cut -d " " -f 1-3)
  echo ${latlong}" "${datetime} >> mynewlyformattedfile
done

this can be done more elaborately on one line, using sed and/or awk I am sure, but I tried to do it this way to show what's happening at what point.

by using the cut command, you can extract any portion of a line and echo it with any prefix/suffix to create your xml file pretty easily in my opinion.

1

Same for sed

sed -E 's&(.*M)( .*);( .*)$&<wpt\2\3>\n\t<time>\1</time>\n</wpt>&;
        $!G;
        s& = ([-.0-9]+)&="\1"&g' log.file

Or put it in the script-file

#!/bin/sed -Ef
s|(.*M)( .*);( .*)$|<wpt\2\3>\n\t<time>\1</time>\n</wpt>|
$!G
s| = ([-.0-9]+)|="\1"|g

And use it

sed -Ef script.file log.file

or if have made it executable by chmod +x ./script.file just

./script.file log.file
1
  1. Converting the data to ;-delimited CSV by replacing the lat = and lon = appropriately. Also output a simple header.

    echo 'time;@lat;@lon'
    sed -e 's/;\{0,1\}[[:blank:]]... = /;/g' file
    

    This removes an optional semi-colon, three characters, and an equal sign surrounded by a space on either side, and replaces all this by a semi colon. The substitution is repeated until nothing more matches (it will substitute twice on each line).

    The output will look like this:

    time;@lat;@lon
    11-06-2014 - 10:49:06PM;41.858657;-91.345142
    11-06-2014 - 10:49:49PM;42.864653;-92.349914
    11-06-2014 - 10:50:35PM;43.874808;-93.350364
    11-06-2014 - 10:51:21PM;44.885047;-94.350058
    11-06-2014 - 10:52:08PM;45.895078;-95.349920
    11-06-2014 - 10:53:30PM;46.905178;-96.349837
    11-06-2014 - 10:54:50PM;47.910303;-97.350606
    
  2. Take this and let csvjson from csvkit convert this into JSON:

    csvjson -d ';'
    

    The output from this will be the equivalent of

    [
    {"time":"11-06-2014 - 10:49:06PM","@lat":41.858657,"@lon":-91.345142},
    {"time":"11-06-2014 - 10:49:49PM","@lat":42.864653,"@lon":-92.349914},
    {"time":"11-06-2014 - 10:50:35PM","@lat":43.874808,"@lon":-93.350364},
    {"time":"11-06-2014 - 10:51:21PM","@lat":44.885047,"@lon":-94.350058},
    {"time":"11-06-2014 - 10:52:08PM","@lat":45.895078,"@lon":-95.34992},
    {"time":"11-06-2014 - 10:53:30PM","@lat":46.905178,"@lon":-96.349837},
    {"time":"11-06-2014 - 10:54:50PM","@lat":47.910303,"@lon":-97.350606}
    ]
    
  3. Use yq from https://kislyuk.github.io/yq/ to trans-code this straight into XML. This is where the keys prefixed by with @ will be treated as attributes, while other keys will be treated as node names.

    yq --xml-root=wpt -x .
    

    Result:

    <wpt lat="41.858657" lon="-91.345142">
      <time>11-06-2014 - 10:49:06PM</time>
    </wpt><wpt lat="42.864653" lon="-92.349914">
      <time>11-06-2014 - 10:49:49PM</time>
    </wpt><wpt lat="43.874808" lon="-93.350364">
      <time>11-06-2014 - 10:50:35PM</time>
    </wpt><wpt lat="44.885047" lon="-94.350058">
      <time>11-06-2014 - 10:51:21PM</time>
    </wpt><wpt lat="45.895078" lon="-95.34992">
      <time>11-06-2014 - 10:52:08PM</time>
    </wpt><wpt lat="46.905178" lon="-96.349837">
      <time>11-06-2014 - 10:53:30PM</time>
    </wpt><wpt lat="47.910303" lon="-97.350606">
      <time>11-06-2014 - 10:54:50PM</time>
    </wpt>
    

Summary:

{
    echo 'time;@lat;@lon'
    sed -e 's/;\{0,1\}[[:blank:]]... = /;/g' file
} |
csvjson -d ';' |
yq --xml-root=wpt -x .
0

Using Raku (formerly known as Perl_6):

$ raku -e 'my @reordered = do for lines() {(.words[3..*],.words[0..2])}; .put for @reordered;' input.txt
lat = 41.858657; lon = -91.345142 11-06-2014 - 10:49:06PM
lat = 42.864653; lon = -92.349914 11-06-2014 - 10:49:49PM
lat = 43.874808; lon = -93.350364 11-06-2014 - 10:50:35PM
lat = 44.885047; lon = -94.350058 11-06-2014 - 10:51:21PM
lat = 45.895078; lon = -95.349920 11-06-2014 - 10:52:08PM
lat = 46.905178; lon = -96.349837 11-06-2014 - 10:53:30PM
lat = 47.910303; lon = -97.350606 11-06-2014 - 10:54:50PM

[...still working on the XML-output solution... ].

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.