using grep (or something similar) to find when rows are missing

Question

I'm trying to find when certain rows are missing in a file. See below. In each record, there is an ItemNumber, MessageNumber, NewItemNumber, and ConfirmedQty field.

Is there a way to find ItemNumber where NewItemNumber is missing, as in the last record here?

    <OrderConfirmationRecord>
            <ItemNumber>30353</ItemNumber>
            <MessageNumber>000</MessageNumber>
            <NewItemNumber>30353</NewItemNumber>
            <ConfirmedQty>1</ConfirmedQty>
    </OrderConfirmationRecord>
    <OrderConfirmationRecord>
            <ItemNumber>30354</ItemNumber>
            <MessageNumber>000</MessageNumber>
            <NewItemNumber>30354</NewItemNumber>
            <ConfirmedQty>1</ConfirmedQty>
    </OrderConfirmationRecord>
    <OrderConfirmationRecord>
            <ItemNumber>30391</ItemNumber>
            <MessageNumber>000</MessageNumber>
            <NewItemNumber>30391</NewItemNumber>
            <ConfirmedQty>1</ConfirmedQty>
    </OrderConfirmationRecord>
    <OrderConfirmationRecord>
            <ItemNumber>71067</ItemNumber>
            <MessageNumber>000</MessageNumber>
            <NewItemNumber>71067</NewItemNumber>
            <ConfirmedQty>1</ConfirmedQty>
    **<OrderConfirmationRecord>
            <ItemNumber>29424</ItemNumber>
            <MessageNumber>035</MessageNumber>
            <ConfirmedQty>0</ConfirmedQty>
    </OrderConfirmationRecord>**

Answer 1

perl -l -0777 -ne '
  for (m{<OrderConfirmationRecord>.*?</OrderConfirmationRecord>}gs) {
    print if /<ItemNumber>/ && ! /<NewItemNumber>/
  }' your-file

Would print all the strings that run from <OrderConfirmationRecord> to the next occurrence of </OrderConfirmationRecord> that contain <ItemNumber> and not <NewItemNumber>.

On your sample, it gives nothing but that's because that sample is missing a closing </OrderConfirmationRecord> for the second last <OrderConfirmationRecord> which I expect is a copy-paste mistake.

If that's a XML file, you could use xq to report the objects that have a ItemNumber attribute and not NewItemNumber one (in JSON format) with:

$ xq '..|objects|select(has("ItemNumber") and (has("NewItemNumber")|not))' your-file.xml
{
  "ItemNumber": "29424",
  "MessageNumber": "035",
  "ConfirmedQty": "0"
}

Or for only the ItemNumber, one per line:

$ xq -r '..|objects|select(has("ItemNumber") and (has("NewItemNumber")|not))|.ItemNumber' your-file.xml
29424

Or in pretty tabular format:

$ xq '..|objects|select(has("ItemNumber") and (has("NewItemNumber")|not))' your-file.xml |
    mlr --barred --j2p cat
+------------+---------------+--------------+
| ItemNumber | MessageNumber | ConfirmedQty |
+------------+---------------+--------------+
| 29424      | 035           | 0            |
+------------+---------------+--------------+

Answer 2

With xmlstarlet, assuming the XML file has the form <root><OrderConfirmationRecord>...</root>:

$ xmlstarlet sel -t -m '/root/OrderConfirmationRecord[not(NewItemNumber)]' -v 'ItemNumber' -n xml.xml 
29424

Answer 3

Here is one way you can do that using basic commands:

$ cat inputfile | paste - - - - - - | awk  '{print NF};NF<6{print "missing record in "  $1}'
6
6
6
6
4
missing record in <ItemNumber>29424</ItemNumber>

Basically we are flattening each block of data into a single row and applying awk to monitor the number of fields for the missing item.

I will leave it to you to tidy up the output.