It's good to have last incorrect comment to correct it, but soon after that, it becomes potentially confusing garbage.

My approach is two-step: store commands that fail when they do, and remove them sometime later.

## Store commands that fail when they do:

    error_handler() {
        FAILED_COMMANDS="$(history | tail -1l | cut -c -5) $FAILED_COMMANDS"
    }

    trap error_handler ERR

Explanation:

`trap command signals` executes `command` when one of `signals` is "raised".  

`history | tail -1l | cut -c -5` outputs history number of last command *saved into history*.  
<sup>(**Note**: if command which failed is not saved into history due to `HISTCONTROL`, `HISTIGNORE`, etc., you get history number of previous command, marking it for deletion.)</sup>

Enclosing command in in `$()`, like `$(command)`, executes the command and captures its output.  
<sup></sup>

Captured output is then prepended (not appended!) into `FAILED_COMMANDS` variable.

## Remove stored commands sometime later:

    exit_handler() {
        for i in $(echo $FAILED_COMMANDS | tr ' ' '\n' | uniq)
        do
            history -d $i
        done
        FAILED_COMMANDS=
    }
    
    trap exit_handler EXIT

Explanation:

When exiting Bash, for each unique history number remove corresponding history entry,  
then clear `FAILED_COMMANDS` to not remove commands which inherited history numbers from already deleted commands.

If you're sure that `FAILED_COMMANDS` will be free from duplicates, you can simple iterate over it  
(ie. write `for i in $FAILED_COMMANDS`). If, however, you expect it to be not sorted from greatest to smallest (in this case it always is), replace `uniq` with `sort -rn`.

History numbers in `FAILED_COMMANDS` must be unique and sorted from greatest to smallest, because when you delete entry, next commands' numbers are shifted &ndash; ie. when you issue `history -d 2`, 3rd entry becomes 2nd, 4th becomes 3rd, etc.

It's probably good idea to hook it at `EXIT`, but you can also call it anytime earlier.