For a Perl solution, you could use the Win32::Filenames module for this.
http://search.cpan.org/~bch/Win32-Filenames-0.01/lib/Win32/Filenames.pm
A simple program using this module:
#!/usr/bin/env perl
use strict;
use warnings;
use Win32::Filenames qw(validate sanitize);
while ( my $filename = shift ) {
    printf( "checking filename %s\t", $filename );
    if ( validate($filename) ) {
        print("ok.\n");
    }
    else {
        printf( "failed, try this instead: %s\n", sanitize($filename) );
    }
}
You invoke it with one or several file names on the command line:
$ ./check.pl '<a>' 'hi?' '"right"' 'pipe|me'
checking filename <a>   failed, try this instead: -a-
checking filename hi?   failed, try this instead: hi-
checking filename "right"       failed, try this instead: -right-
checking filename pipe|me       failed, try this instead: pipe-me
For a whole file hierarchy:
$ find . -exec basename {} \; | xargs ./check.pl
 
                