Your shell can handle bitwise ops, though, for any serious processing, it's going to be awfully slow, and it can't handle anything more than say 20 or so digits at a time. Still:
sh <<\CMD 
    printf 'printf "%%b" "\\0$((%04o^04))"'  "'a" |\
    . /dev/stdin
CMD
#OUTPUT
A
I've used bc in the past for grabbing bytes in binary, so your question set me to googling  for...
Exclusive-or (XOR) for GNU bc
  If you've found your way across the internet to this question, chances
  are you're looking for bc's equivalent of C's ^ operator.
  
  Disturbing fact: no such thing exists in bc. In bc, the up-arrow
  operator is used for integer exponentiation, that is, 2^x returns a
  power of 2 and not x with bit 2 flipped. If you're looking for
  equivalents to the bitwise operators for XOR as well as AND, OR and a
  few more exotic relatives, check out this site's logic.bc, and its
  relatives which contain functions to perform each of these.
  
  If you're looking for a logical XOR to put in an if statement, like
  logical && and ||, try using != and surrounding your conditions with
  brackets. e.g.:
  
  c=0;if((a==1)!=(b==2)){c=3} 
  
  Will set c to 3, if a is 1 or b is 2, but
  not if a is 1 and b is 2 at the same time
  
  (Once upon a time, this was the secret to the internals of the
  logic.bc xor() function, but this has been superseded by a faster
  algorithm.)
The above is from the bc FAQ. The logic.bc function alluded to above includes the bitwise logic you're looking for. It can be found here. Its description:
  A large suite of functions to perform bitwise functions such as AND,
  OR, NOT and XOR. Uses twos complement for negative numbers, unlike
  previous versions of this file, which had no support at all. Some of
  the functions here will use the global bitwidth variable, which itself
  is initialised as part of this file, to emulate byte/word sizes found
  in most computers. If this variable is set to zero, an infinite
  bitwidth is assumed. Many functions will display a warning if there is
  suspicion that a secondary floating point representation of a number
  has been generated, e.g.: 
  
  1.1111... is an SFPR of10.0000...;` 
  
  These warnings can be disabled by setting the global variable sfpr_warn to 0
  (default is 1). 
- Fixed word size 
 
- Infinite word size 
 
- Common bitwise 
 
- Twos complement 
 
- Bit shifting 
 
- Gray code 
 
- 'Multiplication' 
 
- Floating point
 
- Floating point 'Multiplication'
 
- Gray code + Floating point 
 
     
    
cryptsetupjust reads a file as a key. It doesn't care what's in the file. It could be ascii, or arbitrary binary, it makes no difference.