Following on a comment by OzanOzan, it seemed like I was using a wrong matrix to convert XYZ to RGB. Since sRGB is the default color space on the web (or is it?), I'm now using the correct matrix followed by a gamma correction function (a = 0.055).



