How to get a UTF8 encoded string from a Base64 encoded string in Javascript?

Question

AEEAQgBDAGEAYgBj is Base64 for ABCabc however when I run this code:

let a = Buffer.from('AEEAQgBDAGEAYgBj', 'base64').toString('utf-8');
console.log(Buffer.from(a, 'utf8'));

the printed result in the console is <Buffer 00 41 00 42 00 43 00 61 00 62 00 63> which is UTF16 (LE).

I would assume that since I am creating a Buffer from a UTF8 encoded string the result would be <Buffer 41 42 43 61 62 63>. So how can I get an actual UTF8 encoded string from Base64?

Answer 1

The problem is that your original data is Base64 encoded UTF16-BE. If you look at a after your first line, you'll see that it has those zero bytes that you see in the final buffer:

let a = Buffer.from("AEEAQgBDAGEAYgBj", "base64").toString("utf-8");
console.log(a.length);
// 12
console.log([...a].map(ch => ch.charCodeAt(0).toString(16).padStart(2, "0")).join(" "));
// 00 41 00 42 00 43 00 61 00 62 00 63

So the question becomes: How to read the UTF16-BE text you have in the buffer from Buffer.from("AEEAQgBDAGEAYgBj", "base64"). Node.js's Buffer doesn't support UTF16-BE directly (there is no "utf16be" encoding in its standard library), but you can get there via swap16 and then reading the buffer as UTF16-LE ("utf16le", which is in Node.js's standard library):

let a = Buffer.from("AEEAQgBDAGEAYgBj", "base64").swap16().toString("utf16le");
console.log(a.length);
// 6
console.log(a);
// ABCabc

Now a is a normal string. If you want a buffer containing its contents in UTF8, you can use Buffer.from(a).toString("utf8"):

let a = Buffer.from("AEEAQgBDAGEAYgBj", "base64").swap16().toString("utf16le");
console.log(a.length);
// 6
console.log(a);
// ABCabc
let b = Buffer.from(a); // (Default is `"utf8"` but you could supply that explicitly)
console.log(b);
// <Buffer 41 42 43 61 62 63>