Ruby unable to parse Json

Question

The Problem

Trying to convert json file into a hash. I get parsing error. This has been driving me nuts for days. More confusingly that my code works on one computer, but that same exact code does not work on the other.

I've used various contents of the JSON file itself. Even simple one line json fails. Oddly enough, if I copy/paste that same JSON contents into a literal string instead of parsing from file, it works just fine -_- .

My Code

require 'json'
read_json = File.read('DirectRoutingTable.json')
hashed_json = JSON.parse(read_json)
puts hashed_json

Contents of JSON File

{ "name": "manni"}

Error message

/usr/lib/ruby/vendor_ruby/json/common.rb:155:in \`parse': 757: unexpected token at '{ "name": "manni"} (JSON::ParserError)

'

from /usr/lib/ruby/vendor_ruby/json/common.rb:155:in \`parse'

from 5example.rb:3:in \`<main>'

Works on my computer - Specs

• ruby 1.9.3p551 (2014-11-13) [i386-mingw32]
• json (1.8.3)

Fails on VM where I need it to run - Specs

• ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-linux]
• json (1.8.3)

---

UPDATE 3pm So I decided to loop through each character having each character pass through an if condition regex ([A-Za-z0-9:-_ \n{}\"\']). Basically I include all characters that I use in the json. One character gets excluded which prints as nothing in console, the ordinal of the character is 65279, and it is right in the beginning of my string. When I parse json excluding the first character, it works just fine. So my inelegant solution to get it working is to change the file read line to

File.read('DirectRoutingTable.json')[1..-1]

So 2 questions

• What the hell is character 65279?
• Why does File.read() include this character?

Code I used to determine the problem require 'json' read_json = File.read('foo.json')

new_string = ''

puts '********************************************'
counter = 0

read_json.split("").each do |ch|
  print 'Character Number: ' + counter.to_s + "\n"
  print "Character: '" +  ch + "'\n"
  print "Character value: '" +  ch.ord.to_s + "'\n"
  if ch =~ /[A-Za-z0-9:\-\_ \n\{\}\"\']/
    puts 'Matches new string: Yes'
  else
    puts 'Matches new string: No'
  end
  puts '*'
  counter = counter + 1
end

Output

Character Number: 0
Character: ''
Character value: '65279'
Matches regex: No
*
Character Number: 1
Character: '{'
Character value: '123'
Matches regex: Yes
*
Character Number: 2
Character: ' '
Character value: '32'
Matches regex: Yes
*
Character Number: 3
Character: '"'
Character value: '34'
Matches regex: Yes
*
Character Number: 4
Character: 'n'
Character value: '110'
Matches regex: Yes
*
Character Number: 5
Character: 'a'
Character value: '97'
Matches regex: Yes
*
Character Number: 6
Character: 'm'
Character value: '109'
Matches regex: Yes
*
Character Number: 7
Character: 'e'
Character value: '101'
Matches regex: Yes
*
Character Number: 8
Character: '"'
Character value: '34'
Matches regex: Yes
*
Character Number: 9
Character: ':'
Character value: '58'
Matches regex: Yes
*
Character Number: 10
Character: ' '
Character value: '32'
Matches regex: Yes
*
Character Number: 11
Character: '"'
Character value: '34'
Matches regex: Yes
*
Character Number: 12
Character: 'm'
Character value: '109'
Matches regex: Yes
*
Character Number: 13
Character: 'a'
Character value: '97'
Matches regex: Yes
*
Character Number: 14
Character: 'n'
Character value: '110'
Matches regex: Yes
*
Character Number: 15
Character: 'n'
Character value: '110'
Matches regex: Yes
*
Character Number: 16
Character: 'i'
Character value: '105'
Matches regex: Yes
*
Character Number: 17
Character: '"'
Character value: '34'
Matches regex: Yes
*
Character Number: 18
Character: '}'
Character value: '125'
Matches regex: Yes
*
Character Number: 19
Character: '
'
Character value: '10'
Matches regex: Yes
*

Answer 1

Thank you @HomePlaneR

Now knowing that the mystery character causing all the issues is a BOM the real elegant solution is reading the file properly. The following code works.

require 'json'
json_file = File.open('DirectRoutingTable.json', 'r:bom|utf-8')
read_json = json_file.read
hashed_json = JSON.parse(read_json)
puts hashed_json

Answer 2

"65279" character is byte order mark.

https://en.wikipedia.org/wiki/Byte_order_mark

Some parsers might have issues when it is present in the input stream. You can remove it from your JSON file with a text editor like Notepad++.