What I can't figure out is how to emulate this scenario in Ruby. I can't figure how to get Ruby to report the character code (8-bit unsigned integer) 65279. If I try to pack it, this is what I get:

[65279].pack('C*')
# => "\xFF"

Here's what I get if I unpack the BOM:

"\xFE\xFF".unpack('C*')
=> [254, 255]

And if I write it to a file, Ruby always writes the original BOM, "\xEF\xBB\xFB"

File.write '/tmp/bom.txt', [65279].pack('U*'), mode: 'wb'

We may just have to accept this character code is a weird quirk of Node.js and deal with it as such. There are translations going on we just can't see.

I'm now convinced it's best to deal with this in Asciidoctor.js. Our custom unpack method is an ideal place for it. Here's the logic we could use that would make this work:

`self.charCodeAt() === 65279 ? [239, 187, 191] : #{self[0, 3].bytes.select.with_index {|_, i| i.even? }}`

Let me know if you want me to proceed with a fix.

An alternate way to do this to check if self[0, 1].bytes is [255, 254]. Either way seems to work.

Here's something that's interesting:

encodeURIComponent(self.charAt())
// => %EF%BB%BF

and

encodeURIComponent(String.fromCharCode(65279))
// => %EF%BB%BF

So it is recognizing 65279 as a UTF-8 BOM.

I think checking the char code is the right strategy (as proposed above). It's the same as this, with a bit less overhead:

self.charAt() === String.fromCharCode(65279)