Blog is available in backup mode, go to github-pages for newer posts
Back to articles listUpdated on 23. June 2014
Last weekend I watched a video about Java8 and Nashorn. This is a much faster successor to Rhino by Oracle. They claimed to have passing all of more than 11000 tests of es5 spec, even though no browser vendor made compatibility to so scrupulous degree. The presentation was from late 2012, hence I decided to check, how things are going for now.
You can run them by yourself here, but I already did that for most popular browsers and even not popular ones. And moreover, I found that tests often fail not because of the feature in description is implemented incorrectly. Simpler, some tests are not independent. Extrapolating that to all tests would be an survival bias. Hence I'm not saying all tests are badly written. But if one wants to interpret the fact of test fail, reading description is not enough. On the bad note, though, a pair of tests are out of tune with specification. They expect some certain behavior, when it is clearly stated that it may be implementation-dependent.
Here's a hg repo viewer, where separate tests could be found.
If you don't want to go deep into details about different tests and my rants about their quality, here's a list what you'd better not to do from es5:
window
[[Writable]]
, [[Configurable]]
and [[Enumerable]]
are incorrectly set to false
when not provided in re-defining properties on window
- just pass them for every defineProperty
call.this
are still wrapped into objects - in special cases or in browsers (IE9) which, do not support strict mode at all. I personally faced a bug emerged from combination of jQuery.each
on array of primitives and using strict equality for this
.defineProperty
in Safari 5Date.toString
, hence do not rely on exact values.parseInt(n, 10)
or Number(n)
if you're sure there're no trailing characters.String.prototype.localeCompare
returns incorrect results for non-canonical Unicode representations in many browsersArray.prototype
is not only difficult to understand, but is also vulnerable to subtle differences in implementationsListing full test names could be too "noisy" for visual perception, hence I compacted several names with syntax similar to imports in es6. E.g.: "1.1.2, 1.1.3" is written as "1.1.{2,3}".
First of all I need to mention, that six Date
tests do not fail if and only if you live California or elsewhere near. Relax, it is neither discrimination, nor location tracking - just OS setting, timezone should be UTC-8.
15.5.4.9_CE
Tests that
String.prototype.localeCompare
returns 0 when comparing Strings that are considered canonically equivalent by the Unicode standard
All browsers except Chrome and Firefox 29 beta fails that test.
Interesting enough, strings from test case ("\u1111\u1171\u11B6" vs "\ud4db") are even displayed differently in FF29- (and strangely Chrome33-34): the first string looks like linear Hangul, whilst the second one is normal Jamo:
퓛 vs 퓛
OK, mainstream browsers go first in some arbitrary order.
11.2.3-3_3
Call arguments are not evaluated before the check is made to see if the object is actually callable (undefined member)
I.e. in expression ({}).b(f());
engine should call f()
before .b
is evaluated and detected as non-callable, whilst for ({}).a.b(f());
browser should not call f()
. In the second case Chrome incorrectly calls f()
as well.
S15.8.2.8_A6
Checking if Math.exp is approximately equals to its mathematical values on the set of 64 argument values; all the sample values is calculated with LibC
Chrome35 have similar bugs in 3 more tests.
The relative error is 2-50, while precision of IEEE754 doubles used in javascript, is 52 bits. So this error hardly matters, unless you're doing a REAL SCIENCE. Moreover, specification [15.8.2] doesn't impose any certain requirements on computation accuracy for math functions:
The behaviour of the functions
acos
,asin
,atan
,atan2
,cos
,exp
,log
,pow
,sin
,sqrt
, andtan
is not precisely specified here except to require specific results for certain argument values that represent boundary cases of interest. For other argument values, these functions are intended to compute approximations to the results of familiar mathematical functions, but some latitude is allowed in the choice of approximation algorithms.
Test itself is rather arguable. Are those approximations enough or not? Even if this is a problem, it's not a big one.
10.2.1.1.3-4-{16,18}-s
TypeError
should be thrown when changing global.NaN
or global.undefined
in strict mode.
Surprisingly it doesn't fail the previous test.
S10.4.2_A1.{1,2}_T{1-11}
eval
within global execution context - scopes are not chained properly
15.5.4.9_3
Tests that
String.prototype.localeCompare
treats a missing "that" argument, undefined, and "undefined" as equivalent.
str.localeCompare()
, str.localeCompare(undefined)
and str.localeCompare("undefined")
should be the same for any possible value of str
.
10.4.3-1-{104,106}
Strict mode should not ToObject thisArg if not an object.
In es3 and non-strict es5, primitive passed as this
, are wrapped into object (String, Number or Boolean). In strict mode primitives should not be unwrapped. This works in general wherever strict mode is enabled and supported.
In getter, defined on Object.prototype
and used on primitive value (via boxing), this
still should be a primitive. And this is where Firefox fails.
Actually, if you write cross-browser code, you shouldn't rely on aforementioned feature of strict mode.
15.11.4.4-8-2
Error.prototype.toString
return empty string when 'name' is empty string and 'msg' is undefined
FF returns "Error" if name is empty. Not a serious bug, again.
Numerous bugs with Object.defineProperty
.
Sometimes (6 from 15.2.3.6-4-{292-296}) checks interfere with implementation details of mapping between arguments
object and actual named arguments like this:
(function (a) {
Object.defineProperty(arguments, '0', { value: 2 });
return a;
}(1));
BTW in strict mode all browsers demonstrate the same behavior as Firefox.
-354-4, -354-13, -360-3, -360-7, -531-4, -531-13, -538-3, -538-7 and 15.2.3.7-6-a-24
In these cases there's really an error. All 3 properties: writable
, enumerable
and configurable
become false
on re-definition if they're not provided. This is detected on window
only and doesn't happen for normal objects. -531-17 and -538-7 fails even earlier due to inability to redefine numerical properties on window.
15.4.4.{14-22}
Problems with Array.prototype
methods applied to global object.
The same reason as just mentioned - it fails because one cannot declare numerical keys on window
- they continue to be undefined
.
Actually, every method of Array.prototype
has a NOTE like:
The
reduce
function is intentionally generic; it does not require that its this value be an Array object. Therefore it can be transferred to other kinds of objects for use as a method. Whether thereduce
function can be applied successfully to a host object is implementation-dependent.
I.e. spec doesn't require ability to successfully apply those methods to host objects! So tests are just incorrect.
S15.4.4.4_A3_T1
[[Prototype]] of Array instance is Array.prototype, [[Prototype] of Array.prototype is Object.prototype
This test fails due to implementation details of concat
(not own property remains not own in Firefox).
15.4.4.4-5-c-i-1
Array.prototype.concat will concat an Array when index property (read-only) exists in Array.prototype (Step 5.c.i)
It fails because Array.prototype.concat.call(1)[0]
results in number
primitive, and not Number
object. The check of copy's properties existence actually passes.
The latest 11th version fails only one test about localeCompare
IE11 in IE10 emulation mode shows the same behavior.
It fails one more test, which is rather funny:
S15.5.4.7_A1_T11
Fails for eastern hemisphere (UTC+N timezones), cause IE does not display leading zeroes for dates: new Date(0).toString()
→ "Thu Jan 1 04:00:00 UTC+0400 1970"
Yeah, it sounds so arcane, but check in code is really like that:
(date.indexOf(date.getTimezoneOffset()>0 ? '31' : '01')) === 8
One way or another, it reveals a piece of non-standard behavior of IE.
IE9 fails significantly more tests additionally to aforementioned two.
And true IE9 shows results somewhat different from higher versions in corresponding emulation mode (surprise-surprise). The only good point about that is that both emulation modes (IE11@9 and IE10@9) works the same.
Of course, it also can't pass 500+ tests about strict mode. I didn't even check 'em all - just filtered out.
8.12.9-9-c-i_{1,2} (passes in emulation mode)
Redefine a configurable accessor property to be a data property on a non-extensible object
with preventExtensions
.
10.6-13-a-2 (passes in emulation mode)
A direct call to arguments.callee.caller should work
(function (called) {
function test1(flag) { if (!flag) test2(); else called = true; }
function test2() { arguments.callee.caller(true); }
test1();
return called;
}(false));
Rather tricky and strange.
11.2.3-3_{1-2,5-8}
Call arguments are evaluated before the check is made to see if the object is actually callable.
It fails differently than in Chrome.
15.11.4.{2,3}-1
Error.prototype.{name,message} are not enumerable
15.2.4.4-{1,2}
Using
Object.prototype.valueOf
for non-object values
Expected result: primitive value should be wrapped into corresponding object.
Workaround: if you need to wrap primitive into object, Object(value)
seems to be working properly everywhere.
S15.2.4.4_A{12-15}
Object.prototype.valueOf
invoked with null
/undefined
should throw an error.
15.4.4.4-5-c-i-1
Array.prototype.concat will concat an Array when index property (read-only) exists in Array.prototype (Step 5.c.i)
The same as in Firefox.
15.5.4.11-1 (passes in emulation mode)
'this' object used by the
replaceValue
function of aString.prototype.replace
invocation
S15.5.4.14_A1_T{6-9}, ...A2_T7
Split should not coerce its argument to string "undefined" if its argument is undefined
.
15.9.5.40_1
Date.prototype.setFullYear - Date.prototype is itself an instance of Date
Test just calls Date.prototype.setFullYear(2012);
and expects 2012
to be read back. Hardly you'll need that in practice, but this test fails only in IE9. All other higher and lower versions aren't affected.
They are not widespread at all for now, hence only short overviews for each. Nonetheless, number of such client will not drop abruptly in foresee future - because of vendor-lock or similar situation.
parseInt
still accepts octal.JSON.parse('{"__proto__":[]}');
returns object with 2 __proto__
properties!Surprisingly good, only 4 bugs.
parseInt
still accepts octal.[\u2000-\u23ff]
as those in regular expressions. (2 tests).String.prototype.localeCompare
does not treat missing 'that' argument as undefined.parseInt
still accepts octal.String.prototype.localeCompare
does not treat missing 'that' argument as undefined.JSON.parse('{"__proto__":[]}');
returns object with 2 __proto__
properties!String.prototype.localeCompare
does not treat missing 'that' argument as undefined.function.length
for fn instances.Array.prototype
methods applied to the global object (window
)After few hours of night hacking I made test suite running in IE8 - there're several things, which break in the GUI and test runner itself.
Of course, there're no es5 features, but they could be polyfilled to some extent. Nonetheless, some bugs due to subtle discrepancies in syntax parsing can arise.
Bugs are limitless, but I'll try to summarize them in concise way. Keep in mind, that most bugs from IE9 are also reproduced, but they're too minor comparing to the HELL happening below.
Incompatibilities described in terms how IE8 behaves. Correct behavior is either stated explicitly or considered obvious.
eval
call does not set scope to the global contextif
blockcatch
block TypeError
is used instead of ReferenceError
for referencing variabledelete
behave differently in unusual positions. Like delete foo();
delete
operator, workaroundlength
variable with var
keyword in a global scope. Otherwise you'll get an error, trying to set it.prototype
property.jQuery("<div>", { class: "someClass" })
jQuery(someSelector).css({ float: "left" })
let
, interface
and so on.if (...) { ... }; else
do {}; while(..)
try {...}; finally{...}
\\v
VERTICAL TAB, U+2028
LINE SEPARATOR and U+2029
PARAGRAPH SEPARATOR can mess your code everywhere: in code itself, RegExp literals, comments.\s
. E.g. U+00A0
or separators from previous point.U+0130
is considered as \w
(ru-RU locale)/\1(A)(B)\2/.exec("\1ABB");
vs /\1(A)(B)\2/.exec("ABB");
/[^]/
should match anything and /[]/
should match nothing, RegExpError
is thrown insteadRegExpError
is thrown instead of SyntaxError
for wrong syntaxRegExpError
is thrown instead of TypeError
for wrong flags/./ii
) and dynamic construction via RegExp
""
instead of undefined
/(z)((a+)?(b+)?(c))*/.exec("zaacbbbcac") (z)((aa)( ) (c)) (( ) (bbb)(c)) ((a) ( ) (c)) ^ ^ ^ a bbb c
Here "bbb"
incorrectly "fall-through": match[4]
should be undefined
or at least ""
parseInt
still accepts octalNumber.prototype.toString(undefined)
fails with errorNumber.prototype.toFixed(Infinity)
should fail with TypeError
(1e21).toFixed()
should return 1e+21
, not 1000000000000000000000
indexOf
and lastIndexOf
as well as other functional stuff.Array.isArray
is missingarray.slice(n, undefined)
always returns an empty array[].sort(undefined)
fails with errorArray.prototype.splice
on objects doesn't remove valuestrim
and lastIndexOf
methods are missing"".match()
returns null
instead of ""
"".search()
returns null
instead of 0
separator
argument is regex: "101101".split(/1/) === ["0", "0"]
"010".split(/(1)/) === ["0", "0"]
split
should not coerce to string undefined
passed as separator
argumentsplit
should coerce limit
argument with ToUint32
length
value for fromCharCode
, replace
, slice
, search
, indexOf
and lastIndexOf
methodsObject
are not supported. defineProperty
and getOwnPropertyDescriptor
are present. Alas, MSDN tells us that function ... supports DOM objects but not user-defined objects. The enumerable and configurable attributes can be specified, but they are not used
Date.prototype.toISOString
is missingObject.prototype.toString
incorrectly returns "[object Object]"
when applied to window
, DOM nodes, undefined
or null
.U+0010
through U+001F