Blog is available in backup mode, go to github-pages for newer posts

Back to articles list

State of affairs with es5

Updated on 23. June 2014

Last weekend I watched a video about Java8 and Nashorn. This is a much faster successor to Rhino by Oracle. They claimed to have passing all of more than 11000 tests of es5 spec, even though no browser vendor made compatibility to so scrupulous degree. The presentation was from late 2012, hence I decided to check, how things are going for now. You can run them by yourself here, but I already did that for most popular browsers and even not popular ones. And moreover, I found that tests often fail not because of the feature in description is implemented incorrectly. Simpler, some tests are not independent. Extrapolating that to all tests would be an survival bias. Hence I'm not saying all tests are badly written. But if one wants to interpret the fact of test fail, reading description is not enough. On the bad note, though, a pair of tests are out of tune with specification. They expect some certain behavior, when it is clearly stated that it may be implementation-dependent.
Here's a hg repo viewer, where separate tests could be found.

TL;DR

If you don't want to go deep into details about different tests and my rants about their quality, here's a list what you'd better not to do from es5:

Detailed analysis of failed tests

Listing full test names could be too "noisy" for visual perception, hence I compacted several names with syntax similar to imports in es6. E.g.: "1.1.2, 1.1.3" is written as "1.1.{2,3}".

First of all I need to mention, that six Date tests do not fail if and only if you live California or elsewhere near. Relax, it is neither discrimination, nor location tracking - just OS setting, timezone should be UTC-8.

15.5.4.9_CE

Tests that String.prototype.localeCompare returns 0 when comparing Strings that are considered canonically equivalent by the Unicode standard

All browsers except Chrome and Firefox 29 beta fails that test.
Interesting enough, strings from test case ("\u1111\u1171\u11B6" vs "\ud4db") are even displayed differently in FF29- (and strangely Chrome33-34): the first string looks like linear Hangul, whilst the second one is normal Jamo:
퓛 vs

OK, mainstream browsers go first in some arbitrary order.

Chrome 32+

11.2.3-3_3

Call arguments are not evaluated before the check is made to see if the object is actually callable (undefined member)

I.e. in expression ({}).b(f()); engine should call f() before .b is evaluated and detected as non-callable, whilst for ({}).a.b(f()); browser should not call f(). In the second case Chrome incorrectly calls f() as well.

S15.8.2.8_A6

Checking if Math.exp is approximately equals to its mathematical values on the set of 64 argument values; all the sample values is calculated with LibC

Chrome35 have similar bugs in 3 more tests.

The relative error is 2-50, while precision of IEEE754 doubles used in javascript, is 52 bits. So this error hardly matters, unless you're doing a REAL SCIENCE. Moreover, specification [15.8.2] doesn't impose any certain requirements on computation accuracy for math functions:

The behaviour of the functions acos, asin, atan, atan2, cos, exp, log, pow, sin, sqrt, and tan is not precisely specified here except to require specific results for certain argument values that represent boundary cases of interest. For other argument values, these functions are intended to compute approximations to the results of familiar mathematical functions, but some latitude is allowed in the choice of approximation algorithms.

Test itself is rather arguable. Are those approximations enough or not? Even if this is a problem, it's not a big one.

Safari 7 (MacOS/iOS)

10.2.1.1.3-4-{16,18}-s
TypeError should be thrown when changing global.NaN or global.undefined in strict mode.

Safari 6 (iOS)

Surprisingly it doesn't fail the previous test.

S10.4.2_A1.{1,2}_T{1-11}
eval within global execution context - scopes are not chained properly

15.5.4.9_3

Tests that String.prototype.localeCompare treats a missing "that" argument, undefined, and "undefined" as equivalent.

str.localeCompare(), str.localeCompare(undefined) and str.localeCompare("undefined") should be the same for any possible value of str.

Firefox 27-33

10.4.3-1-{104,106}

Strict mode should not ToObject thisArg if not an object.

In es3 and non-strict es5, primitive passed as this, are wrapped into object (String, Number or Boolean). In strict mode primitives should not be unwrapped. This works in general wherever strict mode is enabled and supported.
In getter, defined on Object.prototype and used on primitive value (via boxing), this still should be a primitive. And this is where Firefox fails.

Actually, if you write cross-browser code, you shouldn't rely on aforementioned feature of strict mode.

15.11.4.4-8-2

Error.prototype.toString return empty string when 'name' is empty string and 'msg' is undefined

FF returns "Error" if name is empty. Not a serious bug, again.

Numerous bugs with Object.defineProperty.

Sometimes (6 from 15.2.3.6-4-{292-296}) checks interfere with implementation details of mapping between arguments object and actual named arguments like this:

(function (a) {
    Object.defineProperty(arguments, '0', { value: 2 });
    return a;
}(1));

BTW in strict mode all browsers demonstrate the same behavior as Firefox.

-354-4, -354-13, -360-3, -360-7, -531-4, -531-13, -538-3, -538-7 and 15.2.3.7-6-a-24
In these cases there's really an error. All 3 properties: writable, enumerable and configurable become false on re-definition if they're not provided. This is detected on window only and doesn't happen for normal objects. -531-17 and -538-7 fails even earlier due to inability to redefine numerical properties on window.

15.4.4.{14-22}
Problems with Array.prototype methods applied to global object.
The same reason as just mentioned - it fails because one cannot declare numerical keys on window - they continue to be undefined.

Actually, every method of Array.prototype has a NOTE like:

The reduce function is intentionally generic; it does not require that its this value be an Array object. Therefore it can be transferred to other kinds of objects for use as a method. Whether the reduce function can be applied successfully to a host object is implementation-dependent.

I.e. spec doesn't require ability to successfully apply those methods to host objects! So tests are just incorrect.

S15.4.4.4_A3_T1

[[Prototype]] of Array instance is Array.prototype, [[Prototype] of Array.prototype is Object.prototype

This test fails due to implementation details of concat (not own property remains not own in Firefox).

15.4.4.4-5-c-i-1

Array.prototype.concat will concat an Array when index property (read-only) exists in Array.prototype (Step 5.c.i)

It fails because Array.prototype.concat.call(1)[0] results in number primitive, and not Number object. The check of copy's properties existence actually passes.

IE11

The latest 11th version fails only one test about localeCompare

IE10

IE11 in IE10 emulation mode shows the same behavior.

It fails one more test, which is rather funny:
S15.5.4.7_A1_T11
Fails for eastern hemisphere (UTC+N timezones), cause IE does not display leading zeroes for dates: new Date(0).toString() → "Thu Jan 1 04:00:00 UTC+0400 1970"
Yeah, it sounds so arcane, but check in code is really like that:
(date.indexOf(date.getTimezoneOffset()>0 ? '31' : '01')) === 8
One way or another, it reveals a piece of non-standard behavior of IE.

IE9

IE9 fails significantly more tests additionally to aforementioned two. And true IE9 shows results somewhat different from higher versions in corresponding emulation mode (surprise-surprise). The only good point about that is that both emulation modes (IE11@9 and IE10@9) works the same.
Of course, it also can't pass 500+ tests about strict mode. I didn't even check 'em all - just filtered out.

8.12.9-9-c-i_{1,2} (passes in emulation mode)

Redefine a configurable accessor property to be a data property on a non-extensible object

with preventExtensions.

10.6-13-a-2 (passes in emulation mode)

A direct call to arguments.callee.caller should work

(function (called) {
    function test1(flag) { if (!flag) test2(); else called = true; }
    function test2() { arguments.callee.caller(true); }
    test1();
    return called;   
}(false));

Rather tricky and strange.

11.2.3-3_{1-2,5-8}

Call arguments are evaluated before the check is made to see if the object is actually callable.

It fails differently than in Chrome.

15.11.4.{2,3}-1

Error.prototype.{name,message} are not enumerable

15.2.4.4-{1,2}

Using Object.prototype.valueOf for non-object values

Expected result: primitive value should be wrapped into corresponding object. Workaround: if you need to wrap primitive into object, Object(value) seems to be working properly everywhere.

S15.2.4.4_A{12-15}
Object.prototype.valueOf invoked with null/undefined should throw an error.

15.4.4.4-5-c-i-1

Array.prototype.concat will concat an Array when index property (read-only) exists in Array.prototype (Step 5.c.i)

The same as in Firefox.

15.5.4.11-1 (passes in emulation mode)

'this' object used by the replaceValue function of a String.prototype.replace invocation

S15.5.4.14_A1_T{6-9}, ...A2_T7
Split should not coerce its argument to string "undefined" if its argument is undefined.

15.9.5.40_1

Date.prototype.setFullYear - Date.prototype is itself an instance of Date

Test just calls Date.prototype.setFullYear(2012); and expects 2012 to be read back. Hardly you'll need that in practice, but this test fails only in IE9. All other higher and lower versions aren't affected.

Old non-IEs

They are not widespread at all for now, hence only short overviews for each. Nonetheless, number of such client will not drop abruptly in foresee future - because of vendor-lock or similar situation.

Safari 5 (Win)

Opera 12

Surprisingly good, only 4 bugs.

Android 4.3 (Galaxy S3)

Android 4.1 (ASUS fonepad)

IE8

After few hours of night hacking I made test suite running in IE8 - there're several things, which break in the GUI and test runner itself.

Of course, there're no es5 features, but they could be polyfilled to some extent. Nonetheless, some bugs due to subtle discrepancies in syntax parsing can arise.

Bugs are limitless, but I'll try to summarize them in concise way. Keep in mind, that most bugs from IE9 are also reproduced, but they're too minor comparing to the HELL happening below.

Incompatibilities described in terms how IE8 behaves. Correct behavior is either stated explicitly or considered obvious.

Scope

Syntax

Regex

/(z)((a+)?(b+)?(c))*/.exec("zaacbbbcac")
(z)((aa)( )  (c))
   (( ) (bbb)(c))
   ((a) ( )  (c))
     ^   ^    ^
     a   bbb  c

Here "bbb" incorrectly "fall-through": match[4] should be undefined or at least ""

Numbers

Arrays

Strings

Misc

... supports DOM objects but not user-defined objects. The enumerable and configurable attributes can be specified, but they are not used