10 years of Dart

by Vyacheslav Egorov

[email protected]

Evolution of the Dart platform


                                     dynamic                static
                        ──────────────────────────── ──────────
                        2011     2013     2015         2018     2020
                        ──────────────────────────────────────►
                        0.x       1.0                  2.0      2.12
                        ───────JIT────── ──────JIT or AOT───────
                                                  ╱        ╲    NNBD
                                      development          deployment

Dart 1


                        Dog d = new Cat();  // ignored type annotations
                                            //   access to a field
                        d.f                 // ─ invocation of getter
                                            //  method tear-off
                                            //   noSuchMethod invocation

                        int x = null;       // no primitive types
                        (1 << 100) >> 100;  // == 1, int is bignum

Wikipedia describes Dart as «Dart is an object-oriented, class-based, garbage-collected language with C-style syntax.» which should give you a good idea of what to expect from it. If you want a proper introduction to the language check our language tour. Here I simply want to highlight few features of Dart 1 which make it challenging to compile ahead-of-time (AOT): in Dart 1 type annotations served more or less the role of comments, and were completely ignored during execution (outside of so called checked mode), this in turn meant that all method calls and property lookups were dynamically dispatched. Dart 1 has no primitive types and all reference types are nullable. Finally integers were arbitrary width in Dart 1.

Dart 1 was designed to be JITed


                                /* How to make a JIT */

                        ╔══════════╗               ┌──────────┐
                        ║ Compiler ║ ▶ generates ▶ │   Code   │
                        ╚══════════╝               └──────────┘
                             ▲                          │
                             ╰──────────────────────────╯
                                 execution feedback

These days there is no secret in how to make a JIT compiler for a language like Dart 1: to achieve good peak performance you need to setup an adaptive (re)compilation loop between your compiler(s) and the code they generate. Compiler produces the code, which collects execution profile and feeds it back to compiler which then produces new versions of code. There are of course some technicalities in how to set things up depending on constraints you operate in (e.g. if you need to optimize for startup latency or memory footprint), but the overall structure is the same no matter which high performance VM you look at.

2012 Dart's artisanal JIT

— generate unoptimized code;

— collect type feedback through inline caches;

— generate optimized code using type feedback;

— rinse and repeat.


                            // call-site specific cache
                      ─── ICData { ... }
                     ╱
                 dog.bark();
                     ╲      // generic dispatch stub
                      ─── dispatch(icData, receiver, ...) { ... }

Adaptive JITs with speculative optimizations are well understood these days, so I don't want to spend too much time discussing Dart 1 JIT architecture. I do however want to touch on a fundamental piece: inline caching. In Dart VM each call site (and remember that property loads are also call-sites due to dynamic nature of the language) has two pieces associated with it: a call-site specific cache and a dispatch stub (usually written in assembly). Each time call-site is reached stub would check if cache already contains method resolution result for the receiver's class and dispatch to it if present, otherwise (if cache does not contain the mapping) stub would invoke runtime to update the cache.

`d.bark()` (unoptimized)


                mov RBX, [PP + 0x7f] ; ICData
                call [PP + 0x87]     ; dispatch stub
                ;       ╲
                ;         instead of embedding constants
                ;          into the code we access them
                ;          through object pool

Here is how a call site looked like in machine code. First you lood call-site specific cache into a fixed register (here RBX, prescribed by dispatch stub calling convention). Then you invoke the dispatch stub. One thing I want to highlight here is that we don't embed any references to GC managed objects directly into the generated code - instead an indirection through an auxiliarly object, reachable through a dedicated register is used. This will become relevant a bit later in the talk.


                      ─── ICData { }
                     ╱
                 dog.bark();


                      ─── ICData { Dog: Dog.bark (1) }
                     ╱
                 dog.bark();

When call-site is reached the cache will be updated with a mapping from observed receiver class to the result of method resolution on that class. Here we have reached call-site once with an instance of Dog and resolving bark on Dog yields method Dog.woof.


                      ─── ICData { Dog: Dog.bark (2) }
                     ╱
                 dog.bark();

We executed the same call-site twice now with receivers being instances of the same class (Dog).


                      ─── ICData { Dog: Dog.bark (3) }
                     ╱
                 dog.bark();

We executed the same call-site 3 times now with receivers being instances of the same class (Dog). Inline caches collect frequences to facilitate frequency based optimizations for polymorphic call-sites (e.g. inlining the hottest alternative).


                      ─── ICData { Dog: Dog.bark (555) }
                     ╱
                 dog.bark();

`d.bark()` (optimized, monomorphic)


                ;; Check that RAX contains a Dog
                test al, 0x1
                jz ->deopt  ; deopt if RAX is a Smi
                movzx rcx, word ptr [rax + 0x1]
                cmp rcx, 0x53a
                jnz ->deopt
                ;; Call Dog.bark directly.
                mov r10, qword ptr [PP + 0x5f] ; argdesc
                call qword ptr [PP + 0x67]

`d.bark()` (optimized, monomorphic)


                ;; Check that RAX contains a Dog
                test al, 0x1
                jz ->deopt  ; deopt if RAX is a Smi
                movzx rcx, word ptr [rax + 0x1]
                cmp rcx, 0x53a
                jnz ->deopt
                ;; Call Dog.bark directly.
                mov r10, qword ptr [PP + 0x5f] ; argdesc
                call qword ptr [PP + 0x67]

Optimizing compiler transforms this call-site into a check which verifies the assumption that receiver is indeed an instance of Dog. If checks fails we would deoptimize, switch back to unoptimized code.

`d.bark()` (optimized, monomorphic)


                ;; Check that RAX contains a Dog
                test al, 0x1
                jz ->deopt  ; deopt if RAX is a Smi
                movzx rcx, word ptr [rax + 0x1]
                cmp rcx, 0x53a
                jnz ->deopt
                ;; Call Dog.bark directly.
                mov r10, qword ptr [PP + 0x5f] ; argdesc
                call qword ptr [PP + 0x67]

The check is followed by a direct invocation of Dog.bark method. Note that while this invocation looks very similar to how it looked like in unoptimized code, it does not do the same thing. In unoptimized code we were calling dispatch stub, while in optimized code we are invoking target method directly.


                                 dynamic
                    ──────────────────────────────────────────
                    2011     2013
                    ─────────────────────────────────────────►
                    0.x       1.0
                    ───────JIT─────────────────────────────────


                                 dynamic
                    ──────────────────────────────────────────
                    2011     2013
                    ────────────────────────────────────────►
                    0.x       1.0  │
                    ───────JIT────────────────────────────────
                                   │
                                Flutter

«Flutter is Google’s UI toolkit for building beautiful, natively compiled applications for mobile, web, and desktop from a single codebase.»

— https://flutter.dev

«Flutter is Google’s UI toolkit for building beautiful, natively compiled applications for mobile, web, and desktop from a single codebase.»

— https://flutter.dev

No JITing on iOS

(... unless you have special entitlements)

Need AOT!

2015 JIT ⇒ AOT

disable speculations;
compile all potentially reachable code;
serialize heap

snapshot /ˈsnæpˌʃɒt/

(noun) serialized form for heap object graphs, used for faster startup or exchanging data between isolates. Similar to Smalltalk images. Only supported data, not machine code.

          instructions               pool
      ┌─────────────────────┐    ┌─┬─┬─┬─┬─┬─┬─
      │ object_pool_ ──────────┤ │ │ │ │ ││
      ├─────────────────────┤    └─┴─┴─┴─┴─┴┴─
entry │ mov PP, [RIP-0x25]  │               │
      │ ...                 │            ┌──┴───┐
      │ mov rax, [PP+0x2f]  │            │object│
      │ ...                 │            └──────┘
      └─────────────────────┘

To serialize machine code we need to make it relocatable. However it turns out that machine code was already almost relocatable - because it did not address any heap allocation objects directly, but rather through an indirection via object pool. In JIT each generated chunk of machine code (instructions object) had a pointer to its pool in the header, on entry we would use PC relative addressing to take pool from the header and put it in the dedicated register, the rest of the code would use this dedicated register to access constants. Object pool serialization/ deserialization could be handled by a builtin snapshot mechanism, we just need to figure out how to connect instruction to object pool in a relocatable manner.

          instructions (RX)          pool (RO)
      ┌─────────────────────┐    ┌─┬─┬─┬─┬─┬─┬─
      │ object_pool_ ──────────┤ │ │ │ │ ││
      ├─────────────────────┤    └─┴─┴─┴─┴─┴┴─
entry │ mov PP, [RIP-0x25]  │               │
      │ ...                 │            ┌──┴───┐
      │ mov rax, [PP+0x2f]  │            │object│
      │ ...                 │            └──────┘
      └─────────────────────┘

You might suggest the following: instructions reside in read-execute (.text) section of the binary, while pool resides in read-only (.rodata) section, OS specific runtime linker is then responsible for relocating object_pool_ reference. This design might indeed work in some situations, however it has various drawbacks - and most importantly it is not suitable for Dart at all.

Dart Isolates


╔════════════════════════╗  ╔════════════════════════╗
║ Isolate    ┌─────────┐ ║  ║ Isolate    ┌─────────┐ ║
║           ┌─────────┐│ ║  ║           ┌─────────┐│ ║
║          ┌─────────┐│┘ ║  ║          ┌─────────┐│┘ ║
║          │         │┘  ║  ║          │         │┘  ║
║          └─────────┘   ║  ║          └─────────┘   ║
╚════════════════════════╝  ╚════════════════════════╝

Dart Isolates


╔════════════════════════╗  ╔════════════════════════╗
║ Isolate    ┌─────────┐ ║  ║ Isolate    ┌─────────┐ ║
║           ┌─────────┐│ ║  ║           ┌─────────┐│ ║
║          ┌─────────┐│┘ ║  ║          ┌─────────┐│┘ ║
║          │ Pool    │┘  ║  ║          │ Pool    │┘  ║
║          └─────────┘   ║  ║          └─────────┘   ║
╚════════════════════════╝  ╚════════════════════════╝

          instructions (RX)   ╏      pool (RW)
      ┌─────────────────────┐ ╏  ┌─┬─┬─┬─┬─┬─┬─
      │ object_pool_ ──────────┤ │ │ │ │ ││
      ├─────────────────────┤ ╏  └─┴─┴─┴─┴─┴┴─
entry │ mov PP, [RIP-0x25]  │ ╏             │
      │ ...                 │ ╏          ┌──┴───┐
      │ mov rax, [PP+0x2f]  │ ╏          │object│
      │ ...                 │ ╏ isolate  └──────┘
      └─────────────────────┘ ╏ heap

          instructions (RX)   ╏
                              ╏
                              ╏
      ┌─────────────────────┐ ╏
entry │ mov PP, [RIP-0x25]  │ ╏  ┌──────┐
      │ ...                 │ ╏  │ pool │
      │ mov rax, [PP+0x2f]  │ ╏  └──────┘
      │ ...                 │ ╏ isolate
      └─────────────────────┘ ╏ heap

object_pool_ has to disappear from the instructions object.

          instructions (RX)   ╏           ┌──────┐
                              ╏           │ code │
                 ┌────────────────┐       └────┘
      ┌──────────┴──────────┐ ╏   └─────────┘  │
entry │ mov PP, [RIP-0x25]  │ ╏  ┌──────┐      │
      │ ...                 │ ╏  │ pool ├──────┘
      │ mov rax, [PP+0x2f]  │ ╏  └──────┘
      │ ...                 │ ╏ isolate
      └─────────────────────┘ ╏ heap

Instead Code object would contain both a pointer to the executable machine code (instructions) and the pointer to pool.

          instructions (RX)   ╏           ┌──────┐
                              ╏           │ code │
                 ┌────────────────┐       └────┘
      ┌──────────┴──────────┐ ╏   └─────────┘  │
entry │ mov PP, [CR+pp_ofs] │ ╏  ┌──────┐      │
      │ ...                 │ ╏  │ pool ├──────┘
      │ mov rax, [PP+0x2f]  │ ╏  └──────┘
      │ ...                 │ ╏ isolate
      └─────────────────────┘ ╏ heap

Entry point can no longer find the pool pointer through PC relative addressing, instead we tweak calling conventions to pass code object in a dedicated register on all calls (rewriting them to an equivalent of code->entry(code, ...)), so that entry point could then load pool pointer from the code object itself.


                ; STATIC CALLS
                ; ═ BEFORE ═══════════════════════════════════════
                call [PP + 0x87]          ; direct call
                  mov PP, [RIP - 0x25]   ; in callee load PP

                ; ═ AFTER ════════════════════════════════════════
                mov CR, [PP + 0x87]       ; load Code object
                call [CR + entry_ofs]     ; call through Code
                  mov PP, [CR + pp_ofs]  ; in callee load PP

2015 JIT ⇒ AOT

disable speculations;
compile all potentially reachable code;
serialize heap ✓


                                 dynamic
                    ──────────────────────────────────────────
                    2011     2013     2015 2016
                    ───────────────────────────────────────►
                    0.x       1.0          └ we are here
                    ───────JIT────── ──────JIT or AOT────────
                                              ╱        ╲
                                  development          deployment

everything is a dynamic call

Optimizing Dart 1 AOT

inline caching;
inline fast paths;
local optimizations;
some global optimizations (CHA); remember: AOT was built JIT so it is not really well set for global optimizations


code (rx)         pool (rw)
                 ┌───┐
     ──────────▶│ ────▶ ICData
    ╱            ├───┤⬉
dog.bark();      │   │ can patch these
    ╲            ├───┤⬋
     ──────────▶│ ────▶ DispatchStub
                 ├───┤

Recall that every call-site has two things associated with it: call-site specific cache and dispatch stub. In JIT we would just update the cache (ICData object) and rely on optimizing compiler to specicialize the call-site. In AOT recompilation is not an option, but we can start looking at pool entries themselves as the cache, because both of these can be updated dynamically.


                  code (rx)         pool (rw)
                                   ┌───┐
                       ──────────▶│ ────▶ class Dog
                      ╱            ├───┤
                  dog.woof();      │   │
                      ╲            ├───┤     Dog.woof
                       ──────────▶│ ────▶ ┌────────────────────────────────┐
                                   ├───┤     │ if (this.class != icData)      │
                                      normal │   return monomorphicMiss(...); │
                                       entry⬊├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤

For example, on monomorphic call-sites (those that always see instances of the same class) we can patch pool entries corresponding to the call in such a way that we directly invoke the target method instead of going through the dispatch stub. In this case we will enter the method through a special checked entry, which would validate that receiver is an instances of a fixed expected class (passed to the entry in the same dedicated register which dispatch stub would use for ICData object). We called this switchable calls, but it is essentially the same technique as original inline caching described for Smalltalk-80 system. In Dart VM call-site could be switched between unlinked, monomorphic, range-based-single-target, polymoprhic and megamorphic states.

inline arithmetic fast paths

operators can be overridden
int is nullable
int is arbitrary precision


           ;; Y <- X + 1
           test X, 1
           jnz slow──┐
           mov Y, X   │
           add Y, 2   │
           jo slow───┤
     ┌─▶ done:        │
     │     ...        │
     │   slow: ◀──────┘ ;; "cold" code
     │     ;; ... X.operator+(1) call ...
     └────jmp done

This is an example of a CheckedSmi arithmetic operation which inlines fast path for small integer (tagged integer) receiver while keeping slow-path for all other cases: non-smi boxed integer, null or an object with operator+ override. This sort of inlining allows to reclaim some of the performance previously lost through calling overheads, but does not facilitate any further optimizations (the type of Y is unknown as it might originate from the slow path). It also leads to significant code bloat.

scraping the bottom of the barrel

2015 state of the union

	Development	Production
Native	JIT	JIT
Web	Dartium†	AOT (closed world)

^† Fork of Chromium with Dart VM embedded

2014 DDC & strong mode

Dartium was a maintenance liability
dart2js too slow
An idea was born to build a modular AOT compiler operating on a stronger typed Dart.

At that point it was becoming clear that the dream of native browsers embedding native Dart VM(s) alongside JS envisioned by Dart's founders Lars Bak and Kasper Lund is unlikely to become reality. Maintaining Dartium and rebasing it with each new Chromium version was becoming very hard. dart2js was too slow for interactive development due to its whole program nature. This led to the idea to build a Dart development compiler (DDC) which would compile application modularly to JS. However compiling Dart 1 modularly (and quickly) to a reasonably performant JS was considered hard which gave birth to an idea to define a statically typed subset of Dart.

2016 Dart Kernel & CFE

Pace of Dart language evolution was often hindered by duplication between different language implementations. A project is born to address that.


                              Dart VM (in C++)
                              ┌─────────────────────────┐
                           ┌─▶│ Parser ─▶ AST ─▶ ...  │
                           │  └─────────────────────────┘
                           │  dart2js (in Dart)
                 ╭────────╮  ┌─────────────────────────┐
                 │  Dart  ──▶│ Parser ─▶ AST ─▶ ...  │
                 │ Source │  └─────────────────────────┘
                 ╰────────╯│  dartanalyzer (in Dart)
                           │  ┌─────────────────────────┐
                           └─▶│ Parser ─▶ AST ─▶ ┌─────┐  some tools use
                              └────────────────────│┌─────┐ analyzer as a Dart
                                                   └│     │ parser
                                                    └─────┘


                                                     Dart VM
                                                     ┌──────┐
                                                  ┌─▶│ ...  │
                                                  │  └──────┘
                                CFE         Kernel│  dart2js
                 ╭────────╮   ┌────────┐   ╭─────╮  ┌──────┐
                 │  Dart  │──▶│ Parser │─▶│ AST ──▶│ ...  │
                 │ Source |   └────────┘   ╰─────╯  └──────┘
                 ╰────────╯   (in Dart)           ┊
                                                  ┊

2018 Dart 2 release

CFE/Kernel form front-end foundation
Strong mode replaces Dart 1 optional typing
int becomes 64-bit signed integer

Dart 1


                    Dog d = new Cat();  // ignored type annotations
                                        //   access to a field
                    d.f                 // ─ invocation of getter
                                        //  method tear-off
                                        //   noSuchMethod invocation

                    int x = null;       // no primitive types
                    (1 << 100) >> 100;  // == 1, int is bignum

Dart 1 ⇒ Dart 2


                    Dog d = new Cat();  // compile time error
                                        //
                    d.f                 // calls an implementation
                                        // of Dog.f
                                        //

                    int x = null;       // still no primitive types
                    (1 << 100) >> 100;  //  == 0, int is 64-bit

Dart 2 at least partially addressed some of them. Type annotations can now be trusted and are statically guaranteed by the compiler. Accessing a property f on a variable of type Dog is no longer fully dynamic: instead it is guaranteed that the receiver (which is known to be subtype of Dog) is going to have an implementation of get f and that implementation is a valid override of a getter get f declared in class Dog. This moves us from the real of dynamic dispatch to a real of interface dispatch.

What about VM's AOT?

Dart 2 impact on AOT

strong mode: types you can trust
Kernel: whole program AST

type flow analysis (TFA)

global context sensitive analysis over Kernel to annotate it with more accurate type information than what is available in static type annotations^†.

^† which specify nullable interface types

One thing to note about Dart 2 type annotations is that they only provide compiler with (nullable) interface types, a variable v of type X can contain a null or an instance of any class extending or implementing X. In trivial cases we might be able to combine this with CHA to establish that X is never extended or implemented allowing us to devirtualize method calls on v. However in many cases we need a more precise type information (and for numeric types non-nullability) to perform good optimization. That is why we build TFA.


groom(Animal a) {
  a.method();
}

main() {
  groom(Dog());
  feed(Cat());
}

Here for example if we just look at groom in isolation we would be forced to conservatively assume that a can be any subtype of Animal (including null), which would not allow us to devirtualize method call.


groom(Animal a) {
  a.method();  /* TFA: a is non-nullable Dog */
}    /* TFA: invokes Dog.method */

main() {
  groom(Dog());
  feed(Cat());
}

However TFA looking at this program under closed world assumption would be able to infer that a will always be a Dog, which would not allow us to devirtualize method call into a direct call to Dog.method.


f(int anInt, double aDouble) {
//      ╲            ╱
// conservative *boxed* representation
}

// Dart 1
f(IntegerDog(), FloatingCat());  // ooook

// Dart 2
f(null, 2.0);  // ok

This becomes especially important in context of numeric types, where with Dart 1 we did not have any information to base our optimizations on and with Dart 2, even though specification at least guarantees that numeric types are never extended or implemented, we still have to account for possibility of null, which prevents us from using unboxed representations.


f(int anInt, double aDouble) {
//      ╲            ╱
// In real code TFA can often prove
// that such variables are non-nullable
// allowing to use *unboxed* representation.
// ... btw this was an intern project!
}

devirtualization highlights costs of calling conventions


                ; a lot of code for a static call
                mov CR, [PP + 0x87]       ; load Code object
                call [CR + entry_ofs]     ; call through Code
                  mov PP, [CR + pp_ofs]  ; in callee load PP

Remember that each call needs to pass down a Code object so that callee could load its object pool from it. This is rather far in terms of overhead from direct calls in a language like C++ where we would just perform a relative call.

bare instructions

merge all object pools together
no longer need to call through Code object

To eliminate this overhead we run a project to switch from multiple object pools (one per instructions object) to a single global object pool (GOP). With GOP it is enough to populate PP register when entering Dart code (e.g. from C++) and Dart-to-Dart calls no longer need to change this register removing the need for loading it from Code object (or preserve caller's PP).


                    ; ═ BEFORE ═══════════════════════════════════════
                    mov CR, [PP + 0x87]       ; load Code object
                    call [CR + entry_ofs]     ; call through Code
                      mov PP, [CR + pp_ofs]  ; in callee load PP

                    ; ═ AFTER ════════════════════════════════════════
                    call #relative-offset-to-target

ICs are not that good at handling polymorphism

a lot of monomorphic call-sites are devirtualized by TFA

global dispatch table (GDT)

Table indexed by a combination of receiver's class id and selector id. Classical idea described in literature as row displacement dispatch tables.


   movzx cid, word ptr [obj + 15] ; load class id
   call [GDT + cid * 8 + #(sid * 8)]


┌────────────────────────┐ ╱
│ Selectors are numbered ├
│ such that GDT[cid+sid] │
│ gives dispatch target  │
└────────────────────────┘


   movzx cid, word ptr [obj + 15] ; load class id
   call [GDT + cid * 8 + #(sid * 8)]


┌────────────────────────┐ ╱
│ Naive numbering 0,     ├
│ NumCids, 2*NumCids, ...│
│ is too sparse          │
└────────────────────────┘

Imagine you have a list of selectors Sa, Sb, .... You can assign 0 as an id to Sa, 0 + NumCids to Sb and so on. Such numbering would by construction ensure that sid + cid is a unique index for all sid and cid. However it will also create and large sparse table.


   movzx cid, word ptr [obj + 15] ; load class id
   call [GDT + cid * 8 + #(sid * 8)]


┌────────────────────────┐ ╱
│ Selectors are numbered ├
│ in a way that tries to │
│ minimize GDT size.     │
└────────────────────────┘

An observation here is that not all combinations of (sid, cid) can occur in the program, as each call site will only ever see a subset of classes permitted by the static type of the receiver. This opens holes in the table which can be filled by carefully selecting selector ids. Imagine you have classes A, B, C, D, E (numbered 0 to 4 respectively). A and E answer selector F while B and C answer selector G, call-sites (A|E).G, (B|C).F are impossible and D does not participate in interface dispatch. Naive numbering of selectors would create a table with 10 elements. A good packing would instead produce [A.F, _, B.G, C.G, E.F] assigning F id 0 and G id 1.


                                 dynamic                static
                    ──────────────────────────── ──────────
                    2011     2013     2015         2018     2020
                    ─────────────────────────────────────
                    0.x       1.0  │               2.0│     2.12
                 vm ───────JIT───── ──────────AOT──────────
                                   │                  │
                                Flutter              We are here

Dart 1 ⇒ Dart 2


                    Dog d = new Cat();  // compile time error
                                        //
                    d.f                 // calls an implementation
                                        // of Dog.f
                                        //

                    int x = null;       // still no primitive types
                    (1 << 100) >> 100;  //  == 0, int is 64-bit

Dart 2 ⇒ Dart 2.12


                     
                     
                     
                     
                     
                    // types are now non-nullable by default
                    int x = null;          // compile time error

Dart 2 ⇒ Dart 2.12


                    // types are now non-nullable by default
                    int x = null;          // compile time error
                    int? x = null;         // ok
                    List<Dog>? dogs;
                    if (dogs != null) {
                        dogs.add(null);    // compile time error
                        dogs.add(Dog());   // ok
                    }

Reference types have to explicitly opt into allowing null via ? suffix. At the same time compiler will only allow method calls on non-nullable receiver (with exception of toString, get hashCode and operator == which are defined on null).

More^† optimization opportunities!

^† NNBD is incremental and provides an opt-out escape hatch. Compiler can only rely on static non-nullability if none of the libraries are opted out.

Demand for smaller downloads size and memory footprint.

Diverging AOT from JIT

compressed stack maps
DWARF stack traces

Unhandled exception:
Error occurred!
#0 causeError (test.dart:4)
#1 main (test.dart:9)

Unhandled exception:
Error occurred!
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 83265, tid: 4552355264, name Dart_Initialize
build_id: 'c0371757e9bbea5ce6a27b1f22e5a25a'
isolate_dso_base: 10f396000, vm_dso_base: 10f396000
isolate_instructions: 10f3a0000, vm_instructions: 10f398000
    #00 abs 000000010f3efd87 virt 0000000000059d87 _kDartIsolateSnapshotInstructions+0x4fd87
    #01 abs 000000010f3efd62 virt 0000000000059d62 _kDartIsolateSnapshotInstructions+0x4fd62

DWARF stack traces

Opens doors for dropping symbolic information and parts of program structure.

Demand for low-overhead concurrency.

Dart Isolates


╔════════════════════════╗  ╔════════════════════════╗
║ Isolate    ┌─────────┐ ║  ║ Isolate    ┌─────────┐ ║
║           ┌─────────┐│ ║  ║           ┌─────────┐│ ║
║ ┌───┐    ┌─────────┐│┘ ║  ║  ╌╌╌     ┌─────────┐│┘ ║
║ │MSG│    │ Program │┘  ║  ║ ╎   ╎    │ Program │┘  ║
║ └─┬─┘    └─────────┘   ║  ║  ╌▲╌     └─────────┘   ║
╚═══┆════════════════════╝  ╚═══┆════════════════════╝
    └╌╌╌╌╌╌╌╌╌╌╌ copying ╌╌╌╌╌╌╌┘

Dart Isolate Groups (new!)


                    ╔═════════════════════════════════════╗
                    ║ Isolate    Isolate      ┌─────────┐ ║
                    ║                        ┌─────────┐│ ║
                    ║ ┌───┐       ╌╌╌       ┌─────────┐│┘ ║
                    ║ │MSG│      ╎   ╎      │ Program │┘  ║
                    ║ └─┬─┘       ╌▲╌       └─────────┘   ║
                    ╚═══┆══════════┆══════════════════════╝
                        └╌ copying ┘

Dart Isolate Groups (new!)


                    ╔═════════════════════════════════════╗
                    ║ Isolate    Isolate      ┌─────────┐ ║
                    ║                        ┌─────────┐│ ║
                    ║ ┌───┐                 ┌─────────┐│┘ ║
                    ║ │MSG│                 │ Program │┘  ║
                    ║ └───┘                 └─────────┘   ║
                    ╚═════════════════════════════════════╝

Dart Isolate Groups (new!)


                    ╔═════════════════════════════════════╗
                    ║            Isolate      ┌─────────┐ ║
                    ║                        ┌─────────┐│ ║
                    ║ ┌───┐                 ┌─────────┐│┘ ║
                    ║ │MSG│                 │ Program │┘  ║
                    ║ └───┘                 └─────────┘   ║
                    ╚═════════════════════════════════════╝

Dart Isolate Groups (new!)


                    ╔═════════════════════════════════════╗
                    ║            Isolate      ┌─────────┐ ║
                    ║                        ┌─────────┐│ ║
                    ║   ┌───┐               ┌─────────┐│┘ ║
                    ║   │MSG│               │ Program │┘  ║
                    ║   └───┘               └─────────┘   ║
                    ╚═════════════════════════════════════╝

Dart Isolate Groups (new!)


                    ╔═════════════════════════════════════╗
                    ║            Isolate      ┌─────────┐ ║
                    ║                        ┌─────────┐│ ║
                    ║     ┌───┐             ┌─────────┐│┘ ║
                    ║     │MSG│             │ Program │┘  ║
                    ║     └───┘             └─────────┘   ║
                    ╚═════════════════════════════════════╝

Dart Isolate Groups (new!)


                    ╔═════════════════════════════════════╗
                    ║            Isolate      ┌─────────┐ ║
                    ║                        ┌─────────┐│ ║
                    ║       ┌───┐           ┌─────────┐│┘ ║
                    ║       │MSG│           │ Program │┘  ║
                    ║       └───┘           └─────────┘   ║
                    ╚═════════════════════════════════════╝

Dart Isolate Groups (new!)


                    ╔═════════════════════════════════════╗
                    ║            Isolate      ┌─────────┐ ║
                    ║                        ┌─────────┐│ ║
                    ║           ┌───┐       ┌─────────┐│┘ ║
                    ║           │MSG│       │ Program │┘  ║
                    ║           └───┘       └─────────┘   ║
                    ╚═════════════════════════════════════╝

10 years of Dart

Evolution of the Dart platform

Dart 1

Dart 1 was designed to be JITed

2012 Dart's artisanal JIT

d.bark() (unoptimized)

d.bark() (optimized, monomorphic)

d.bark() (optimized, monomorphic)

d.bark() (optimized, monomorphic)

No JITing on iOS

(... unless you have special entitlements)

Need AOT!

2015 JIT ⇒ AOT

snapshot /ˈsnæpˌʃɒt/

Dart Isolates

Dart Isolates

2015 JIT ⇒ AOT

everything is a dynamic call

Optimizing Dart 1 AOT

inline arithmetic fast paths

scraping the bottom of the barrel

2015 state of the union

2014 DDC & strong mode

2016 Dart Kernel & CFE

2018 Dart 2 release

Dart 1

Dart 1 ⇒ Dart 2

What about VM's AOT?

Dart 2 impact on AOT

type flow analysis (TFA)

devirtualization highlights costs of calling conventions

bare instructions

ICs are not that good at handling polymorphism

global dispatch table (GDT)

Dart 1 ⇒ Dart 2

Dart 2 ⇒ Dart 2.12

Dart 2 ⇒ Dart 2.12

More† optimization opportunities!

Demand for smaller downloads size and memory footprint.

Diverging AOT from JIT

DWARF stack traces

Demand for low-overhead concurrency.

Dart Isolates

Dart Isolate Groups (new!)

Dart Isolate Groups (new!)

Dart Isolate Groups (new!)

Dart Isolate Groups (new!)

Dart Isolate Groups (new!)

Dart Isolate Groups (new!)

Dart Isolate Groups (new!)

Learnings

Everything counts

at some point

It is all about the balance

specialized vs generic

specialized solutions

but generic foundations.

`d.bark()` (unoptimized)

`d.bark()` (optimized, monomorphic)

`d.bark()` (optimized, monomorphic)

`d.bark()` (optimized, monomorphic)

More^† optimization opportunities!