Commit cab7a45
committed
[JSC] Inline small sorting in DFG / FTL
https://bugs.webkit.org/show_bug.cgi?id=313668
rdar://175871778
Reviewed by Yijia Huang.
This patch enables fully-inlining Array.prototype.sort function in DFG /
FTL graph, including inlining a comparator call.
1. We fully inline Contiguous / Int32 / Undecided array sorting up to 16
elements. Right now, we do not support Double array.
2. The form is
scratch = ArraySortCompact(array)
... sorting scratch ...
ArraySortCommit(scratch, array)
And we do sorting inline against scratch. ArraySortCompact filters
hole array case etc., and they are going to the slow path call (which
just uses normal sort). ArraySortCommit will apply sorted result back
to array.
3. The most notable interesting part is OSR exit handling. In
Array.prototype.sort, all OSR exit will be handled as "retry entire
sorting again". This is fine because the spec does not define
anything about how the comparator is called. This characteristics is
leveraged in V8 too.
4. When comparator is inlined and OSR exit happens, then comparator must
continue running its code. But after returning from the comparator,
we are returning to array_sort_comparator_return_trampoline. Then it
adjust the next PC and restart `op_call` of Array.prototype.sort again.
Tests: JSTests/stress/array-sort-inline-closure-comparator.js
JSTests/stress/array-sort-inline-constant-comparator.js
JSTests/stress/array-sort-inline-contiguous.js
JSTests/stress/array-sort-inline-full.js
JSTests/stress/array-sort-inline-holey.js
JSTests/stress/array-sort-inline-len2.js
JSTests/stress/array-sort-inline-osr-exit-in-comparator.js
JSTests/stress/array-sort-inline-small.js
JSTests/stress/array-sort-inline-stack-trace.js
* JSTests/stress/array-sort-inline-boolean-comparator.js: Added.
(check):
(sortIt):
(expected):
(i.const.cmp):
(i.const.neg.valueOf):
(i.const.pos.valueOf):
(i.const.zero.valueOf):
(i.catch):
* JSTests/stress/array-sort-inline-closure-comparator.js: Added.
(test):
(descendingTest):
* JSTests/stress/array-sort-inline-comparator-speculation-exit.js: Added.
(cmp):
(sortIt):
(assertSorted):
(throwingCmp):
(catch):
* JSTests/stress/array-sort-inline-constant-comparator.js: Added.
(cmp):
(test):
* JSTests/stress/array-sort-inline-contiguous.js: Added.
(sortIt):
(return.a.sort):
(throw.new.Error):
* JSTests/stress/array-sort-inline-full.js: Added.
(sortAsc):
(runSize):
(bigTest):
(stableTest):
(catch):
* JSTests/stress/array-sort-inline-holey.js: Added.
(cmp):
(sortIt):
(runHoley):
* JSTests/stress/array-sort-inline-large.js: Added.
(cmp):
(cmpDesc):
(doSort):
(makeReversed):
(makeRandomish):
(assertSorted):
(alternating):
(catch):
* JSTests/stress/array-sort-inline-len2.js: Added.
(cmpAsc):
(sortWith):
(countingCmp):
* JSTests/stress/array-sort-inline-mixed-sizes.js: Added.
(cmp):
(makeSmall):
(makeLarge):
(harness):
(vm.useFTLJIT):
(check):
(large.sort):
* JSTests/stress/array-sort-inline-noncallable-typeerror.js: Added.
(expectThrow):
(makeEmpty):
(makeSingle):
(sortIt):
(i.expectThrow.sortIt.makeEmpty):
(i.expectThrow.sortIt.makeSingle):
* JSTests/stress/array-sort-inline-osr-exit-in-comparator-with-spread.js: Added.
(cmp):
(sortAndSpread):
(check):
* JSTests/stress/array-sort-inline-osr-exit-in-comparator.js: Added.
(inlinedComparator):
(test):
(multiset):
(sameMultiset):
* JSTests/stress/array-sort-inline-polymorphic-comparator.js: Added.
(cmpAsc):
(cmpDesc):
(cmpMod3Asc):
(cmpByAbs):
(doSort):
(expectedFor):
(w.try.doSort.seed.slice):
(w.catch):
(cmpThrow):
(catch):
* JSTests/stress/array-sort-inline-small.js: Added.
(cmp):
(callSort):
* JSTests/stress/array-sort-inline-stack-trace.js: Added.
(inlined):
(test):
(catch):
* Source/JavaScriptCore/bytecode/BytecodeList.rb:
* Source/JavaScriptCore/bytecode/InlineCallFrame.cpp:
(WTF::printInternal):
* Source/JavaScriptCore/bytecode/InlineCallFrame.h:
(JSC::InlineCallFrame::callModeFor):
(JSC::InlineCallFrame::specializationKindFor):
* Source/JavaScriptCore/dfg/DFGAbstractInterpreterInlines.h:
(JSC::DFG::AbstractInterpreter<AbstractStateType>::executeEffects):
* Source/JavaScriptCore/dfg/DFGByteCodeParser.cpp:
(JSC::DFG::ByteCodeParser::emitExitOK):
(JSC::DFG::ByteCodeParser::handleCallVariant):
(JSC::DFG::ByteCodeParser::handleIntrinsicCall):
(JSC::DFG::ByteCodeParser::handleArraySort):
* Source/JavaScriptCore/dfg/DFGClobberize.h:
(JSC::DFG::clobberize):
* Source/JavaScriptCore/dfg/DFGDoesGC.cpp:
(JSC::DFG::doesGC):
* Source/JavaScriptCore/dfg/DFGFixupPhase.cpp:
(JSC::DFG::FixupPhase::fixupNode):
* Source/JavaScriptCore/dfg/DFGMayExit.cpp:
* Source/JavaScriptCore/dfg/DFGNode.h:
(JSC::DFG::Node::hasArrayMode):
* Source/JavaScriptCore/dfg/DFGNodeType.h:
* Source/JavaScriptCore/dfg/DFGOSRExitCompilerCommon.cpp:
(JSC::DFG::callerReturnPC):
* Source/JavaScriptCore/dfg/DFGOperations.cpp:
(JSC::DFG::JSC_DEFINE_JIT_OPERATION):
* Source/JavaScriptCore/dfg/DFGOperations.h:
* Source/JavaScriptCore/dfg/DFGPredictionPropagationPhase.cpp:
* Source/JavaScriptCore/dfg/DFGSafeToExecute.h:
(JSC::DFG::safeToExecute):
* Source/JavaScriptCore/dfg/DFGSpeculativeJIT.cpp:
* Source/JavaScriptCore/dfg/DFGSpeculativeJIT.h:
* Source/JavaScriptCore/dfg/DFGSpeculativeJIT32_64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* Source/JavaScriptCore/dfg/DFGSpeculativeJIT64.cpp:
(JSC::DFG::SpeculativeJIT::compile):
* Source/JavaScriptCore/dfg/DFGStoreBarrierInsertionPhase.cpp:
* Source/JavaScriptCore/ftl/FTLCapabilities.cpp:
(JSC::FTL::canCompile):
* Source/JavaScriptCore/ftl/FTLLowerDFGToB3.cpp:
(JSC::FTL::DFG::LowerDFGToB3::compileNode):
(JSC::FTL::DFG::LowerDFGToB3::compileGetCellButterflySlot):
(JSC::FTL::DFG::LowerDFGToB3::compilePutCellButterflySlot):
(JSC::FTL::DFG::LowerDFGToB3::compileArraySortCompact):
(JSC::FTL::DFG::LowerDFGToB3::compileArraySortCommit):
* Source/JavaScriptCore/llint/LLIntSlowPaths.cpp:
(JSC::LLInt::dispatchToCurrentInstructionDuringExit):
(JSC::LLInt::llint_slow_path_array_sort_comparator_return):
* Source/JavaScriptCore/llint/LLIntSlowPaths.h:
* Source/JavaScriptCore/llint/LLIntThunks.cpp:
(JSC::LLInt::arraySortComparatorReturnTrampolineThunk):
* Source/JavaScriptCore/llint/LLIntThunks.h:
* Source/JavaScriptCore/llint/LowLevelInterpreter.asm:
* Source/JavaScriptCore/runtime/ArrayPrototype.cpp:
(JSC::ArrayPrototype::finishCreation):
* Source/JavaScriptCore/runtime/Intrinsic.h:
* Source/JavaScriptCore/runtime/VM.cpp:
(JSC::VM::VM):
(JSC::VM::visitAggregateImpl):
* Source/JavaScriptCore/runtime/VM.h:
Canonical link: https://commits.webkit.org/312983@main1 parent 4c86535 commit cab7a45
48 files changed
Lines changed: 2050 additions & 11 deletions
File tree
- JSTests/stress
- Source/JavaScriptCore
- bytecode
- dfg
- ftl
- llint
- runtime
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
Lines changed: 89 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
0 commit comments