Skip to content

[Arm64] Implement FMA, FMS, MLA, MLS - #31899

Merged
echesakov merged 17 commits into
dotnet:masterfrom
echesakov:Arm64-Fused-Or-Not-Multiply-Add-Subtract
Feb 11, 2020
Merged

echesakov merged 17 commits into
dotnet:masterfrom
echesakov:Arm64-Fused-Or-Not-Multiply-Add-Subtract

Conversation

@echesakov

@echesakov echesakov commented Feb 7, 2020

Copy link
Copy Markdown
Contributor

This implements the fused multiply-add and multiply-subtract intrincsics and multiply-add multiply-subtract for integer types.

Note that this PR doesn't have an implementation for FMLA <Vd>.<T>, <Vn>.<T>, <Vm>.<Ts>[<index>] - this will be done in a separate PR.

Part of #24794

@echesakov echesakov added arch-arm64 area-System.Runtime.Intrinsics area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI labels Feb 7, 2020
case NI_AdvSimd_Arm64_FusedMultiplyAdd:
case NI_AdvSimd_Arm64_FusedMultiplySubtract:
case NI_AdvSimd_MultiplyAdd:
case NI_AdvSimd_MultiplySubtract:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be worth checking the x86 handling here. Given it is three operands and doing a * b + c there was special handling added such that a and b could still be considered commutative (etc).

x86 also has a (I believe in lowering) optimization for converting things like FusedMultiplyAdd(a, b, -c) to a FusedMultiplySubtract(a, b, c)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tannergooding

It might be worth checking the x86 handling here. Given it is three operands and doing a * b + c there was special handling added such that a and b could still be considered commutative (etc).

Yes, there is a special handling on x86 but, as far as I understand, it takes into account whether any of a, b or c operands is a memory operand and whether the upper bits of operands are preserved.

On arm64 there is only one form for each of the instructions (e.g. FMLA <Vc>, <Va>, <Vb>) so I am not sure what else we can do here?

x86 also has a (I believe in lowering) optimization for converting things like FusedMultiplyAdd(a, b, -c) to a FusedMultiplySubtract(a, b, c)

Yes, @EgorBo added this optimization in dotnet/coreclr#27060. I was planning working on these later but not as part of this PR.

@tannergooding tannergooding Feb 7, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of the x86 handling is because you can have op1 * op3 + op2 or op2 * op1 + op3 or op2 * op3 + op1. That handling is x86 specific.

There is also handling for allowing a * b + c or b * a + c since the multiplication is commutative. That handling also applies to ARM.

I was planning working on these later but not as part of this PR.

Sounds good to me, as long as it is being tracked.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I think I see.

Since it takes 4 registers (a destination and three inputs) there is no need for it to be commutative. I had misread and thought it was 3 inputs where one was also the destination.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The vector version (FMLA) looks interesting however, since it does only take 3 registers where the accumulator is also the destination

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What instructions are we talking about here?

  • FMADD, FNMADD, FMSUB, FNMSUB take four registers - destination and three source registers - these instructions are emitted during FusedMultiplyAddScalar, FusedMultiplyAddNegatedScalar, FusedMultiplySubtractScalar, FusedMultiplySubtractNegatedScalar. LSRA does not have any special logic for those.

  • On other hand, FMLA, FMLS, MLA, MLS take only three - destination and incoming accumulator operand register and two multipliers' registers. Those are used for vector forms of FusedMultiplyAdd, FusedMultiplySubract, MultiplyAdd, MultiplySubtract. The logic in lsraarm64.cpp applies only to these instructions/intrinsics. My question how we can use commutativity of operands in this particular case?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had been looking at FMLA initially as it only takes three and I missed that FMADD took four.
I had also thought that the src/dest operand was one of the multipliers and so commutativity mattered.

@@ -210,6 +210,18 @@ public abstract partial class AdvSimd : System.Runtime.Intrinsics.Arm.ArmBase
public static System.Runtime.Intrinsics.Vector64<uint> CompareTest(System.Runtime.Intrinsics.Vector64<uint> left, System.Runtime.Intrinsics.Vector64<uint> right) { throw null; }
public static System.Runtime.Intrinsics.Vector64<double> DivideScalar(System.Runtime.Intrinsics.Vector64<double> left, System.Runtime.Intrinsics.Vector64<double> right) { throw null; }
public static System.Runtime.Intrinsics.Vector64<float> DivideScalar(System.Runtime.Intrinsics.Vector64<float> left, System.Runtime.Intrinsics.Vector64<float> right) { throw null; }
public static System.Runtime.Intrinsics.Vector128<float> FusedMultiplyAdd(System.Runtime.Intrinsics.Vector128<float> acc, System.Runtime.Intrinsics.Vector128<float> left, System.Runtime.Intrinsics.Vector128<float> right) { throw null; }

@tannergooding tannergooding Feb 7, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may want to discuss the ordering in API review.

The scalar versions are FMADD <Sd>, <Sn>, <Sm>, <Sa>, where it does Sd = Sa + Sn * Sm (registers ordered dest, left, right, acc)

The vector versions are FMLA <Vd>, <Vn>, <Vm> where it does Vd = Vd + Vn * Vm (registers ordered dest/acc, left, right as you have here)

x86 intrinsics and the public Math.FusedMultiplyAdd methods are all dest, left, right, acc

@CarolEidt CarolEidt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@echesakov
echesakov merged commit 7b58790 into dotnet:master Feb 11, 2020
@echesakov
echesakov deleted the Arm64-Fused-Or-Not-Multiply-Add-Subtract branch February 11, 2020 00:23
@ghost ghost locked as resolved and limited conversation to collaborators Dec 10, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

arch-arm64 area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI area-System.Runtime.Intrinsics new-api-needs-documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants