Skip to content

Fix HICKS_HENNE_CAMBER deformation under MPI - #2897

Open
pragyaangaur wants to merge 2 commits into
su2code:developfrom
pragyaangaur:fix-hh-camber-mpi
Open

pragyaangaur wants to merge 2 commits into
su2code:developfrom
pragyaangaur:fix-hh-camber-mpi

Conversation

@pragyaangaur

Copy link
Copy Markdown

Proposed Changes

This PR fixes HICKS_HENNE_CAMBER producing wrong deformations when SU2_DEF runs on more than one MPI rank. The fix is in CSurfaceMovement::SetHicksHenneCamber in Common/src/grid_movement/CSurfaceMovement.cpp.

Before it places the camber bumps, the function finds the trailing edge and the leading edge of the airfoil and computes the angle of attack from them. It searched only the vertices of the current rank and had no MPI communication. A rank whose partition did not contain the true trailing edge or leading edge computed its own wrong frame, so different parts of the airfoil were deformed in different frames. SetHicksHenne in the same file already gathers both points over all ranks with Allgather, and this PR follows the same approach.

The trailing-edge search also had two defects that made the result depend on the vertex order, and the vertex order changes with the partitioning:

  • It read the normal of vertex 0 for every vertex. It now uses the normal of the current vertex to decide if the vertex is on the upper surface, on the lower surface, or on a blunt trailing-edge face.
  • It compared each x against TPCoord[0], which was still zero at that point, so it kept the last matching vertex. It now keeps the vertex with the largest x on each surface.

The changes are:

  • Each rank finds its upper and lower trailing-edge candidates, the candidates of all ranks are gathered, and every rank keeps the one with the largest x.
  • The leading-edge candidate of each rank (the vertex farthest from the trailing edge) is gathered together with its distance, and every rank keeps the farthest one.
  • Only vertices owned by the rank are candidates. Halo vertices belong to another rank, and their normal can be incomplete.
  • Candidates are compared lexicographically, by x and then y for the trailing edge and by distance, x and then y for the leading edge. The reduction starts again from a sentinel after the gather. Every rank therefore processes the same data in the same order, and ties are broken in the same way for any partitioning.
  • A rank that holds no vertex of the design marker no longer reads vertex[iMarker][0]. It sends a sentinel that never wins the reduction.
  • The displacement of each vertex is reset before it is computed. Before this change, a vertex on a marker without design variables received the displacement of the previous vertex.

Related Work

Closes #2875.

Verification

I ran SU2_DEF on TestCases/deformation/naca0012/def_NACA0012_camber.cfg (mesh from su2code/TestCases) with develop and with this branch. The last design variable is a camber bump of 0.005 near the trailing edge, so a correct result moves the upper and lower surfaces by the same amount and keeps the thickness. The columns below are the largest vertical displacement of each surface, the largest change in thickness, and the largest difference of any mesh node to the serial run.

Develop:

Ranks Upper surface Lower surface Thickness change Difference to serial
1 4.973e-03 4.973e-03 1.4e-17 0
2 2.841e-03 2.841e-03 1.8e-05 1.9e-02
4 1.330e-03 6.5e-10 1.3e-03 5.0e-03

With 4 ranks the lower surface does not move at all, which is the behaviour described in the issue.

This branch:

Ranks Upper surface Lower surface Thickness change Difference to serial
1 4.973e-03 4.973e-03 1.4e-17 0
2 4.973e-03 4.973e-03 1.4e-17 1.5e-13
3 4.973e-03 4.973e-03 1.4e-17 1.3e-13
4 4.973e-03 4.973e-03 1.4e-17 3.2e-13
8 4.973e-03 4.973e-03 1.4e-17 3.4e-13

The serial output mesh of this branch is byte-identical to the serial output mesh of develop.

The reference value of the naca0012_def_camber parallel regression test changes from 0.00854844 to 0.0201692. This test runs with mpirun -n 2, so the old value came from the defect. The new value is the deformation residual of the 2-rank run, and it equals the serial value. I computed it on macOS arm64, where the 2-rank run of develop gives 0.00855292 against the stored 0.00854844. The CI machine may therefore differ in the last digits, and I will update the value from the CI log if the test reports a small difference.

pre-commit passes on the changed files. Compiling CSurfaceMovement.cpp with -Wall -Wextra gives no warnings in the changed function, both in the normal build and with -DCODI_REVERSE_TYPE.

PR Checklist

  • I am submitting my contribution to the develop branch.
  • My contribution generates no new compiler warnings (try with --warnlevel=3 when using meson).
  • My contribution is commented and consistent with SU2 style (https://su2code.github.io/docs_v7/Style-Guide/).
  • I used the pre-commit hook to prevent dirty commits and used pre-commit run --all to format old commits.
  • I have added a test case that demonstrates my contribution, if necessary. (The existing naca0012_def_camber parallel test covers this, and its reference value is updated.)
  • I have updated appropriate documentation (Tutorials, Docs Page, config_template.cpp), if necessary. (Not needed, there is no user-facing change.)

SetHicksHenneCamber found the trailing edge, the leading edge and the angle of attack from the vertices of the current rank only, so each rank could deform its part of the airfoil in a different frame. The trailing-edge and leading-edge candidates are now gathered over all ranks and reduced in the same order on every rank. The trailing-edge search now uses the normal of each vertex and keeps the vertex with the largest x on each surface, and only owned vertices are candidates.

The reference value of the naca0012_def_camber parallel test is updated. That test runs on 2 ranks, and the old value came from this defect.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant