I have lost more simulation time to wrong port setups than to mesh settings, so here is the checklist I now run before trusting any S-parameters.
First, match the port to the physics. A wave port assumes a guided mode travelling into a uniform cross-section, which fits a connector launch or a transmission line feed. A lumped port forces a voltage across a small gap, which fits a component pad or an on-board feed where no clean waveguide mode exists. Swapping them gives plausible-looking curves that are quietly wrong.
Second, size the wave port properly. The port face must be large enough to contain the mode and its fringing fields, usually several times the trace width and substrate height, and it must touch the ground reference it actually references. A port that is too small truncates the field and shifts the impedance.
Third, check the port impedance the solver reports before looking at S11. If the port impedance is far from the intended line impedance, the mismatch is in the setup, not in the design, and tuning the layout will only chase the error.
Fourth, keep the excitation away from discontinuities. Put some line length between the port and the first bend, via, or component, so evanescent fields from the port do not mix with the discontinuity you are trying to measure.
Fifth, validate with a trivial case. Simulate a straight through line first: if its insertion loss and phase do not look sane, do not move on to the real structure.
What is your own red flag that a port setup is wrong before you even look at the results?