Good question — 320 MHz with 4096-QAM is where a PA that looked fine at 80 MHz starts to fall apart, and the failure is rarely a single number.
First, set the EVM budget from the constellation, not from the datasheet headline. 4096-QAM needs roughly -38 dB EVM or better at the antenna port, and I keep at least 2-3 dB of margin for the rest of the chain — filter ripple, matching, and the receiver itself. If the PA alone is already at -36 dB, there is no headroom left, and the system will fail on a bad channel. Measure EVM across the full 320 MHz, and also in 80 MHz sub-bands: the edges usually fail first, because gain and phase flatness roll off there and the matching network was tuned at mid-band.
Second, treat DPD as a wideband problem. A DPD trained on a narrowband signal, or with an envelope bandwidth that does not cover the full 320 MHz plus its spectral regrowth, will look good on a tone and bad on the real waveform. Train and validate with the actual 320 MHz modulated signal at the operating power, and check memory effects separately: run a long burst, watch EVM in the first and last symbols, and look for a slow drift that points to thermal or bias memory rather than static AM-AM/AM-PM.
Third, back-off is a trade, not a fix. A couple of dB of extra back-off often buys 3-5 dB of EVM, but it costs efficiency and heat, and at 6 GHz the thermal drift then moves the EVM again over a long session. I would rather fix the flatness and the DPD first, then choose the smallest back-off that holds EVM with margin over temperature. Also check the supply: a PA pulling fast envelope current through a slow bias network will show EVM that depends on the data pattern, which no amount of static back-off cures.
If you can share the band, the back-off you are running, and the full-band versus sub-band EVM, it will be much easier to say whether this is a flatness problem, a DPD bandwidth problem, or simply not enough back-off.