Recently I test-drove a 2021 Prius Prime LE and a 2021 Prius Prime Limited. On the limited, the larger screen seemed to drop down further and my knee bumped the unit, but there was no knee-bumping on the LE. This might have been just perception, though. Are there specifications on the physical casing size of the two units? Or has someone with both measured knee room differences?
I haven't done any actual measurements but based on the photos, there is a difference in height of the MFD between the 7" and the 11.6". I noticed it in the official Toyota photos when the shifter on the 11.6" screen appeared to be lower than that of the 7".
The HVAC center vents on the XLS/Limited are on the side while the LE is on top of the HU. The difference in duct work and HU size could be the reason why you knee bumped the center stack vs the LE. You may have more options later down the road to swap out the LE HU than the XLE and Limited. The XLE and Limited HU is not Android Auto compatible.