This is probably going to be a "yeah duh u dummy" comment, but in case anyone doesn't know:
When setting up a home network, split out your wifi bands if you can (e.g. ssidname, ssidname-24) so that the 5GHz and 2.4GHz networks are separate.
Setup all of your non-high speed/high bandwidth devices on the 2.4GHz -- for things like HomePods you have to be connected to the 2.4GHz network on the device setting it up to do this.
Put everything else high speed on the 5GHz network.
I dramatically underestimated the effect of noise on speed/latency. I have 3 HomePod Minis, 1 HomePod, 2 wifi smart lightbulbs, 2 Dyson air purifiers, and 2 smartplugs in a large house with 2 Asus GT-6 wifi mesh routers. Devices were fairly evenly split between them and those IoT devices basically never do much data transfer, but just their connection alone degraded 5GHz band performance significantly.
With all of those low-speed devices on 5GHz, I was getting 400-450Mbps down and 60ms+ latency in speed tests while sitting in front of the router.
With all of those low-speed devices on 2.4Gz, I was getting 650-700Mbps down and 20ms- latency.