Your RT-AC68U nodes are only dual-band (as you probably know). Since you are using wired backhaul, then I would suggest making the RT-AC5300 5GHz-2 channel available as a second 5GHz channel, as it is not needed for the backhaul. Disabling the 5GHz-2 channel on the RT-AC5300 will not increase the power of the 5GHz-1, but it could avoid channel conflicts/overlap.
To cover a larger area, you could add more nodes. I don't have any experience with adding larger antennae. If you are trying to increase coverage outside on your property, then you may need to look at dedicated outside access points.