I came to the conclusion that not much can be done to fix the issue. The primary goggles can detect a bad packet and then direct the drone to re-send it. The Audience goggles can't do this.
I forgot where I saw someone claim that reducing the bandwidth between the drone and primary goggles...