It's really tricky eh, I can't keep up with all the shows I want to listen to. Rogan is great at times (some of his episodes are pseudoscience BS) as he's really a master at the long-form interview, kind of like Stern. For me part of it is just being picky about which episodes I listen to based on the guest.
I also listen at about 1.25x speed on an app call Overcast which also splices out blank spots it detects in speech, which helps a bit.