If I'm flying from one Terminal and the lounge is in another, can I still access it?

This will depend on the security restrictions at the airport. Your Priority Pass won't allow you to pass through airport security to access a lounge.