@mitchellh this situation can’t be avoided in office either can it? 
3rd person can’t be there for all conversations, team culture should encourage adding the third person in their discussion and getting their input. it’s not that hard to reach people.