A hallmark of human cognition is the flexibility to plan with others across novel situations in the presence of uncertainty. We act together with partners of variable sophistication and knowledge and against adversaries who are themselves both heterogeneous and flexible. While a team of agents may be united by common goals, there are often multiple ways for the group to actually achieve those goals. In the absence of centralized planning or perception and constrained or costly communication, teams of agents must efficiently coordinate their plans with respect to the underlying differences across agents. Different agents may have different skills, competencies or access to knowledge. When environments and goals are changing, this coordination has elements of being ad-hoc. Miscoordination can lead to unsafe interactions and cause injury and property damage and so ad-hoc teamwork between humans and agents must be not only efficient but robust. We will both investigate human ad-hoc and dynamic collaboration and build formal computational models that reverse-engineer these capacities. These models are a key step towards building machines that can collaborate like people and with people.