Autistic adults sometimes get along better with other autistic people compared to non-autistic people, but so far this has only been studied in two-person interactions. This study examined how well autistic and non-autistic people develop rapport in a group setting and whether rapport differs when group members share or do not share a diagnosis. We assigned 143 adults to 36 groups of four adults each. Some groups only had autistic members, some only had non-autistic members, and some were "mixed" groups of autistic and non-autistic members. Groups participated in a tower-building task for 5 minutes and afterwards completed a survey about rapport with the group. The groups of all-autistic participants expressed that their interactions were more enjoyable and friendly than the mixed groups. Autistic participants reported lower rapport when interacting with non-autistic adults, while non-autistic participants reported similar rapport whether interacting with autistic or non-autistic group members. Overall, findings are not consistent with a social deficit model of autism, as autistic adults often established rapport with partners in a group setting. Their level of rapport, however, depended strongly on the social context, particularly whether other autistic people were also in the group.