Brain activity during reciprocal social interaction investigated using conversational robots as control condition

Description: We present a novel paradigm for social neuroscience comparing a human social interaction (human-human interaction, HHI) to an interaction with a conversational robot (human-robot interaction, HRI) during functional magnetic resonance imaging (fMRI). We recorded 1-minute blocks of live bidirectional discussion between a participant in scanner and another human (confederate) or a robot agent outside the scanner. A cover story provides the topic of the discussion while hiding to participants the true objectives of the experiment. To this end, we collected multimodal data including fMRI data, behaviour (speech from the participant and human or robot agent, video capture of the human and robot agent, and the gaze movement of the scanned participant) and physiology (BOLD signal, respiration and peripheral blood flow pulse) to form a corpus. Experimental paradigm The MRI recordings consisted of four sessions of each six 1-minute blocks of conversation each, showing anthropomorphized fruits and vegetables as “super-heroes” images in the first and third sessions and images of anthropomorphized “rotten fruits” images in the second and fourth sessions. The order was kept constant across participants each session alternating the three images per session and two interacting agents and starting with the human agent (ie Image1/Human, Image2/Robot, Image3/Human, Image1/Robot, Image2/Human, Image3/Robot). Each image was thus shown twice in each session, once per interacting agent. Blocks started with the presentation of one image for 8.3 seconds, followed by a 3.3 second black screen, after which there was a live bidirectional conversation with the interacting agent for one minute, followed by an inter block interval black screen of 4.6 seconds. In the absence of live video feed from inside the scanner, a light signaled to the confederate that the conversation had started. The participant initiated the conversation, instructed to talk freely with the other agent about the image and their suggestions on the topic of the advertisement campaign. One block lasted 76.2 seconds and one session 8 minutes and 2 seconds of fMRI recording. We recorded 3 minutes of conversation per interacting agent and session, for a total of 24 minutes of conversation per participant. Audio and video set-up of the conversation was tested beforehand, and audio adjusted individually for each participant. As participants were always connected via audio with the confederate, they mentioned if they couldn’t hear well, giving us the chance to adapt the audio if required. This information was recorded for future use. MRI acquisition MRI data was collected with a 3T Siemens Prisma (Siemens Medical, Erlangen, Germany) using a 20-channel head coil. Blood oxygen level-dependent (BOLD) sensitive functional images were acquired using an EPI sequence in the 4 runs. Parameters were as follows: Echo time (TE) 30 ms, repetition time (TR) 1205 ms, flip angle 65°, 54 axial slices co-planar to the anterior / posterior commissure plane, FOV 210mm x 210mm, matrix seize 84 x 84, voxel size 2.5 x 2.5 x 2.5 mm3, with multiband acquisition factor 3. After functional scanning, structural images were acquired with a GR_IR sequence (TE/TR 0.00228/2.4 ms, 320 sagittal slices, voxel size 0.8 x 0.8 x 0.8 mm, field of view 204,8 x 256 x 256mm). MRI data analysis MRI data was analysed using SPM12 (Statistical Parametric Mapping, http://www.fil.ion.ucl.ac.uk/spm/). First, we calculated the voxel displacement map. The time series for each voxel was then realigned temporally to the acquisition of the slice in the middle in time to correct for differences in slice time acquisition. The image time series were unwarped using the voxel-displacement map to take into account local distortion of the magnetic field and spatially realigned using a sinc interpolation algorithm that estimates rigid body transformations (translations, rotations). Images were then spatially smoothed using an isotropic 5 mm full-width-at-half-maximum Gaussian kernel. The first realigned and unwarped functional image was coregistered with an unwarped single-band-reference image recorded at the onset of each trial, which was itself coregistered with the T1 and T2 anatomical images. These anatomical images were segmented into grey matter (GM), white matter (WM), and cerebral spinal fluid (CSF) using SPM12 “New segment”. GM, WM, and CSF tissue probability maps were used to form a DARTEL template (Ashburner, 2007). The deformation flow fields from individual spaces to this template were used to normalize the beta images resulting from the individual subjects’ analyses (i.e. in subjects’ individual space) for use in a random-effect second-level analysis. Potential artefacts from blood pulse and respiration were controlled using the Translational Algorithms for Psychiatry-Advancing Science (TAPAS) toolbox standard procedure (https://www.tnu.ethz.ch/de/software/tapas/documentations/physio-toolbox.html; Kasper et al., 2017). Realignment parameters (translation and rotation) as well as their derivatives and the square product of both parameters and their derivatives were used as covariates to control for movement-related artefacts. We also used the Artefact Detection Tools (ART) to control for any movement-related artefacts (www.nitrc.org/projects/artifact_detect/) using the standard threshold of 2 mm. The fMRI time series were analysed using the General Linear Model (GLM) approach implemented in SPM. Single-subject models consisted of one regressor representing the one-minute discussion for each of the two interacting agents, and another one representing the presentation of the images. After normalization, beta estimates images were entered in a mixed-model analysis of variance (using SPM “full ANOVA”) with participants and sessions as random factors and the nature of the interacting agent as factor of interest for inferences at the population level. A mask was created on the basis of the mean of DARTEL normalized anatomical GM and WM tissue classes of each participant, also used for rendering results in Figure 3. We first assessed the main effect of the conversation with both agents against the implicit baseline. We then looked specifically at the effects of each of the interacting agent contrasted to the other one, with a clear focus on brain areas involved in mentalizing and social motivation in the contrast HHI versus HRI. All statistical inference was performed applying a threshold of p = 0.05 False-Discovery Rate (FDR) corrected for the whole brain at the cluster-level (Friston, Holmes, Poline, Price, & Frith, 1996). Anatomical localization of the resulting clusters relied on the projection of the results onto the mean anatomical image of our pool participants resulting from DARTEL coregistration. Description partly taken from Rauchbauer, B. et (2016; pp 8 - 10), under revision

Related article: http://doi.org/10.1098/rstb.2018.0033