Task-relevant representational spaces in human memory traces

During encoding, stimuli are embedded into memory traces that allow for their later retrieval. However, we cannot remember every aspect of our experiences. Here, we show that memory traces consist of multidimensional representational spaces whose formats are flexibly strengthened or weakened during encoding and consolidation. In a series of behavioral experiments, participants compared pairs of natural images on either two conceptual or two perceptual dimensions, leading them to incorporate the images into representational ‘spaces’. We found that representations from deep neural networks relate to both behavioral similarity and memory confidence judgements. Furthermore, we found that distances in task-relevant but not irrelevant spaces affected memory strengths. Interestingly, conceptual encoding did not impair subsequent rejection of similar lures, suggesting that task-irrelevant perceptual information remained in the memory trace. However, targeted memory reactivation following conceptual encoding deteriorated perceptual discrimination, indicating that it weakened the accessibility of perceptual formats. Our results demonstrate that representational formats are flexibly incorporated into memory, and more generally show how the organization of information in cognitive representational spaces shapes behavior.