Gadolinium based contrast agents (GBCAs) are widely used in clinical MRI since the mid-1980s. Recently, concerns have been raised that trace amounts of Gadolinium (Gd), detected in brains even long time after GBCA application, may cause yet unrecognized clinical consequences. We therefore assessed the behavioral phenotype, neuro-histopathology, and Gd localization after repeated administration of linear (gadodiamide) or macrocyclic (gadobutrol) GBCA in rats. While most behavioral tests revealed no difference between treatment groups, we observed a transient and reversible decrease of the startle reflex after gadodiamide application. Residual Gd in the lateral cerebellar nucleus was neither associated with a general gene expression pathway deregulation nor with neuronal cell loss, but in gadodiamide-treated rats Gd was associated with the perineuronal net protein aggrecan and segregated to high molecular weight fractions. Our behavioral finding together with Gd distribution and speciation support a substance class difference for Gd presence in the brain after GBCA application.