Seeds have evolved in association with diverse microbial assemblages that may influence plant growth and health. However, little is known about the composition of seed-associated microbial assemblages and the ecological processes shaping their structures. In this work, we monitored the relative influence of the host genotypes and terroir on the structure of the seed microbiota through metabarcoding analysis of different microbial assemblages associated to five different bean cultivars harvested in two distinct farms. Overall, few bacterial and fungal operational taxonomic units (OTUs) were conserved across all seed samples. The lack of shared OTUs between samples is explained by a significant effect of the farm site on the structure of microbial assemblage, which explained 12.2% and 39.7% of variance in bacterial and fungal diversity across samples. This site-specific effect is reflected by the significant enrichment of 70 OTUs in Brittany and 88 OTUs in Luxembourg that lead to differences in co-occurrence patterns. In contrast, variance in microbial assemblage structure was not explained by host genotype. Altogether, these results suggest that seed-associated microbial assemblage is determined by niche-based processes and that the terroir is a key driver of these selective forces.