Understanding the mechanisms generating species distributions remains a challenge, especially in hyperdiverse tropical forests. We evaluated the role of rainfall variation, soil gradients and herbivory on seedling mortality, and how variation in seedling performance along these gradients contributes to habitat specialisation. In a 4-year experiment, replicated at the two extremes of the Amazon basin, we reciprocally transplanted 4638 tree seedlings of 41 habitat-specialist species from seven phylogenetic lineages among the three most important forest habitats of lowland Amazonia. Rainfall variation, flooding and soil gradients strongly influenced seedling mortality, whereas herbivory had negligible impact. Seedling mortality varied strongly among habitats, consistent with predictions for habitat specialists in most lineages. This suggests that seedling performance is a primary determinant of the habitat associations of adult trees across Amazonia. It further suggests that tree diversity, currently mostly harboured in terra firme forests, may be strongly impacted by the predicted climate changes in Amazonia.