The species–environment relationships for woody species may vary according to the forest layers considered. In fragmented forest, spatial configuration may also influence forest layer composition. We investigated the relationships between four forest layer compositions and environmental conditions, and spatial variables accounting for forest fragmentation, in 59 forest stands. Field and shrub layer compositions were mainly linked to environmental conditions, particularly to soil pH and slope aspect, while the upper layer compositions were principally correlated to the spatial configuration. The distance from the forest edge was correlated with all the forest layer compositions. Our results suggest that woody species respond to factors acting at different spatial and temporal scales, depending on the forest layer they belong to. The species–environment relationship seems to weaken from the lower to upper layer, the upper layer being more closely linked to the spatial configuration and probably to the past management. This study underlines the importance of taking spatial configuration in addition to environmental conditions into account when studying woody plant diversity for different forest layers in stands located in deciduous fragmented forests. Moreover, stand history seems to have a lasting effect on woody plant composition, particularly for the tree layer.