• A transition of segregation pattern in S–D system was achieved in circular drum.
• Effect of segregation mechanisms and particle diffusion in system were discussed.
• Buoyancy mechanism and particle diffusion mainly effected pattern transition.
• A dimensionless parameter was proposed to predict the segregation pattern.
Complex segregation occurs in a binary particle system with differing particle sizes and densities, particularly when the larger particles are heavier (S–D system, i.e., size minus density system). Predicting the segregation pattern driven by multiple mechanisms simultaneously is often challenging. This study explores the segregation mechanisms in a quasi-2D circular drum containing a S–D system, realizing a transition between the S-core and Core-and-band patterns by adjusting the drum rotation speed. During the transition of the segregation pattern, only the S-core pattern chiefly driven by the percolation mechanism is initially observed. As the rotation speed increases, the buoyancy mechanism and particle diffusion gradually strengthen, jointly driving the formation of the Core-and-band pattern. A dimensionless strength ratio, λ = H/h, where H and h respectively represent the diffusion and buoyancy strengths at length scales, is introduced to elucidate this transition. The Core-and-band pattern emerges when λ reached 1.4.