Efficient dry beneficiation of low-grade oil shale requires precise regulation of bed density in high-density gas–solid fluidized beds. This study develops a ternary dense-medium system comprising ferrosilicon powder, magnetite powder and oil shale particles, and investigates the coupling between medium composition, hydrodynamics and machine-learning-assisted density prediction. The results demonstrated that the ternary density regulation strategy significantly enhances fluidization uniformity and separation efficiency in the dry dense-medium fluidized bed. When the oil-shale mass fraction increases from 0 % to 20 %, the critical fluidization velocity rises from 12.54 to 14.08 cm/s, while the bed expansion ratio grows from 5.19 % to 8.83 %. Compared with the conventional binary medium, the ternary system lowers the mean bed density from 2.567 to 2.382 g cm⁻³ and achieves the minimum density fluctuation (standard deviation, SD = 0.097) at an optimal oil-shale mass fraction of 8 %. A back-propagation neural network optimized by a genetic algorithm (GA-BP) using seven process features predicts bed density with correlation coefficient R = 0.979 and root-mean-square error (RMSE) of 0.049 on 167 test samples—an 18 % error reduction over the conventional BP model. The proposed ternary medium strategy and GA-BP predictor therefore offer a robust framework for stable, energy-efficient dry separation of oil shale.