To enhance the system performance of the modular multilevel converter (MMC) under unbalanced grid conditions, this paper proposes a fractional order integral sliding mode control (FO-I-SMC) strategy. Firstly, the topology of the MMC is analyzed, and the fundamental frequency external characteristic equations of positive- and negative-sequence voltages and output currents, as well as the double frequency internal characteristic equations of positive-, negative-, and zero-sequence circulating currents, are derived. Secondly, combining the control objectives with the mathematical model of the MMC, a fractional order sliding mode controller is designed for application in unbalanced grid voltages. This controller aims to reduce the harmonic content in the AC-side output current and the DC-side circulating current. Finally, a corresponding model is established on the Matlab/Simulink simulation platform to verify the effectiveness of the proposed algorithm. The research results demonstrate that the system performance of the MMC using the FO-I-SMC strategy is significantly better than that using the proportional integral (PI) control strategy and the integral sliding mode control (ISMC) strategy.