As the depth of oil and gas exploration increases, downhole electronics face the threat of high-temperature failure. At present, passive cooling technology has the problem of short working time, while active cooling technology has low energy utilization. This paper presents a thermal management system of vapor compression with a combination of active and passive cooling. The system uses insulation materials to isolate the high-temperature environment, thermally conductive silicone grease to strengthen the heat exchange in the evaporator, and vapor compression refrigeration cycles to absorb internal heat. The coefficient of performance (COP), exergy destruction and exergy efficiency of octane, nonane, and cyclohexane as refrigerants were examined, and the effects of different insulation materials on refrigeration performance were studied from both theoretical and numerical perspectives. The results showed that cyclohexane exhibited the best cooling capacity with a COP of 1.296 and a exergy efficiency of 49.21%. The thermal management system cooling performance is optimal when the insulation material is a vacuum flask, with an effective cooling capacity of 121.7 W.