On-board cryogenic storage of liquid hydrogen (LH) and liquid natural gas (LNG) plays an important role in the alternative gaseous fuel to reduce greenhouse gaseous emissions in transportation. Metallic pressure vessels are the common choice for liquid gases storage; however metallic materials are usually denser and corrosion-prone. Non-metallic pressure vessels have a better prospect of liquid gases storage due to some unique advantages, e.g., light weight, good corrosion resistance, and low thermal conductivity. This paper made a summarization about non-metallic pressure vessels for cryogenic application for further reference. Several cryogenic pressure vessels were introduced, including the application scenario and service conditions. The failure of non-metallic cryogenic pressure vessels was reviewed, and the corresponding causes and efforts for preventing the failure were introduced.