The Cave levels are the third and fourth floors of The Dungeon.
The Caves are entirely made up of natural and mossy stone. Stray rocks, mushrooms, and crystals litter the ground. Many chasms and lava pools can also be found in a large amount of the rooms, as well as some uncommon rooms which contain trees and grass.
As of Update #5 on January 18, the caves have "gotten a complete overhaul". The caves are no longer flat, quite the opposite. The floor and the ceiling is made of blocky slopes of varying heights. This makes for much better generation and odd corners and angles. The rooms are now more defined, and many consist of lava pits and chasms which, if you fall in, will most certainly kill you, unless you have a teleport scroll ready. There are some rooms which will spawn will a grass floor and some trees, however these serve aesthetic purposes only.