The city of La Paz, Bolivia is shaped like a basin, with a symmetrical half on the northern part towards El Alto.
Lake Titikaka, the world´s highest navigable lake in the world is less than 90 miles awa, and is 400 meters higher than La Paz.
One theory for it´s landscape is that La Paz was once a lake, fed by the Andes meltwater. Tectonic plate movement million of years ago probably broke up the lake, causing a gradual draining that headed to the southern part of the city. Because of the acidity in the water, the southeast section of the city had an erosion that took place more gradually. This region is now is the site of gigantic stalagmites.
This area is called the Valley of the Souls and is about 10 miles from Sopocachi, the best and safest tourist neighborhood in la Paz.
You can reach the Valley of the Souls using public transportation (minibus that goes to "Uni.") This minibus goes down the street Sanchez Lima, but is usually full. Another way to go is to take public transportation to the market of Cota Cota ($0.35 cents of a dollar) and take a taxi from there ($ 3.50 dollars.) Many buses go to Cota Cota/Chasquipampa. You can take any one of them.
If you are the adventurous kind, you may follow your instinct (or research in the web.) Otherwise, local guides charge around $20 dollars per person for a three hour trek.