but each time we went into a walled courtyard , and the castle was edged with a maze of those empty little useless pockets that contained nothing more than a bench or fire pit , we could hear voices from the outer grounds .