title: Spark - process single file at a time from directory
body: <p>Im in a situation with using Spark where I have a few hundred files that uncompressed are about a few gigs each, and I don't think I could fit that on a single machine, and having enough servers could be cost prohibitive. Im wondering if its possible to get Spark to instead of trying to load all the files and then process them, to load a single file, process it, and then move onto the next. </p>