title: How to organize multiple python files into a single module without it behaving like a package?
body: <p>Is there a way to use <code>__init__.py</code> to organize multiple files into a <strong>module</strong>?</p>
  
  <p>Reason:  Modules are easier to use than packages, because they don't have as many layers of namespace.</p>
  
  <p>Normally it makes a package, this I get.  Problem is with a package, 'import thepackage' gives me an empty namespace.  Users must then either use "from thepackage import *" (frowned upon) or know exactly what is contained and manually pull it out into a usable namespace.</p>
  
  <p>What I want to have is the user do 'import thepackage' and have nice clean namespaces that look like this, exposing functions and classes relevant to the project for use.</p>
  
  <pre><code>current_module
  \
    doit_tools/
    \
     - (class) _hidden_resource_pool
     - (class) JobInfo
     - (class) CachedLookup
     - (class) ThreadedWorker
     - (Fn) util_a
     - (Fn) util_b
     - (Fn) gather_stuff
     - (Fn) analyze_stuff
  </code></pre>
  
  <p>The maintainer's job would be to avoid defining the same name in different files, which should be easy when the project is small like mine is.</p>
  
  <p>It would also be nice if people can do <code>from doit_stuff import JobInfo</code> and have it retrieve the class, rather than a module containing the class.</p>
  
  <p>This is easy if all my code is in one gigantic file, but I like to organize when things start getting big. What I have on disk looks sort of like this:</p>
  
  <pre><code>place_in_my_python_path/
    doit_tools/
      __init__.py
      JobInfo.py
        - class JobInfo:
      NetworkAccessors.py
        - class _hidden_resource_pool:
        - class CachedLookup:
        - class ThreadedWorker:
      utility_functions.py
        - def util_a()
        - def util_b()
      data_functions.py
        - def gather_stuff()
        - def analyze_stuff()
  </code></pre>
  
  <p>I only separate them so my files aren't huge and unnavigable.  They are all related, though someone (possible me) may want to use the classes by themselves without importing everything.</p>
  
  <p>I've read a number of suggestions in various threads, here's what happens for each suggestion I can find for how to do this:</p>
  
  <p>If I <b>do not use an <code>__init__.py</code></b>, I cannot import anything because Python doesn't descend into the folder from sys.path.</p>
  
  <p>If I <b>use a blank <code>__init__.py</code></b>, when I <code>import doit_tools</code> it's an empty namespace with nothing in it. None of my files imported, which makes it more difficult to use.</p>
  
  <p>If I <b>list the submodules in <code>__all__</code></b>, I can use the (frowned upon?) <code>from thing import *</code> syntax, but all of my classes are behind unnecessary namespace barriers again. The user has to (1) know they should use <code>from x import *</code> instead of <code>import x</code>, (2) manually reshuffle classes until they can reasonably obey line width style constraints.</p>
  
  <p>If I <b>add <code>from thatfile import X</code> statements to <code>__init__.py</code></b>, I get closer but I have namespace conflicts (?) and extra namespaces for things I didn't want to be in there.  In the below example, you'll see that:</p>
  
  <ol>
  <li>The class JobInfo overwrote the module object named JobInfo because their names were the same. Somehow Python can figure this out, because JobInfo is of type <code>&lt;class 'doit_tools.JobInfo.JobInfo'&gt;</code>.  (doit_tools.JobInfo is a class, but doit_tools.JobInfo.JobInfo is that same class...  this is tangled and seems very bad, but doesn't seem to break anything.)</li>
  <li>Each filename made its way into the doit_tools namespace, which makes it more confusing to look through if anyone is looking at the contents of the module.  I want doit_tools.utility_functions.py to hold some code, not define a new namespace.</li>
  </ol>
  
  <p>.</p>
  
  <pre><code>current_module
  \
    doit_tools/
    \
     - (module) JobInfo
        \
         - (class) JobInfo
     - (class) JobInfo
     - (module) NetworkAccessors
        \
         - (class) CachedLookup
         - (class) ThreadedWorker
     - (class) CachedLookup
     - (class) ThreadedWorker
     - (module) utility_functions
        \
         - (Fn) util_a
         - (Fn) util_b
     - (Fn) util_a
     - (Fn) util_b
     - (module) data_functions
        \
         - (Fn) gather_stuff
         - (Fn) analyze_stuff
     - (Fn) gather_stuff
     - (Fn) analyze_stuff
  </code></pre>
  
  <p>Also someone importing just the data abstraction class would get something different than they expect when they do 'from doit_tools import JobInfo':</p>
  
  <pre><code>current_namespace
  \
   JobInfo (module)
    \
     -JobInfo (class)
  
  instead of:
  
  current_namespace
  \
   - JobInfo (class)
  </code></pre>
  
  <p>So, is this just a wrong way to organize Python code? If not, what is a correct way to split related code up but still collect it in a module-like way?</p>
  
  <p>Maybe the best case scenario is that doing 'from doit_tools import JobInfo' is a little confusing for someone using the package?</p>
  
  <p>Maybe a python file called 'api' so that people using the code do the following?:</p>
  
  <pre><code>import doit_tools.api
  from doit_tools.api import JobInfo
  </code></pre>
  
  <p>============================================</p>
  
  <p>Examples in response to comments:</p>
  
  <p>Take the following package contents, inside folder 'foo' which is in python path.</p>
  
  <p><code>foo/__init__.py</code></p>
  
  <pre><code>__all__ = ['doit','dataholder','getSomeStuff','hold_more_data','SpecialCase']
  from another_class import doit
  from another_class import dataholder
  from descriptive_name import getSomeStuff
  from descriptive_name import hold_more_data
  from specialcase import SpecialCase
  </code></pre>
  
  <p><code>foo/specialcase.py</code></p>
  
  <pre><code>class SpecialCase:
      pass
  </code></pre>
  
  <p><code>foo/more.py</code></p>
  
  <pre><code>def getSomeStuff():
      pass
  
  class hold_more_data(object):
      pass
  </code></pre>
  
  <p><code>foo/stuff.py</code></p>
  
  <pre><code>def doit():
      print "I'm a function."
  
  class dataholder(object):
      pass
  </code></pre>
  
  <p>Do this:</p>
  
  <pre><code>&gt;&gt;&gt; import foo
  &gt;&gt;&gt; for thing in dir(foo): print thing
  ... 
  SpecialCase
  __builtins__
  __doc__
  __file__
  __name__
  __package__
  __path__
  another_class
  dataholder
  descriptive_name
  doit
  getSomeStuff
  hold_more_data
  specialcase
  </code></pre>
  
  <p><code>another_class</code> and <code>descriptive_name</code> are there cluttering things up, and also have extra copies of e.g. doit() underneath their namespaces.</p>
  
  <p>If I have a class named Data inside a file named Data.py, when I do 'from Data import Data' then I get a namespace conflict because Data is a class in the current namespace that is inside module Data, somehow is also in the current namespace.  (But Python seems to be able to handle this.)</p>