title: Protocol Buffer import resolution
body: <p>Before reading through this rather long question, I've raised a bug <a href="https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1103" rel="nofollow noreferrer">https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1103</a>.</p>
  
  <p>The documentation for Proto <a href="https://developers.google.com/protocol-buffers/docs/proto#services" rel="nofollow noreferrer">Packages and Name Resolution</a> states</p>
  
  <blockquote>
    <p>You can use definitions from other .proto files by importing them. To
    import another .proto's definitions, you add an import statement to
    the top of your file.</p>
  </blockquote>
  
  <p>My <code>example.proto</code> depends on <a href="https://github.com/googleapis/googleapis/blob/master/google/api/annotations.proto" rel="nofollow noreferrer">annotations.proto</a> to <a href="https://cloud.google.com/endpoints/docs/grpc/transcoding" rel="nofollow noreferrer">transcode HTTP/JSON to gRPC</a>. This is a trivial example, but note I use the import path from the <a href="https://github.com/googleapis/googleapis/tree/master/google/api" rel="nofollow noreferrer">googleapis/google/api</a> Git repo (i.e. <code>google/api/annotations.proto</code>):</p>
  
  <pre><code>syntax = "proto3";
  import "google/api/annotations.proto";
  
  message MyExample {
    // Message definition here.
  }
  </code></pre>
  
  <p>Note, <a href="https://github.com/googleapis/googleapis/blob/master/google/api/annotations.proto" rel="nofollow noreferrer">annotations.proto</a> depends on <a href="https://github.com/googleapis/googleapis/blob/master/google/api/http.proto" rel="nofollow noreferrer">http.proto</a> - they are siblings in the same package (<a href="https://github.com/googleapis/googleapis/tree/master/google/api" rel="nofollow noreferrer">googleapis/google/api</a>)</p>
  
  <p><a href="https://i.stack.imgur.com/GELJy.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/GELJy.png" alt="enter image description here"></a></p>
  
  <p>My local project directory contains three .proto files:</p>
  
  <ol>
  <li><code>example.proto</code></li>
  <li><code>google/api/annotations.proto</code></li>
  <li><code>google/api/http.proto</code></li>
  </ol>
  
  <p>...or as a tree:</p>
  
  <pre><code>|____google
  | |____api
  | | |____annotations.proto
  | | |____http.proto
  |____example.proto
  </code></pre>
  
  <p>The target (or, 'out') directories are added too, ready to receive the generated python files:</p>
  
  <pre><code>|____generated_pb2
  | |____google
  | | |____api
  </code></pre>
  
  <p>There for my complete project directory structure is:</p>
  
  <ul>
  <li><code>example.proto</code></li>
  <li><code>google/api/annotations.proto</code></li>
  <li><code>google/api/http.proto</code></li>
  <li><code>generated_pb2/google/api</code></li>
  </ul>
  
  <p>...or as a tree:</p>
  
  <pre><code>|____example.proto
  |____google
  | |____api
  | | |____annotations.proto
  | | |____http.proto
  |____generated_pb2
  | |____google
  | | |____api
  </code></pre>
  
  <p>With this in place I can compile my protos (formatting added for readability):</p>
  
  <pre><code>python -m grpc_tools.protoc
    --python_out=generated_pb2
    --grpc_python_out=generated_pb2
    -I ~/protoc/include/google/protobuf
    -I /google/api
    example.proto
  </code></pre>
  
  <p>Breaking this down:</p>
  
  <ul>
  <li><code>generated_pb2</code> - Destination for generated python files and gprc files.</li>
  <li><code>~/protoc/include/google/protobuf</code> - Location of common protos shipped with protoc binary, needed since <a href="https://github.com/googleapis/googleapis/blob/master/google/api/annotations.proto" rel="nofollow noreferrer">annotations.proto</a> depends on <a href="https://github.com/google/protobuf/blob/master/src/google/protobuf/descriptor.proto" rel="nofollow noreferrer">google/protobuf/descriptor.proto</a>.</li>
  <li><code>google/api</code> - Location of <code>annotations.proto</code> and <code>http.proto</code></li>
  </ul>
  
  <p>This compiles <code>example.proto</code> giving:</p>
  
  <ul>
  <li><code>generated_pb2/example_pb2.py</code></li>
  <li><code>generated_pb2/example_pb2_gprc.py</code></li>
  </ul>
  
  <p>However the first line of <code>generated_pb2/example_pb2.py</code> imports the generated files for <code>annotations.proto</code>:</p>
  
  <pre><code>from google.api import annotations_pb2 as google_dot_api_dot_annotations__pb2
  </code></pre>
  
  <p>This file doesn't exist. No problem, I'll compile <code>annotations.proto</code> separately:</p>
  
  <pre><code>python -m grpc_tools.protoc
    --python_out=generated_pb2/google/api
    --grpc_python_out=generated_pb2/google/api
    -I ~/protoc/include/google/protobuf
    -I google/api annotations.proto
  </code></pre>
  
  <p>Breaking this down:</p>
  
  <ul>
  <li><code>generated_pb2/google/api</code> - Destination for generated python files and gprc files.</li>
  <li><code>~/protoc/include/google/protobuf</code> - Location of common protos shipped with protoc binary, needed since <a href="https://github.com/googleapis/googleapis/blob/master/google/api/annotations.proto" rel="nofollow noreferrer">annotations.proto</a> depends on <a href="https://github.com/google/protobuf/blob/master/src/google/protobuf/descriptor.proto" rel="nofollow noreferrer">google/protobuf/descriptor.proto</a>.</li>
  <li><code>google/api</code> - Location of <code>http.proto</code> upon which <code>annotations.proto</code> depends.</li>
  </ul>
  
  <p>Unfortunately I get an error at this point:</p>
  
  <pre><code>google/api/http.proto: File not found.
  annotations.proto: Import "google/api/http.proto" was not found or had errors.
  annotations.proto:30:3: "HttpRule" is not defined.
  </code></pre>
  
  <p>I guess this is because <code>annotations.proto</code> looks for <code>http.proto</code> in <code>google/api</code>:</p>
  
  <pre><code>syntax = "proto3";
  package google.api;
  
  import "google/api/http.proto";
  import "google/protobuf/descriptor.proto";
  </code></pre>
  
  <p>However it's unclear how this dependency is resolved.  <code>protoc --help</code> documents the <code>-I</code> flag:</p>
  
  <pre><code>-IPATH, --proto_path=PATH   Specify the directory in which to search for
                              imports.  May be specified multiple times;
                              directories will be searched in order.  If not
                              given, the current working directory is used.
  </code></pre>
  
  <p>How can <code>http.proto</code> upon which <code>annotations.proto</code> depends, be resolved?</p>