title: How do you filter search results in Wagtail based on a ManyToManyField?
body: <p>I have a Wagtail site which defines an <code>Event</code> model. These Events have multiple Event Sponsors, which are associated by a <code>ManyToManyField</code> on the <code>EventSponsor</code> model:</p>
  
  <pre><code>class Event(index.Indexed, ClusterableModel):
  
      title       = models.CharField(max_length=255)
      start_date  = models.DateTimeField()
      end_date    = models.DateTimeField(null=True, blank=True)
      description = RichTextField(blank=True)
  
      search_fields = [
          index.SearchField('title', partial_match=True, boost=2.0),
          index.SearchField('description'),
          index.RelatedFields('sponsors', [
              index.SearchField('name', partial_match=True)
          ]),
  
          index.FilterField('end_date'),
          index.FilterField('sponsors'),
      ]
  
  class EventSponsor(index.Indexed, models.Model):
  
      sponsor_id = models.IntegerField()
      name = models.CharField(max_length=255)
      url = models.URLField(blank=True)
  
      events = models.ManyToManyField(Event, related_name='sponsors')
  
      search_fields = [
          index.SearchField('name', partial_match=True),
      ]
  </code></pre>
  
  <p>In addition to this, different Sites on my Wagtail server include Events in their calendar based on a set of selected Event Sponsors specific to that site.</p>
  
  <p>So building the calendar listing queryset for each site looks like this:</p>
  
  <pre><code>def get_events_for_current_site(request, listing):
      try:
          event_sponsor_settings = EventSponsorSettings.objects.get(site=request.site)
      except EventSponsorSettings.DoesNotExist:
          # If there's no EventSponsorSettings for this Site, return an empty QuerySet. This shouldn't really ever happen.
          return Event.objects.none()
  
      # Return the selected Events in decending order of start date.
      query = Event.objects.filter(sponsors__in=event_sponsor_settings.selected_event_sponsors)
      if listing == 'upcoming_events':
          return query.order_by('start_date').filter(end_date__gte=timezone.now())
      else:
          return query.order_by('-start_date').filter(end_date__lt=timezone.now())
  </code></pre>
  
  <p><code>event_sponsor_settings.selected_event_sponsors</code> is a list of <code>EventSponsor</code> objects. This queryset works just fine for the listing pages.</p>
  
  <p>I need the search functionality (using the Elasticsearch backend) on each Site to include only the Events which would appear on the current Site's calendar. So I want my base queryset to be the same one used by the calendar pages (or to at least do the same filtering). So my Event search code basically calls:</p>
  
  <pre><code>backend.search(search_query, get_events_for_current_site())
  </code></pre>
  
  <p>However, I've run into two problems:</p>
  
  <p>1) If I use <code>index.FilterField('sponsors')</code> in <code>Event.search_fields</code>, I get this error when I run <code>manage.py update_index</code>:</p>
  
  <pre><code>Traceback (most recent call last):
    File "./manage.py", line 33, in &lt;module&gt;
      execute_from_command_line(argv)
    File "/multitenant-ve/lib/python2.7/site-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
      utility.execute()
    File "/multitenant-ve/lib/python2.7/site-packages/django/core/management/__init__.py", line 345, in execute
      self.fetch_command(subcommand).run_from_argv(self.argv)
    File "/multitenant-ve/lib/python2.7/site-packages/django/core/management/base.py", line 348, in run_from_argv
      self.execute(*args, **cmd_options)
    File "/multitenant-ve/lib/python2.7/site-packages/django/core/management/base.py", line 399, in execute
      output = self.handle(*args, **options)
    File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/management/commands/update_index.py", line 120, in handle
      self.update_backend(backend_name, schema_only=options.get('schema_only', False))
    File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/management/commands/update_index.py", line 77, in update_backend
      index.add_model(model)
    File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/backends/elasticsearch.py", line 536, in add_model
      index=self.name, doc_type=mapping.get_document_type(), body=mapping.get_mapping()
    File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/backends/elasticsearch.py", line 137, in get_mapping
      self.get_field_mapping(field) for field in self.model.get_search_fields()
    File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/backends/elasticsearch.py", line 137, in &lt;genexpr&gt;
      self.get_field_mapping(field) for field in self.model.get_search_fields()
    File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/backends/elasticsearch.py", line 119, in get_field_mapping
      return self.get_field_column_name(field), mapping
    File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/backends/elasticsearch.py", line 72, in get_field_column_name
      return field.get_attname(self.model) + '_filter'
    File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/index.py", line 178, in get_attname
      return field.attname
  AttributeError: 'ManyToManyRel' object has no attribute 'attname'
  </code></pre>
  
  <p>2) If I take out <code>index.FilterField('sponsors')</code>, <code>manage.py update_index</code> works, but I get an error when I search:</p>
  
  <pre><code>Cannot filter search results with field "eventsponsor_id". Please add index.FilterField('eventsponsor_id') to Event.search_fields.
  </code></pre>
  
  <p>So I tried adding <code>index.FilterField('eventsponsor_id')</code>, bit it gives this warning during <code>update_index</code>: <code>Event.search_fields contains field 'eventsponsor_id' but it doesn't exist</code>, and causes this traceback at search time:</p>
  
  <pre><code>Traceback:
  File "/multitenant-ve/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
    174.                     response = self.process_exception_by_middleware(e, request)
  File "/multitenant-ve/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
    172.                     response = response.render()
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/response.py" in render
    160.             self.content = self.rendered_content
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/response.py" in rendered_content
    137.         content = template.render(context, self._request)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/backends/django.py" in render
    95.             return self.template.render(context)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render
    206.                     return self._render(context)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in _render
    197.         return self.nodelist.render(context)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render
    992.                 bit = node.render_annotated(context)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render_annotated
    959.             return self.render(context)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/loader_tags.py" in render
    173.         return compiled_parent._render(context)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in _render
    197.         return self.nodelist.render(context)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render
    992.                 bit = node.render_annotated(context)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render_annotated
    959.             return self.render(context)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/loader_tags.py" in render
    173.         return compiled_parent._render(context)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in _render
    197.         return self.nodelist.render(context)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render
    992.                 bit = node.render_annotated(context)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render_annotated
    959.             return self.render(context)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/loader_tags.py" in render
    69.                 result = block.nodelist.render(context)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render
    992.                 bit = node.render_annotated(context)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render_annotated
    959.             return self.render(context)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/defaulttags.py" in render
    220.                     nodelist.append(node.render_annotated(context))
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/base.py" in render_annotated
    959.             return self.render(context)
  File "/multitenant-ve/lib/python2.7/site-packages/django/template/defaulttags.py" in render
    325.             if match:
  File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/backends/base.py" in __len__
    174.         return len(self.results())
  File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/backends/base.py" in results
    137.             self._results_cache = self._do_search()
  File "/multitenant-ve/src/wagtail/wagtail/wagtailsearch/backends/elasticsearch.py" in _do_search
    452.         hits = self.backend.es.search(**params)
  File "/multitenant-ve/lib/python2.7/site-packages/elasticsearch/client/utils.py" in _wrapped
    69.             return func(*args, params=params, **kwargs)
  File "/multitenant-ve/lib/python2.7/site-packages/elasticsearch/client/__init__.py" in search
    531.             doc_type, '_search'), params=params, body=body)
  File "/multitenant-ve/lib/python2.7/site-packages/elasticsearch/transport.py" in perform_request
    273.             body = self.serializer.dumps(body)
  File "/multitenant-ve/lib/python2.7/site-packages/elasticsearch/serializer.py" in dumps
    47.             raise SerializationError(data, e)
  
  Exception Type: SerializationError at /search
  Exception Value: ({u'query': {u'filtered': {u'filter': {u'and': [{u'prefix': {u'content_type': u'event'}}, {'and': [{u'terms': {u'eventsponsor_id_filter': [&lt;EventSponsor: Division of Geological and Planetary Sciences (9003)&gt;]}}, {u'range': {u'end_date_filter': {'gte': datetime.datetime(2017, 3, 29, 0, 42, 7, 462939, tzinfo=&lt;UTC&gt;)}}}]}]}, u'query': {u'multi_match': {u'query': u'geo', u'fields': [u'_all', u'_partials']}}}}}, TypeError("Unable to serialize &lt;EventSponsor: Division of Geological and Planetary Sciences (9003)&gt; (type: &lt;class 'templated_cms.models.events.EventSponsor'&gt;)",))
  </code></pre>
  
  <p>So, I tried changing the queryset in <code>get_events_for_current_site()</code> to <code>Event.objects.filter(sponsors__id__in=[s.id for s in event_sponsor_settings.selected_event_sponsors])</code></p>
  
  <p>This fixes the error... but I get no search results at all. </p>
  
  <p>I'm entirely stumped on how to deal with this. :(</p>