title: Configure thinking sphinx with morphology other than english or russian
body: <p>I need to configure thinking sphinx with Spanish stemming and I can't get it to work.</p>
  
  <p>I learned [1] that I needed to compile the sphinx source code with the libstemmer_c library and install it. Additionally, I had to change the configuration of thinking sphinx by adding the libstemmer_es stemmer to morphology.</p>
  
  <p>In detail, this is what I did</p>
  
  <ol>
  <li><p>Remove existing sphinx installation with apt-get</p>
  
  <pre><code> apt-get remove sphinxsearch
  </code></pre></li>
  <li><p>Download and unpack source code of sphinx and the libstemmer_c library and copy content of latter to libstemmer_c directory</p>
  
  <pre><code> wget http://sphinxsearch.com/files/sphinx-2.2.11-release.tar.gz
   tar xvf sphinx-2.2.11-release.tar.gz
   wget http://snowball.tartarus.org/dist/libstemmer_c.tgz
   tar xvf libstemmer_c.tgz
   cp -rf libstemmer_c/* sphinx-2.2.11-release/libstemmer_c/
  </code></pre></li>
  <li><p>Configure, compile and install sphinx with the libstemmer_c library</p>
  
  <pre><code>cd sphinx-2.2.11-release
  ./configure --with-mysql-includes=/usr/include/mysql --with-mysql-libs=/usr/lib/x86_64-linux-gnu --with-libstemmer
  make          
  make install
  </code></pre></li>
  <li><p>Add libstemmer_es stemmer to morphology in thinking_sphinx.yml</p>
  
  <pre><code>development:
    mysql41: 3563
    address: &lt;%= ENV['SPHINX_HOST'] || '' %&gt;
    enable_star: true
    charset_type: utf-8
    min_infix_len: 2
    morphology: libstemmer_es
    ...
  </code></pre></li>
  <li><p>Reconfigure sphinx and regenerate indices</p>
  
  <pre><code>bundle exec rake ts:configure
  bundle exec rake ts:generate
  </code></pre></li>
  <li><p>Restart docker containers and rails server</p></li>
  </ol>
  
  <p>I'm working on a website with various products that are indexed with sphinx. With stemming enabled searching for "cameras" should yield all products with "cameras" or "camera". Currently, searching "cameras" only returns products with "cameras" in the string, but no products with "camera" only.</p>
  
  <p>I'm using Rails 3.2, thinking-sphinx 3.2 and sphinx 2.2.11 on Ubuntu 14.04.4 LTS. Maybe worth to mention that I'm using docker containers. The searchd runs in a separate container apart from the rails application.</p>
  
  <p>UPDATE 1: I can't do <code>rake ts:regenerate</code> since I'm running <em>searchd</em> in a separate docker-container, i.e. my sphinx container. Instead I stop the sphinx container, enter a worker container, run <code>rake ts:clear_rt</code> and <code>rake ts:configure</code>, then restart the sphinx container which also restarts -<em>searchd</em>, enter the sphinx container and then finall run <code>rake ts:generate</code></p>
  
  <p>UPDATE 2: Content of log/development.searchd.log is</p>
  
  <pre><code>[Thu Mar 16 12:24:59.147 2017] [  127] listening on all interfaces, port=3563
  [Thu Mar 16 12:24:59.161 2017] [  127] binlog: replaying log .../development/binlog.001
  [Thu Mar 16 12:24:59.161 2017] [  127] binlog: replay stats: 0 rows in 0 commits; 0 updates, 0 reconfigure; 0 indexes
  [Thu Mar 16 12:24:59.162 2017] [  127] binlog: finished replaying /opt/sharetribe/tmp/binlog/development/binlog.001; 0.0 MB in 0.000 sec
  [Thu Mar 16 12:24:59.162 2017] [  127] binlog: finished replaying total 1 in 0.001 sec
  [Thu Mar 16 12:24:59.163 2017] [  127] DEBUG: SaveMeta: Done.
  [Thu Mar 16 12:24:59.163 2017] [  127] accepting connections
  [Thu Mar 16 12:25:04.175 2017] [  127] DEBUG: ReadLock 0xe42ef8
  [Thu Mar 16 12:25:04.175 2017] [  127] DEBUG: Unlock 0xe42ef8
  [Thu Mar 16 12:25:04.175 2017] [  127] DEBUG: ReadLock 0xe42ef8
  [Thu Mar 16 12:25:04.175 2017] [  127] DEBUG: Unlock 0xe42ef8
  [Thu Mar 16 12:25:04.175 2017] [  127] DEBUG: ReadLock 0xe42ef8
  [Thu Mar 16 12:25:04.175 2017] [  127] DEBUG: Unlock 0xe42ef8
  ... /* many more ReadLock and Unlock */
  [Thu Mar 16 12:28:50.467 2017] [  128] listening on all interfaces, port=3563
  [Thu Mar 16 12:28:50.478 2017] [  128] DEBUG: SaveMeta: Done.
  [Thu Mar 16 12:28:50.478 2017] [  128] accepting connections
  [Thu Mar 16 12:28:55.503 2017] [  128] DEBUG: ReadLock 0x1522ef8
  [Thu Mar 16 12:28:55.503 2017] [  128] DEBUG: Unlock 0x1522ef8
  [Thu Mar 16 12:25:04.175 2017] [  127] DEBUG: ReadLock 0xe42ef8
  [Thu Mar 16 12:25:04.175 2017] [  127] DEBUG: Unlock 0xe42ef8
  [Thu Mar 16 12:25:04.175 2017] [  127] DEBUG: ReadLock 0xe42ef8
  [Thu Mar 16 12:25:04.175 2017] [  127] DEBUG: Unlock 0xe42ef8
  ... /* many more ReadLock and Unlock */
  [Thu Mar 16 12:29:09.806 2017] [  128] caught SIGHUP (seamless=1, in queue=1)
  [Thu Mar 16 12:29:09.806 2017] [  128] DEBUG: CheckRotate invoked
  [Thu Mar 16 12:29:09.806 2017] [  128] DEBUG: /opt/sharetribe/db/sphinx/development/custom_field_value_core.new.sph is not readable. Skipping
  [Thu Mar 16 12:29:09.806 2017] [  128] DEBUG: /opt/sharetribe/db/sphinx/development/listing_core.new.sph is not readable. Skipping
  [Thu Mar 16 12:29:09.806 2017] [  128] WARNING: nothing to rotate after SIGHUP ( in queue=0 )
  [Thu Mar 16 12:29:10.541 2017] [  128] DEBUG: ReadLock 0x1522ef8
  [Thu Mar 16 12:29:10.541 2017] [  128] DEBUG: Unlock 0x1522ef8
  [Thu Mar 16 12:25:04.175 2017] [  127] DEBUG: ReadLock 0xe42ef8
  [Thu Mar 16 12:25:04.175 2017] [  127] DEBUG: Unlock 0xe42ef8
  [Thu Mar 16 12:25:04.175 2017] [  127] DEBUG: ReadLock 0xe42ef8
  [Thu Mar 16 12:25:04.175 2017] [  127] DEBUG: Unlock 0xe42ef8
  ... /* many more ReadLock and Unlock */
  </code></pre>
  
  <p>UPDATE 3: I'm defining a real time index on listings of products with attributes such as title, description, author name etc. </p>
  
  <pre><code>ThinkingSphinx::Index.define :listing, :with =&gt; :real_time do
    indexes title
    indexes description
    indexes custom_field_values_sphinx
    indexes origin_loc.google_address
    indexes author.given_name
    indexes author.username
    indexes location.province
  ...
  </code></pre>
  
  <p>This the underlying model</p>
  
  <pre><code>class Listing &lt; ActiveRecord::Base 
  
    after_save ThinkingSphinx::RealTime.callback_for(:listing)
  ...
  </code></pre>
  
  <p>The Listing.search method is called in a public method of the model</p>
  
  <pre><code>  Listing.search(
        escaped_query,
        :select =&gt; "*, #{SPHINX_WEIGHT_FUNCTION} as w",
        :sql =&gt; {:include =&gt; params[:include]},
        :star =&gt; true,
        :with =&gt; with,
        :with_all =&gt; with_all,
        :order =&gt; params[:sort],
        :per_page =&gt; per_page,
        :page =&gt; page
    )
  </code></pre>
  
  <p>[1] <a href="http://freelancing-gods.com/thinking-sphinx/advanced_config.html#word-stemming--morphology" rel="nofollow noreferrer">http://freelancing-gods.com/thinking-sphinx/advanced_config.html#word-stemming--morphology</a></p>