 <p>We use a simular approach. We use an Oracle database as primary datastore. </p> <p>We use PLSQL to flatten/convert data. For initial load we add data(records) to a "oneshot" table. Updates of the data will be flatten/converted and result in records in an "update" table. The oneshot and update table will be mapped to a single index in Elasticsearch.</p> <p>Initial load of ES:</p> <p>[Oracle DB]--->flatten data (pl sql)-->[records to animal_oneshot_river table, records to user_oneshot_river table]</p> <p>The data will be pulled by the river to for example <a href="http://localhost/9200/zoo/animal" rel="nofollow">http://localhost/9200/zoo/animal</a> and <a href="http://localhost/9200/zoo/user" rel="nofollow">http://localhost/9200/zoo/user</a>)</p> <p>Updates</p> <p>[Software]---->Change data--->[Oracle DB]--->flatten data (pl sql)-->[records to animal_update_river table, records to user_update_river table]</p> <p>The update tables also contains a type of change (insert, update or delete).</p> <p>The river wil poll the update_river tables for updates and mutates the data in Elasticsearch (we use a pull). The records will be deleted after processing by the river.</p> <p>Data changes to Elasticsearch won't be send to Oracle. All changes on the primary datastore will be done by our own bussiness logic software.</p> <p>We also write data to _spare tables (animal_oneshot_river_spare) because that makes it possible to reload the Elasticsearch without downtime and without synchronisation issues (we switch aliasses after reloading Elasticsearch). </p>