Take the 2-minute tour ×

Everything seemed to be working fine as I added items to my Postgres database. Without changing anything, my Rails app started failing whenever it started Madeleine anywhere in my app:

EncodingError in EventsController#update
  invalid encoding symbol
  app/controllers/events_controller.rb:137:in `update'

137 is the problem line:

135 def update
136   @event = Event.find(params[:id])
137   m = SnapshotMadeleine.new("bayes_data")

I'm able to classify things in the console, though, which is part of what's causing me confusion. In the console, this works fine:

m = SnapshotMadeleine.new("bayes_data") {}
   => #<Madeleine::DefaultSnapshotMadeleine:0x000...
   => #<Classifier::Bayes:0x000...
m.system.classify "test"

I'm using the latest Classifier gem with Madeleine.

I figured that something was corrupted in my snapshot that it attempted to load, so I removed it, but that didn't solve the problem. Here's my second-to-last snapshot (now the most recent one):

a = File.read('bayes_data/000000000000000000041.snapshot')
   => #<Encoding:UTF-8>
   => true

Not sure what's going on here. I saw some people with Ruby 1.9.3-p125 had similar problems, so I upgraded to the latest stable release, 1.9.3-p194, but that didn't help, either.

Here's a link to the documentation for Classifier, which mentions how to use Madeleine as well: http://classifier.rubyforge.org/

I'd really appreciate any insight into what's going on here. Thanks!

share|improve this question
Did you ever solve this? I am finding the same issue! –  joshs Sep 22 '12 at 22:49
Never did. I wound up abandoning the feature for the sake of time. I'd recommend finding another, more stable gem. –  airlok Oct 11 '12 at 21:26
you should clean your string before feeding it to the classifier, for example, transliterate the accented characters –  Zack Xu May 7 '13 at 14:42
add comment

3 Answers

i had an issue with the rails_admin gem with mysql adapter which was not encoding aware, maybe you can check if your postgres adaptar is and if not try another one

share|improve this answer
add comment

I don't know why the standard Marshal class doesn't work, but I had good results using

m = SnapshotMadeleine.new("bayes_data", YAML) do 
  b = Classifier::Bayes.new "Positive", "Negative

and then

m = SnapshotMadeleine.new("bayes_data", YAML)

something broken with Marshal? Not really sure.

share|improve this answer
add comment

You have to make sure the strings that you use to train the data are UTF-8 encoded. If you are training the data set, an ugly hack is to have

Encoding.default_external = Encoding::UTF_8
Encoding.default_internal = Encoding::UTF_8

in your script.

share|improve this answer
add comment

Your Answer


