title: Python code for PSO to optimise SVM parameters
body: <p>I'm trying to implement Particle swarm optimization on support vector machine (SVM) to optimizing cost and gamma parameters (Radial basis function) to improve the accuracy.The SVM program is taking data from mysql db and is working fine.Now I need a PSO code to optimize the cost and gamma parameters and use it in svm program.
  following are the svm and pso codes. Svm works fine but the PSO code does not work.
  the code is written in python.
  Please help..</p>
  
  <p>SVM code:</p>
  
  <pre><code>import MySQLdb
  import cgi
  import cgitb
  import pandas as pd 
  import numpy as np 
  from sklearn.cross_validation import train_test_split
  from sklearn import preprocessing, cross_validation, svm 
  from sklearn.svm import SVR
  import mysql.connector as sql
  import pandas as pd
  cgitb.enable()
  print 'Content-type: text/html\r\n\r'
  form = cgi.FieldStorage()
  e1= form.getvalue('EARNING_PER_SHARE', '')
  e2 = form.getvalue('CASH_INVESTMENT', '')
  e3 = form.getvalue('CURRENT_LIABILITY', '')
  e4 = form.getvalue('TOTAL_REVENUE', '')
  e5 = form.getvalue('GROSS_PROFIT', '')
  db = MySQLdb.connect(host="127.0.0.1", db="cisco", user="root", passwd="")
  cursor = db.cursor()
  cursor.execute("""
  INSERT INTO table1 (EARNING_PER_SHARE, CASH_INVESTMENT, CURRENT_LIABILITY, 
  TOTAL_REVENUE,GROSS_PROFIT) 
  VALUES (%s, %s, %s, %s, %s)
  """, (e1, e2, e3, e4,e5))
  db.commit()
  db.close()
  db_connection = sql.connect(host='127.0.0.1', database='cisco', user='root', 
  password='')
  db_cursor = db_connection.cursor()
  db_cursor.execute('SELECT * FROM table1')
  table_rows = db_cursor.fetchall()
  df = pd.DataFrame(table_rows)
  np = df.as_matrix()
  X = df.drop([4], 1)
  np1 = X.as_matrix()
  y = df[4]
  np2 = y.as_matrix()
  x_train, x_test, y_train, y_test = train_test_split(np1, np2, test_size=0.3)
  clf = svm.SVR()
  clf.fit(np1, np2)
  confidence = clf.score(np1, np2)
  for k in ['rbf']:
  clf = svm.SVR(kernel=k, C=100, gamma=0.0001)
  clf.fit(np1, np2)
  confidence = clf.score(np1, np2)
  print(k,confidence)
  a = clf.predict(np1)
  print ('\npredicted values')
  print (a)
  print ('\nreal values')
  print (np2)
  </code></pre>
  
  <p>PSO code:</p>
  
  <pre><code>def fitness_function(a,x,np2,np1,C,gamma,):    
  C = x[0]
  gamma = x[1]
  clf = svm.SVR(kernel=k, C=10, gamma=0.0001,swarmsize=50)
  clf.fit(np1, np2)
  confidence = clf.score(np1, np2)
  print(k,confidence)
  mse = sqrt(mean_squared_error(np2, a))
  return mse
  lb = [10, 0.0001]
  ub = [1000,0.1]
  xopt, fopt = pso(fitness_function, lb, ub)
  </code></pre>
  
  <p>I want to use RMSE as the fitness function but I think fitness function has to include the parameters that is to be optimized. But, in SVR, the objective function is too complicated to write in python.</p>