{"action":"created","issue":{"url":"https://api.github.com/repos/PaddlePaddle/Paddle/issues/14501","repository_url":"https://api.github.com/repos/PaddlePaddle/Paddle","labels_url":"https://api.github.com/repos/PaddlePaddle/Paddle/issues/14501/labels{/name}","comments_url":"https://api.github.com/repos/PaddlePaddle/Paddle/issues/14501/comments","events_url":"https://api.github.com/repos/PaddlePaddle/Paddle/issues/14501/events","html_url":"https://github.com/PaddlePaddle/Paddle/issues/14501","id":382507597,"node_id":"MDU6SXNzdWUzODI1MDc1OTc=","number":14501,"title":"求助,怎么把原始数据灌入到模型里去?","user":{"login":"lucywsq","id":38712162,"node_id":"MDQ6VXNlcjM4NzEyMTYy","avatar_url":"https://avatars3.githubusercontent.com/u/38712162?v=4","gravatar_id":"","url":"https://api.github.com/users/lucywsq","html_url":"https://github.com/lucywsq","followers_url":"https://api.github.com/users/lucywsq/followers","following_url":"https://api.github.com/users/lucywsq/following{/other_user}","gists_url":"https://api.github.com/users/lucywsq/gists{/gist_id}","starred_url":"https://api.github.com/users/lucywsq/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lucywsq/subscriptions","organizations_url":"https://api.github.com/users/lucywsq/orgs","repos_url":"https://api.github.com/users/lucywsq/repos","events_url":"https://api.github.com/users/lucywsq/events{/privacy}","received_events_url":"https://api.github.com/users/lucywsq/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-11-20T05:34:21Z","updated_at":"2018-11-20T07:41:59Z","closed_at":null,"author_association":"NONE","body":"\r\n![1](https://user-images.githubusercontent.com/38712162/48753398-c6ef8380-ecc8-11e8-9995-f74affb31a77.png)\r\n\r\n\r\n![2](https://user-images.githubusercontent.com/38712162/48753249-4466c400-ecc8-11e8-9133-2a760905d663.png)\r\n\r\n\r\n每个文件的数据内容:我有很多这种csv格式的文件,我想借用cnn对图片分类识别的方法来分类识别这些文件,每个文件可以理解成为一张图片,用cnn对每个文件的数据进行卷积运算,但是现在不知道怎么把这个源数据灌进去,哪位能指导一下吗?下面是我找的个cnn识别手写数据的代码,最好能帮我改一下代码,将图片数据替换成我的csv数据,谢谢!\r\n\r\n###########################################################################################\r\n\r\ncoding:utf-8\r\nimport os\r\nfrom PIL import Image\r\nimport numpy as np\r\nimport paddle.v2 as paddle\r\n\r\n设置是否用gpu，0为否，1为是\r\nwith_gpu = os.getenv('WITH_GPU', '0') != '0'\r\n\r\n定义网络结构\r\n保存参数\r\ndef convolutional_neural_network(img):\r\n 第一层卷积层\r\nconv_pool_1 = paddle.networks.simple_img_conv_pool(\r\ninput=img,\r\nfilter_size=5,\r\nnum_filters=20,\r\nnum_channel=1,\r\npool_size=2,\r\npool_stride=2,\r\nact=paddle.activation.Relu())\r\n加一层dropout层\r\ndrop_1 = paddle.layer.dropout(input=conv_pool_1, dropout_rate=0.2)\r\n第二层卷积层\r\nconv_pool_2 = paddle.networks.simple_img_conv_pool(\r\ninput=drop_1,\r\nfilter_size=5,\r\nnum_filters=50,\r\nnum_channel=20,\r\npool_size=2,\r\npool_stride=2,\r\nact=paddle.activation.Relu())\r\n 加一层dropout层\r\ndrop_2 = paddle.layer.dropout(input=conv_pool_2, dropout_rate=0.5)\r\n 全连接层\r\nfc1 = paddle.layer.fc(input=drop_2, size=10, act=paddle.activation.Linear())\r\nbn = paddle.layer.batch_norm(input=fc1,act=paddle.activation.Relu(),\r\nlayer_attr=paddle.attr.Extra(drop_rate=0.2))\r\npredict = paddle.layer.fc(input=bn, size=10, act=paddle.activation.Softmax())\r\nreturn predict\r\n\r\ndef main():\r\n初始化定义跑模型的设备\r\npaddle.init(use_gpu=with_gpu, trainer_count=1)\r\n\r\n读取数据\r\nimages = paddle.layer.data(\r\nname='pixel', type=paddle.data_type.dense_vector(784))\r\nlabel = paddle.layer.data(\r\nname='label', type=paddle.data_type.integer_value(10))\r\n\r\n 调用之前定义的网络结构\r\npredict = convolutional_neural_network(images)\r\n\r\n 定义损失函数\r\ncost = paddle.layer.classification_cost(input=predict, label=label)\r\n\r\n指定训练相关的参数\r\nparameters = paddle.parameters.create(cost)\r\n\r\n 定义训练方法\r\noptimizer = paddle.optimizer.Momentum(\r\nlearning_rate=0.1 / 128.0,\r\nmomentum=0.9,\r\nregularization=paddle.optimizer.L2Regularization(rate=0.0005 * 128))\r\n\r\n 训练模型\r\ntrainer = paddle.trainer.SGD(\r\ncost=cost, parameters=parameters, update_equation=optimizer)\r\n\r\n\r\nlists = []\r\n\r\n 定义event_handler，输出训练过程中的结果\r\ndef event_handler(event):\r\nif isinstance(event, paddle.event.EndIteration):\r\nif event.batch_id % 100 == 0:\r\nprint \"Pass %d, Batch %d, Cost %f, %s\" % (\r\nevent.pass_id, event.batch_id, event.cost, event.metrics)\r\nif isinstance(event, paddle.event.EndPass):\r\n 保存参数\r\nwith open('params_pass_%d.tar' % event.pass_id, 'w') as f:\r\nparameters.to_tar(f)\r\n\r\nresult = trainer.test(reader=paddle.batch(\r\npaddle.dataset.mnist.test(), batch_size=128))\r\nprint \"Test with Pass %d, Cost %f, %s\\n\" % (\r\nevent.pass_id, result.cost, result.metrics)\r\nlists.append((event.pass_id, result.cost,\r\nresult.metrics['classification_error_evaluator']))\r\n\r\ntrainer.train(\r\nreader=paddle.batch(\r\npaddle.reader.shuffle(paddle.dataset.mnist.train(), buf_size=8192),\r\nbatch_size=128),\r\nevent_handler=event_handler,\r\nnum_passes=10)\r\n\r\n找到训练误差最小的一次结果\r\nbest = sorted(lists, key=lambda list: float(list[1]))[0]\r\nprint 'Best pass is %s, testing Avgcost is %s' % (best[0], best[1])\r\nprint 'The classification accuracy is %.2f%%' % (100 - float(best[2]) * 100)\r\n\r\n加载数据 \r\ndef load_image(file):\r\nim = Image.open(file).convert('L')\r\nim = im.resize((28, 28), Image.ANTIALIAS)\r\nim = np.array(im).astype(np.float32).flatten()\r\nim = im / 255.0\r\nreturn im\r\n\r\n测试结果\r\ntest_data = []\r\ncur_dir = os.path.dirname(os.path.realpath(__file__))\r\ntest_data.append((load_image(cur_dir + '/image/infer_3.png'), ))\r\n\r\nprobs = paddle.infer(\r\noutput_layer=predict, parameters=parameters, input=test_data)\r\nlab = np.argsort(-probs) # probs and lab are the results of one batch data\r\nprint \"Label of image/infer_3.png is: %d\" % lab[0][0]\r\n\r\n\r\nif __name__ == '__main__':\r\nmain()"},"comment":{"url":"https://api.github.com/repos/PaddlePaddle/Paddle/issues/comments/440174310","html_url":"https://github.com/PaddlePaddle/Paddle/issues/14501#issuecomment-440174310","issue_url":"https://api.github.com/repos/PaddlePaddle/Paddle/issues/14501","id":440174310,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0MDE3NDMxMA==","user":{"login":"yeyupiaoling","id":26297768,"node_id":"MDQ6VXNlcjI2Mjk3NzY4","avatar_url":"https://avatars1.githubusercontent.com/u/26297768?v=4","gravatar_id":"","url":"https://api.github.com/users/yeyupiaoling","html_url":"https://github.com/yeyupiaoling","followers_url":"https://api.github.com/users/yeyupiaoling/followers","following_url":"https://api.github.com/users/yeyupiaoling/following{/other_user}","gists_url":"https://api.github.com/users/yeyupiaoling/gists{/gist_id}","starred_url":"https://api.github.com/users/yeyupiaoling/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yeyupiaoling/subscriptions","organizations_url":"https://api.github.com/users/yeyupiaoling/orgs","repos_url":"https://api.github.com/users/yeyupiaoling/repos","events_url":"https://api.github.com/users/yeyupiaoling/events{/privacy}","received_events_url":"https://api.github.com/users/yeyupiaoling/received_events","type":"User","site_admin":false},"created_at":"2018-11-20T07:41:59Z","updated_at":"2018-11-20T07:41:59Z","author_association":"CONTRIBUTOR","body":"@lucywsq 可以参考这个：https://github.com/yeyupiaoling/LearnPaddle/blob/master/note4/code/MyReader.py\r\n"}}
{"actor":{"display_login":"yeyupiaoling"}}