{"action":"created","issue":{"url":"https://api.github.com/repos/f90/Wave-U-Net/issues/13","repository_url":"https://api.github.com/repos/f90/Wave-U-Net","labels_url":"https://api.github.com/repos/f90/Wave-U-Net/issues/13/labels{/name}","comments_url":"https://api.github.com/repos/f90/Wave-U-Net/issues/13/comments","events_url":"https://api.github.com/repos/f90/Wave-U-Net/issues/13/events","html_url":"https://github.com/f90/Wave-U-Net/issues/13","id":383077575,"node_id":"MDU6SXNzdWUzODMwNzc1NzU=","number":13,"title":"Question - About Prediction time over CPU and GPU","user":{"login":"loretoparisi","id":163333,"node_id":"MDQ6VXNlcjE2MzMzMw==","avatar_url":"https://avatars3.githubusercontent.com/u/163333?v=4","gravatar_id":"","url":"https://api.github.com/users/loretoparisi","html_url":"https://github.com/loretoparisi","followers_url":"https://api.github.com/users/loretoparisi/followers","following_url":"https://api.github.com/users/loretoparisi/following{/other_user}","gists_url":"https://api.github.com/users/loretoparisi/gists{/gist_id}","starred_url":"https://api.github.com/users/loretoparisi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/loretoparisi/subscriptions","organizations_url":"https://api.github.com/users/loretoparisi/orgs","repos_url":"https://api.github.com/users/loretoparisi/repos","events_url":"https://api.github.com/users/loretoparisi/events{/privacy}","received_events_url":"https://api.github.com/users/loretoparisi/received_events","type":"User","site_admin":false},"labels":[{"id":1135585719,"node_id":"MDU6TGFiZWwxMTM1NTg1NzE5","url":"https://api.github.com/repos/f90/Wave-U-Net/labels/improvement","name":"improvement","color":"3647db","default":false},{"id":912642580,"node_id":"MDU6TGFiZWw5MTI2NDI1ODA=","url":"https://api.github.com/repos/f90/Wave-U-Net/labels/question","name":"question","color":"d876e3","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-11-21T11:35:22Z","updated_at":"2018-11-21T15:05:33Z","closed_at":null,"author_association":"NONE","body":"I'm doing some tests for CPU and GPU environment usages for prediction (Predict.py).\r\nI'm using an audio file `Audio: mp3, 44100 Hz, stereo, fltp, 192 kb/s` of duration `00:03:15.29`\r\n\r\n```sh\r\n$ ffprobe /audio/12380187.mp3\r\nffprobe version 4.0 Copyright (c) 2007-2018 the FFmpeg developers\r\n  built with Apple LLVM version 9.1.0 (clang-902.0.39.1)\r\n  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.0 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-gpl --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma\r\n  libavutil      56. 14.100 / 56. 14.100\r\n  libavcodec     58. 18.100 / 58. 18.100\r\n  libavformat    58. 12.100 / 58. 12.100\r\n  libavdevice    58.  3.100 / 58.  3.100\r\n  libavfilter     7. 16.100 /  7. 16.100\r\n  libavresample   4.  0.  0 /  4.  0.  0\r\n  libswscale      5.  1.100 /  5.  1.100\r\n  libswresample   3.  1.100 /  3.  1.100\r\n  libpostproc    55.  1.100 / 55.  1.100\r\nInput #0, mp3, from '/audio/12380187.mp3':\r\n  Metadata:\r\n    encoder         : Lavf56.40.101\r\n  Duration: 00:03:15.29, start: 0.025057, bitrate: 192 kb/s\r\n    Stream #0:0: Audio: mp3, 44100 Hz, stereo, fltp, 192 kb/s\r\n    Metadata:\r\n      encoder         : Lavc56.60\r\n```\r\n\r\nOn a  **Intel i7 - 12 core CPU** the prediction time log says **Completed after 0:03:19**\r\n\r\n```sh\r\n$ time python Predict.py with cfg.full_44KHz input_path=/audio/12380187.mp3 output_path=/audio_sep/\r\nTraining full singing voice separation model, with difference output and input context (valid convolutions) and stereo input/output, and learned upsampling layer, and 44.1 KHz sampling rate\r\nWARNING - Waveunet Prediction - No observers have been added to this run\r\nINFO - Waveunet Prediction - Running command 'main'\r\nINFO - Waveunet Prediction - Started\r\nProducing source estimates for input mixture file /audio/12380187.mp3\r\nTesting...\r\n2018-11-20 14:54:05.306099: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA\r\nNum of variables64\r\nINFO:tensorflow:Restoring parameters from checkpoints/full_44KHz/full_44KHz-236118\r\nINFO - tensorflow - Restoring parameters from checkpoints/full_44KHz/full_44KHz-236118\r\nPre-trained model restored for song prediction\r\nINFO - Waveunet Prediction - Completed after 0:03:19\r\n\r\nreal\t3m26.034s\r\nuser\t13m30.420s\r\nsys\t4m40.200s\r\n```\r\n\r\nwhile on **Intel Xeon 12 core plus 2x Nvidia GeForce GTX 1080** says **Completed after 0:00:16**\r\n\r\n```sh\r\n$ time python Predict.py with cfg.full_44KHz input_path=/audio/12380187.mp3\r\n/usr/local/lib/python2.7/dist-packages/scikits/audiolab/soundio/play.py:48: UserWarning: Could not import alsa backend; most probably, you did not have alsa headers when building audiolab\r\n  warnings.warn(\"Could not import alsa backend; most probably, \"\r\nTraining full singing voice separation model, with difference output and input context (valid convolutions) and stereo input/output, and learned upsampling layer, and 44.1 KHz sampling rate\r\nWARNING - Waveunet Prediction - No observers have been added to this run\r\nINFO - Waveunet Prediction - Running command 'main'\r\nINFO - Waveunet Prediction - Started\r\nProducing source estimates for input mixture file /audio/12380187.mp3\r\nTesting...\r\n2018-11-21 12:34:13.829481: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\r\n2018-11-21 12:34:13.830157: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 0 with properties: \r\nname: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.8475\r\npciBusID: 0000:01:00.0\r\ntotalMemory: 7.92GiB freeMemory: 7.46GiB\r\n2018-11-21 12:34:13.961794: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\r\n2018-11-21 12:34:13.962562: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 1 with properties: \r\nname: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.8475\r\npciBusID: 0000:02:00.0\r\ntotalMemory: 7.93GiB freeMemory: 7.81GiB\r\n2018-11-21 12:34:13.963292: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0, 1\r\n2018-11-21 12:34:14.531254: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:\r\n2018-11-21 12:34:14.531305: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929]      0 1 \r\n2018-11-21 12:34:14.531329: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0:   N Y \r\n2018-11-21 12:34:14.531336: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 1:   Y N \r\n2018-11-21 12:34:14.531830: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7209 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0, compute capability: 6.1)\r\n2018-11-21 12:34:14.589915: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:1 with 7543 MB memory) -> physical GPU (device: 1, name: GeForce GTX 1080, pci bus id: 0000:02:00.0, compute capability: 6.1)\r\nNum of variables64\r\nINFO:tensorflow:Restoring parameters from checkpoints/full_44KHz/full_44KHz-236118\r\nINFO - tensorflow - Restoring parameters from checkpoints/full_44KHz/full_44KHz-236118\r\nPre-trained model restored for song prediction\r\nINFO - Waveunet Prediction - Completed after 0:00:16\r\n\r\nreal\t0m18.340s\r\nuser\t0m15.972s\r\nsys\t0m5.528s\r\n```\r\n\r\nI'm not sure from logging if tensorflow is using both gpu devices or gpu `0` only. If I'm not wrong, most of the work is done in the `Models.py` here https://github.com/f90/Wave-U-Net/blob/master/Models/UnetSpectrogramSeparator.py#L39 when the computation graph is calculated. I assume that these operations go on `gpu:0` in this configuration, so `gpu:1` will not be used - but I'm not sure of it.\r\n\r\nThank you very much!"},"comment":{"url":"https://api.github.com/repos/f90/Wave-U-Net/issues/comments/440696260","html_url":"https://github.com/f90/Wave-U-Net/issues/13#issuecomment-440696260","issue_url":"https://api.github.com/repos/f90/Wave-U-Net/issues/13","id":440696260,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0MDY5NjI2MA==","user":{"login":"f90","id":23065103,"node_id":"MDQ6VXNlcjIzMDY1MTAz","avatar_url":"https://avatars0.githubusercontent.com/u/23065103?v=4","gravatar_id":"","url":"https://api.github.com/users/f90","html_url":"https://github.com/f90","followers_url":"https://api.github.com/users/f90/followers","following_url":"https://api.github.com/users/f90/following{/other_user}","gists_url":"https://api.github.com/users/f90/gists{/gist_id}","starred_url":"https://api.github.com/users/f90/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/f90/subscriptions","organizations_url":"https://api.github.com/users/f90/orgs","repos_url":"https://api.github.com/users/f90/repos","events_url":"https://api.github.com/users/f90/events{/privacy}","received_events_url":"https://api.github.com/users/f90/received_events","type":"User","site_admin":false},"created_at":"2018-11-21T15:05:33Z","updated_at":"2018-11-21T15:05:33Z","author_association":"OWNER","body":"The part where prediction for an input song is made is actually here:\r\n\r\nhttps://github.com/f90/Wave-U-Net/blob/master/Evaluate.py#L109\r\n\r\nWhat could be changed without a lot of effort would be to change batch size from 1 to the default (16), however that also means prediction requires more RAM/GPU memory. We would have to make sure though that prediction still works exactly the same way as before.\r\n\r\nMulti-GPU implementation is also possible, but requires a bit more effort to get right. Keep in mind this is also all supposed to work right out of the gate without people having to configure the GPU setup.\r\n\r\nIn case someone wants to provide such fast implementations, I am all ears."}}
{"actor":{"display_login":"f90"}}