{"action":"created","issue":{"url":"https://api.github.com/repos/dmlc/mxnet/issues/1828","repository_url":"https://api.github.com/repos/dmlc/mxnet","labels_url":"https://api.github.com/repos/dmlc/mxnet/issues/1828/labels{/name}","comments_url":"https://api.github.com/repos/dmlc/mxnet/issues/1828/comments","events_url":"https://api.github.com/repos/dmlc/mxnet/issues/1828/events","html_url":"https://github.com/dmlc/mxnet/issues/1828","id":147605701,"number":1828,"title":"How sentence data are batched in Model_parallel_lstm?","user":{"login":"harouwu","id":5057263,"avatar_url":"https://avatars.githubusercontent.com/u/5057263?v=3","gravatar_id":"","url":"https://api.github.com/users/harouwu","html_url":"https://github.com/harouwu","followers_url":"https://api.github.com/users/harouwu/followers","following_url":"https://api.github.com/users/harouwu/following{/other_user}","gists_url":"https://api.github.com/users/harouwu/gists{/gist_id}","starred_url":"https://api.github.com/users/harouwu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/harouwu/subscriptions","organizations_url":"https://api.github.com/users/harouwu/orgs","repos_url":"https://api.github.com/users/harouwu/repos","events_url":"https://api.github.com/users/harouwu/events{/privacy}","received_events_url":"https://api.github.com/users/harouwu/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-04-12T01:35:28Z","updated_at":"2016-04-13T02:07:02Z","closed_at":null,"body":"Hi,\r\n\r\nI am reading the model parallel lstm code. I found that in lstm_ptb.py, function load_data/replicate_data, the code loads the contents and splits them using whitespace. However, after that it reshape the data into a ndarray of size [(num_word)/batchsize, batchsize] using Fortan order.\r\n\r\nDoes it means that\r\n- sentences are just marked at its end using <eos> without spliting them?\r\n- words in a sentence is distributed in a column instead of a row? How do you read a sentence in the following code?\r\n- In the previous lstm, a batch is a collection of sentences. What does a batch mean here?"},"comment":{"url":"https://api.github.com/repos/dmlc/mxnet/issues/comments/209193611","html_url":"https://github.com/dmlc/mxnet/issues/1828#issuecomment-209193611","issue_url":"https://api.github.com/repos/dmlc/mxnet/issues/1828","id":209193611,"user":{"login":"pluskid","id":3542,"avatar_url":"https://avatars.githubusercontent.com/u/3542?v=3","gravatar_id":"","url":"https://api.github.com/users/pluskid","html_url":"https://github.com/pluskid","followers_url":"https://api.github.com/users/pluskid/followers","following_url":"https://api.github.com/users/pluskid/following{/other_user}","gists_url":"https://api.github.com/users/pluskid/gists{/gist_id}","starred_url":"https://api.github.com/users/pluskid/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pluskid/subscriptions","organizations_url":"https://api.github.com/users/pluskid/orgs","repos_url":"https://api.github.com/users/pluskid/repos","events_url":"https://api.github.com/users/pluskid/events{/privacy}","received_events_url":"https://api.github.com/users/pluskid/received_events","type":"User","site_admin":false},"created_at":"2016-04-13T02:07:02Z","updated_at":"2016-04-13T02:07:02Z","body":"In @tqchen 's particular LSTM model parallel example, it seems it is using a slightly unconventional way to represent the data. Each row is a time-step in a mini-batch. So each column is sentence (marked with `eos` for new sentences) -- checkout `set_rnn_inputs`.\r\n\r\nIn the other LSTM examples, especially the ones using high-level model API, since the data for each index is fed separately, it does not really matter originally it is in row order or column order. In a recent PR  #1833 we are refactoring this by having the convention that a mini-batch of sentence is of the shape `(batch-size, seq-len)`."}}