{"action":"created","issue":{"url":"https://api.github.com/repos/dmlc/mxnet/issues/3367","repository_url":"https://api.github.com/repos/dmlc/mxnet","labels_url":"https://api.github.com/repos/dmlc/mxnet/issues/3367/labels{/name}","comments_url":"https://api.github.com/repos/dmlc/mxnet/issues/3367/comments","events_url":"https://api.github.com/repos/dmlc/mxnet/issues/3367/events","html_url":"https://github.com/dmlc/mxnet/issues/3367","id":179040226,"number":3367,"title":"Implementing ImageNet in R","user":{"login":"hoaphumanoid","id":3491412,"avatar_url":"https://avatars.githubusercontent.com/u/3491412?v=3","gravatar_id":"","url":"https://api.github.com/users/hoaphumanoid","html_url":"https://github.com/hoaphumanoid","followers_url":"https://api.github.com/users/hoaphumanoid/followers","following_url":"https://api.github.com/users/hoaphumanoid/following{/other_user}","gists_url":"https://api.github.com/users/hoaphumanoid/gists{/gist_id}","starred_url":"https://api.github.com/users/hoaphumanoid/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hoaphumanoid/subscriptions","organizations_url":"https://api.github.com/users/hoaphumanoid/orgs","repos_url":"https://api.github.com/users/hoaphumanoid/repos","events_url":"https://api.github.com/users/hoaphumanoid/events{/privacy}","received_events_url":"https://api.github.com/users/hoaphumanoid/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/dmlc/mxnet/labels/R-Package","name":"R-Package","color":"006b75"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2016-09-24T17:01:12Z","updated_at":"2016-09-26T13:59:03Z","closed_at":null,"body":"I'm trying to implement Imagenet in R using [alexnet ](https://github.com/dmlc/mxnet/blob/master/example/image-classification/symbol_alexnet.R)architecture. I successfully made it work in python, with the exception of a couple of architectures that are [not working](https://github.com/dmlc/mxnet/issues/3359). However, alexnet is working in python.\r\n\r\nWhat I get right now is that the accuracy is not going up: `0.00100694444444444, 0.00112268518518518, 0.00107638888888889, 0.00110416666666666, 0.00107638888888889...`. It goes like this forever.\r\n\r\nI'm using 4 GPUs and a batch size of 576 (I tried different batch sizes). This is the code I have:\r\n\r\n```R\r\n# Train imagenet\r\nrequire(mxnet)\r\nrequire(argparse)\r\n\r\nget_iterator <- function(args) {\r\n  data.shape <- c(args$data_shape, args$data_shape, 3)\r\n  train = mx.io.ImageRecordIter(\r\n    path.imgrec     = file.path(args$data_dir, args$train_dataset),\r\n    batch.size      = args$batch_size,\r\n    data.shape      = data.shape,\r\n    mean.r          = 123.68,\r\n    mean.g          = 116.779,\r\n    mean.b          = 103.939,\r\n    rand.crop       = TRUE,\r\n    rand.mirror     = TRUE\r\n  )\r\n  \r\n  val = mx.io.ImageRecordIter(\r\n    path.imgrec     = file.path(args$data_dir, args$val_dataset),\r\n    batch.size      = args$batch_size,\r\n    data.shape      = data.shape,\r\n    mean.r          = 123.68,\r\n    mean.g          = 116.779,\r\n    mean.b          = 103.939,\r\n    rand.crop       = FALSE,\r\n    rand.mirror     = FALSE\r\n  )\r\n  ret = list(train=train, value=val)\r\n}\r\n\r\nparse_args <- function() {\r\n  parser <- ArgumentParser(description='train an image classifer on ImageNet')\r\n  parser$add_argument('--network', type='character', default='alexnet',\r\n                      choices = c('alexnet', 'vgg','resnet'),\r\n                      help = 'the cnn to use')\r\n  parser$add_argument('--data-dir', type='character', default='../../data',\r\n                      help='the input data directory')\r\n  parser$add_argument('--train-dataset', type='character', default=\"train.rec\",\r\n                      help='train dataset name')\r\n  parser$add_argument('--val-dataset', type='character', default=\"val.rec\",\r\n                      help=\"validation dataset name\")\r\n  parser$add_argument('--gpus', type='character',\r\n                      help='the gpus will be used, e.g \"0,1,2,3\"')\r\n  parser$add_argument('--batch-size', type='integer', default=128,\r\n                      help='the batch size')\r\n  parser$add_argument('--lr', type='double', default=.01,\r\n                      help='the initial learning rate')\r\n  parser$add_argument('--clip-gradient', type='double', default=5,\r\n                      help='clip min/max gradient to prevent extreme value')\r\n  parser$add_argument('--model-prefix', type='character',\r\n                      help='the prefix of the model to load/save')\r\n  parser$add_argument('--num-round', type='integer', default=10,\r\n                      help='the number of iterations over training data to train the model')\r\n  parser$add_argument('--kv-store', type='character', default='local',\r\n                      help='the kvstore type')\r\n  parser$add_argument('--num-examples', type='integer', default=1281167,\r\n                      help='the number of training examples')\r\n  parser$add_argument('--num-classes', type='integer', default=1000,\r\n                      help='the number of classes')\r\n  parser$add_argument('--data-shape', type='integer', default=224,\r\n                      help='set images shape')\r\n  parser$parse_args()\r\n}\r\nargs <- parse_args()\r\n\r\n# network\r\nif (args$network == 'alexnet') {\r\n  source(\"symbol_alexnet.R\")\r\n} else if (args$network == 'vgg'){\r\n  source(\"symbol_vgg.R\")\r\n} else {\r\n  source(\"symbol_resnet.R\")\r\n}\r\nnetwork <- get_symbol(args$num_classes)\r\n\r\n# data\r\ndata <- get_iterator(args)\r\ntrain <- data$train\r\nval <- data$value\r\n\r\n# devices\r\nif (is.null(args$gpus)) {\r\n  devs <- mx.cpu()  \r\n} else {\r\n  devs <- lapply(unlist(strsplit(args$gpus, \",\")), function(i) {\r\n    mx.gpu(as.integer(i))\r\n  })\r\n}\r\n\r\n# save model\r\nif (is.null(args$model_prefix)) {\r\n  checkpoint <- NULL\r\n} else {\r\n  checkpoint <- mx.callback.save.checkpoint(args$model_prefix)\r\n}\r\n\r\n# learning rate scheduler\r\nif (args$lr_factor < 1){\r\n  epoch_size <- args$num_examples/args$batch_size\r\n  #lr_scheduler <- mx.lr_scheduler.FactorScheduler(\r\n  FactorScheduler <- mxnet:::FactorScheduler\r\n  lr_scheduler <- FactorScheduler(\r\n    step = as.integer(max(epoch_size * args$lr_factor_epoch, 1)),\r\n    factor_val = args$lr_factor)\r\n} else{\r\n  lr_scheduler = NULL\r\n}\r\n\r\n# train\r\nmodel = mx.model.FeedForward.create(\r\n  X                  = train,\r\n  eval.data          = val,\r\n  ctx                = devs,\r\n  symbol             = network,\r\n  eval.metric        = mx.metric.accuracy,\r\n  num.round          = args$num_round,\r\n  learning.rate      = args$lr,\r\n  momentum           = 0.9,\r\n  wd                 = 0.00001,\r\n  kvstore            = args$kv_store,\r\n  array.batch.size   = args$batch_size,\r\n  clip_gradient      = args$clip_gradient,\r\n  lr_scheduler       = lr_scheduler,\r\n  epoch.end.callback = checkpoint,\r\n  initializer        = mx.init.Xavier(factor_type=\"in\", magnitude=2),\r\n  batch.end.callback = mx.callback.log.train.metric(50))\r\n```\r\nAny ideas?"},"comment":{"url":"https://api.github.com/repos/dmlc/mxnet/issues/comments/249577118","html_url":"https://github.com/dmlc/mxnet/issues/3367#issuecomment-249577118","issue_url":"https://api.github.com/repos/dmlc/mxnet/issues/3367","id":249577118,"user":{"login":"hoaphumanoid","id":3491412,"avatar_url":"https://avatars.githubusercontent.com/u/3491412?v=3","gravatar_id":"","url":"https://api.github.com/users/hoaphumanoid","html_url":"https://github.com/hoaphumanoid","followers_url":"https://api.github.com/users/hoaphumanoid/followers","following_url":"https://api.github.com/users/hoaphumanoid/following{/other_user}","gists_url":"https://api.github.com/users/hoaphumanoid/gists{/gist_id}","starred_url":"https://api.github.com/users/hoaphumanoid/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hoaphumanoid/subscriptions","organizations_url":"https://api.github.com/users/hoaphumanoid/orgs","repos_url":"https://api.github.com/users/hoaphumanoid/repos","events_url":"https://api.github.com/users/hoaphumanoid/events{/privacy}","received_events_url":"https://api.github.com/users/hoaphumanoid/received_events","type":"User","site_admin":false},"created_at":"2016-09-26T13:59:03Z","updated_at":"2016-09-26T13:59:03Z","body":"It looks that if you remove remove the num_parts = kv.num_workers and part_index = kv.rank in the image iterator in python, the training doesn't converge. \r\nIn python it's easy to access `num_workers` and `rank`, they have [getters](https://github.com/dmlc/mxnet/blob/5ff545f2345f9b607b81546a168665bd63d02d9f/python/mxnet/kvstore.py#L272). However, in R there is only a call to a [cpp function](https://github.com/dmlc/mxnet/blob/5ff545f2345f9b607b81546a168665bd63d02d9f/R-package/R/kvstore.R).\r\n\r\nDo you know @mli @thirdwing how I can access `num_workers` and `rank` in R?"}}
{"actor":{"display_login":"hoaphumanoid"}}