{"action":"created","comment":{"url":"https://api.github.com/repos/kaldi-asr/kaldi/pulls/comments/54849356","id":54849356,"diff_hunk":"@@ -0,0 +1,411 @@\n+#!/bin/bash\n+\n+# Copyright 2012-2015 Johns Hopkins University (Author: Daniel Povey).  \n+#           2015-2016 Vimal Manohar\n+# Apache 2.0.\n+\n+# This script is similar to steps/nnet3/get_egs.sh but used\n+# when getting general targets (not from alignment directory) for raw nnet \n+#\n+# This script, which will generally be called from other neural-net training\n+# scripts, extracts the training examples used to train the neural net (and also\n+# the validation examples used for diagnostics), and puts them in separate archives.\n+#\n+# This script dumps egs with several frames of labels, controlled by the\n+# frames_per_eg config variable (default: 8).  This takes many times less disk\n+# space because typically we have 4 to 7 frames of context on the left and\n+# right, and this ends up getting shared.  This is at the expense of slightly\n+# higher disk I/O while training.\n+\n+\n+# Begin configuration section.\n+cmd=run.pl\n+feat_type=raw       # set it to 'lda' to use LDA features.\n+target_type=sparse  # dense to have dense targets, \n+                    # sparse to have posteriors targets\n+num_targets=        # required for target-type=sparse with raw nnet\n+frames_per_eg=8   # number of frames of labels per example.  more->less disk space and\n+                  # less time preparing egs, but more I/O during training.\n+                  # note: the script may reduce this if reduce_frames_per_eg is true.\n+left_context=4    # amount of left-context per eg (i.e. extra frames of input features\n+                  # not present in the output supervision).\n+right_context=4   # amount of right-context per eg.\n+valid_left_context=   # amount of left_context for validation egs, typically used in\n+                      # recurrent architectures to ensure matched condition with\n+                      # training egs\n+valid_right_context=  # amount of right_context for validation egs\n+compress=true   # set this to false to disable compression (e.g. if you want to see whether\n+                # results are affected).\n+\n+reduce_frames_per_eg=true  # If true, this script may reduce the frames_per_eg\n+                           # if there is only one archive and even with the\n+                           # reduced frames_per_eg, the number of\n+                           # samples_per_iter that would result is less than or\n+                           # equal to the user-specified value.\n+num_utts_subset=300     # number of utterances in validation and training\n+                        # subsets used for shrinkage and diagnostics.\n+num_valid_frames_combine=0 # #valid frames for combination weights at the very end.\n+num_train_frames_combine=10000 # # train frames for the above.\n+num_frames_diagnostic=4000 # number of frames for \"compute_prob\" jobs\n+samples_per_iter=400000 # this is the target number of egs in each archive of egs\n+                        # (prior to merging egs).  We probably should have called\n+                        # it egs_per_iter. This is just a guideline; it will pick\n+                        # a number that divides the number of samples in the\n+                        # entire data.\n+\n+transform_dir=     \n+\n+stage=0\n+nj=6         # This should be set to the maximum number of jobs you are\n+             # comfortable to run in parallel; you can increase it if your disk\n+             # speed is greater and you have more machines.\n+online_ivector_dir=  # can be used if we are including speaker information as iVectors.\n+cmvn_opts=  # can be used for specifying CMVN options, if feature type is not lda (if lda,\n+            # it doesn't make sense to use different options than were used as input to the\n+            # LDA transform).  This is used to turn off CMVN in the online-nnet experiments.\n+\n+echo \"$0 $@\"  # Print the command line for logging\n+\n+if [ -f path.sh ]; then . ./path.sh; fi\n+. parse_options.sh || exit 1;\n+\n+if [ $# != 3 ]; then\n+  echo \"Usage: $0 [opts] <data> <targets-scp> <egs-dir>\"\n+  echo \" e.g.: $0 data/train data/train/snr_targets.scp exp/tri4_nnet/egs\"\n+  echo \"\"\n+  echo \"Main options (for others, see top of script file)\"\n+  echo \"  --config <config-file>                           # config file containing options\"\n+  echo \"  --nj <nj>                                        # The maximum number of jobs you want to run in\"\n+  echo \"                                                   # parallel (increase this only if you have good disk and\"\n+  echo \"                                                   # network speed).  default=6\"\n+  echo \"  --cmd (utils/run.pl;utils/queue.pl <queue opts>) # how to run jobs.\"\n+  echo \"  --samples-per-iter <#samples;400000>             # Target number of egs per archive (option is badly named)\"\n+  echo \"  --feat-type <lda|raw>                            # (raw is the default).  The feature type you want\"\n+  echo \"                                                   # to use as input to the neural net.\"\n+  echo \"  --frames-per-eg <frames;8>                       # number of frames per eg on disk\"\n+  echo \"  --left-context <width;4>                         # Number of frames on left side to append for feature input\"\n+  echo \"  --right-context <width;4>                        # Number of frames on right side to append for feature input\"\n+  echo \"  --num-frames-diagnostic <#frames;4000>           # Number of frames used in computing (train,valid) diagnostics\"\n+  echo \"  --num-valid-frames-combine <#frames;10000>       # Number of frames used in getting combination weights at the\"\n+  echo \"                                                   # very end.\"\n+  echo \"  --stage <stage|0>                                # Used to run a partially-completed training process from somewhere in\"\n+  echo \"                                                   # the middle.\"\n+\n+  exit 1;\n+fi\n+\n+data=$1\n+targets_scp=$2\n+dir=$3\n+\n+# Check some files.\n+[ ! -z \"$online_ivector_dir\" ] && \\\n+  extra_files=\"$online_ivector_dir/ivector_online.scp $online_ivector_dir/ivector_period\"\n+\n+for f in $data/feats.scp $targets_scp $extra_files; do\n+  [ ! -f $f ] && echo \"$0: no such file $f\" && exit 1;\n+done\n+\n+sdata=$data/split$nj\n+utils/split_data.sh $data $nj\n+\n+mkdir -p $dir/log $dir/info\n+\n+\n+# Get list of validation utterances.\n+awk '{print $1}' $data/utt2spk | utils/shuffle_list.pl | head -$num_utts_subset | sort \\\n+    > $dir/valid_uttlist || exit 1;\n+\n+if [ -f $data/utt2uniq ]; then  # this matters if you use data augmentation.\n+  echo \"File $data/utt2uniq exists, so augmenting valid_uttlist to\"\n+  echo \"include all perturbed versions of the same 'real' utterances.\"\n+  mv $dir/valid_uttlist $dir/valid_uttlist.tmp\n+  utils/utt2spk_to_spk2utt.pl $data/utt2uniq > $dir/uniq2utt\n+  cat $dir/valid_uttlist.tmp | utils/apply_map.pl $data/utt2uniq | \\\n+    sort | uniq | utils/apply_map.pl $dir/uniq2utt | \\\n+    awk '{for(n=1;n<=NF;n++) print $n;}' | sort  > $dir/valid_uttlist\n+  rm $dir/uniq2utt $dir/valid_uttlist.tmp\n+fi\n+\n+awk '{print $1}' $data/utt2spk | utils/filter_scp.pl --exclude $dir/valid_uttlist | \\\n+   utils/shuffle_list.pl | head -$num_utts_subset | sort > $dir/train_subset_uttlist || exit 1;\n+\n+if [ ! -z \"$transform_dir\" ] && [ -f $transform_dir/trans.1 ] && [ $feat_type != \"raw\" ]; then\n+  echo \"$0: using transforms from $transform_dir\"\n+  if [ $stage -le 0 ]; then\n+    $cmd $dir/log/copy_transforms.log \\\n+      copy-feats \"ark:cat $transform_dir/trans.* |\" \"ark,scp:$dir/trans.ark,$dir/trans.scp\"\n+  fi\n+fi\n+if [ -f $transform_dir/raw_trans.1 ] && [ $feat_type == \"raw\" ]; then\n+  echo \"$0: using raw transforms from $transform_dir\"\n+  if [ $stage -le 0 ]; then\n+    $cmd $dir/log/copy_transforms.log \\\n+      copy-feats \"ark:cat $transform_dir/raw_trans.* |\" \"ark,scp:$dir/trans.ark,$dir/trans.scp\"\n+  fi\n+fi\n+\n+\n+\n+## Set up features.\n+echo \"$0: feature type is $feat_type\"\n+\n+case $feat_type in\n+  raw) feats=\"ark,s,cs:utils/filter_scp.pl --exclude $dir/valid_uttlist $sdata/JOB/feats.scp | apply-cmvn $cmvn_opts --utt2spk=ark:$sdata/JOB/utt2spk scp:$sdata/JOB/cmvn.scp scp:- ark:- |\"\n+    valid_feats=\"ark,s,cs:utils/filter_scp.pl $dir/valid_uttlist $data/feats.scp | apply-cmvn $cmvn_opts --utt2spk=ark:$data/utt2spk scp:$data/cmvn.scp scp:- ark:- |\"\n+    train_subset_feats=\"ark,s,cs:utils/filter_scp.pl $dir/train_subset_uttlist $data/feats.scp | apply-cmvn $cmvn_opts --utt2spk=ark:$data/utt2spk scp:$data/cmvn.scp scp:- ark:- |\"\n+    echo $cmvn_opts >$dir/cmvn_opts # caution: the top-level nnet training script should copy this to its own dir now.\n+   ;;\n+  lda)\n+    splice_opts=`cat $transform_dir/splice_opts 2>/dev/null`\n+    # caution: the top-level nnet training script should copy these to its own dir now.\n+    cp $transform_dir/{splice_opts,cmvn_opts,final.mat} $dir || exit 1;\n+    [ ! -z \"$cmvn_opts\" ] && \\\n+       echo \"You cannot supply --cmvn-opts option if feature type is LDA.\" && exit 1;\n+    cmvn_opts=$(cat $dir/cmvn_opts)\n+    feats=\"ark,s,cs:utils/filter_scp.pl --exclude $dir/valid_uttlist $sdata/JOB/feats.scp | apply-cmvn $cmvn_opts --utt2spk=ark:$sdata/JOB/utt2spk scp:$sdata/JOB/cmvn.scp scp:- ark:- | splice-feats $splice_opts ark:- ark:- | transform-feats $dir/final.mat ark:- ark:- |\"\n+    valid_feats=\"ark,s,cs:utils/filter_scp.pl $dir/valid_uttlist $data/feats.scp | apply-cmvn $cmvn_opts --utt2spk=ark:$data/utt2spk scp:$data/cmvn.scp scp:- ark:- | splice-feats $splice_opts ark:- ark:- | transform-feats $dir/final.mat ark:- ark:- |\"\n+    train_subset_feats=\"ark,s,cs:utils/filter_scp.pl $dir/train_subset_uttlist $data/feats.scp | apply-cmvn $cmvn_opts --utt2spk=ark:$data/utt2spk scp:$data/cmvn.scp scp:- ark:- | splice-feats $splice_opts ark:- ark:- | transform-feats $dir/final.mat ark:- ark:- |\"\n+    ;;\n+  *) echo \"$0: invalid feature type --feat-type '$feat_type'\" && exit 1;\n+esac\n+\n+if [ -f $dir/trans.scp ]; then\n+  feats=\"$feats transform-feats --utt2spk=ark:$sdata/JOB/utt2spk scp:$dir/trans.scp ark:- ark:- |\"\n+  valid_feats=\"$valid_feats transform-feats --utt2spk=ark:$data/utt2spk scp:$dir/trans.scp ark:- ark:- |\"\n+  train_subset_feats=\"$train_subset_feats transform-feats --utt2spk=ark:$data/utt2spk scp:$dir/trans.scp ark:- ark:- |\"\n+fi\n+\n+if [ ! -z \"$online_ivector_dir\" ]; then\n+  ivector_dim=$(feat-to-dim scp:$online_ivector_dir/ivector_online.scp -) || exit 1;\n+  echo $ivector_dim > $dir/info/ivector_dim\n+  ivector_period=$(cat $online_ivector_dir/ivector_period) || exit 1;\n+\n+  ivector_opt=\"--ivectors='ark,s,cs:utils/filter_scp.pl $sdata/JOB/utt2spk $online_ivector_dir/ivector_online.scp | subsample-feats --n=-$ivector_period scp:- ark:- |'\"\n+  valid_ivector_opt=\"--ivectors='ark,s,cs:utils/filter_scp.pl $dir/valid_uttlist $online_ivector_dir/ivector_online.scp | subsample-feats --n=-$ivector_period scp:- ark:- |'\"\n+  train_subset_ivector_opt=\"--ivectors='ark,s,cs:utils/filter_scp.pl $dir/train_subset_uttlist $online_ivector_dir/ivector_online.scp | subsample-feats --n=-$ivector_period scp:- ark:- |'\"\n+else\n+  echo 0 >$dir/info/ivector_dim\n+fi\n+\n+if [ $stage -le 1 ]; then\n+  echo \"$0: working out number of frames of training data\"\n+  num_frames=$(steps/nnet2/get_num_frames.sh $data)\n+  echo $num_frames > $dir/info/num_frames\n+  echo \"$0: working out feature dim\"\n+  feats_one=\"$(echo $feats | sed s:JOB:1:g)\"\n+  feat_dim=$(feat-to-dim \"$feats_one\" -) || exit 1;\n+  echo $feat_dim > $dir/info/feat_dim\n+else\n+  num_frames=$(cat $dir/info/num_frames) || exit 1;\n+  feat_dim=$(cat $dir/info/feat_dim) || exit 1;\n+fi\n+\n+# the + 1 is to round up, not down... we assume it doesn't divide exactly.\n+num_archives=$[$num_frames/($frames_per_eg*$samples_per_iter)+1]\n+# (for small data)- while reduce_frames_per_eg == true and the number of\n+# archives is 1 and would still be 1 if we reduced frames_per_eg by 1, reduce it\n+# by 1.\n+reduced=false\n+while $reduce_frames_per_eg && [ $frames_per_eg -gt 1 ] && \\\n+  [ $[$num_frames/(($frames_per_eg-1)*$samples_per_iter)] -eq 0 ]; do\n+  frames_per_eg=$[$frames_per_eg-1]\n+  num_archives=1\n+  reduced=true\n+done\n+$reduced && echo \"$0: reduced frames_per_eg to $frames_per_eg because amount of data is small.\"\n+\n+# We may have to first create a smaller number of larger archives, with number\n+# $num_archives_intermediate, if $num_archives is more than the maximum number\n+# of open filehandles that the system allows per process (ulimit -n).\n+max_open_filehandles=$(ulimit -n) || exit 1\n+num_archives_intermediate=$num_archives\n+archives_multiple=1\n+while [ $[$num_archives_intermediate+4] -gt $max_open_filehandles ]; do\n+  archives_multiple=$[$archives_multiple+1]\n+  num_archives_intermediate=$[$num_archives/$archives_multiple+1];\n+done\n+# now make sure num_archives is an exact multiple of archives_multiple.\n+num_archives=$[$archives_multiple*$num_archives_intermediate]\n+\n+echo $num_archives >$dir/info/num_archives\n+echo $frames_per_eg >$dir/info/frames_per_eg\n+# Work out the number of egs per archive\n+egs_per_archive=$[$num_frames/($frames_per_eg*$num_archives)]\n+! [ $egs_per_archive -le $samples_per_iter ] && \\\n+  echo \"$0: script error: egs_per_archive=$egs_per_archive not <= samples_per_iter=$samples_per_iter\" \\\n+  && exit 1;\n+\n+echo $egs_per_archive > $dir/info/egs_per_archive\n+\n+echo \"$0: creating $num_archives archives, each with $egs_per_archive egs, with\"\n+echo \"$0:   $frames_per_eg labels per example, and (left,right) context = ($left_context,$right_context)\"\n+\n+\n+\n+if [ -e $dir/storage ]; then\n+  # Make soft links to storage directories, if distributing this way..  See\n+  # utils/create_split_dir.pl.\n+  echo \"$0: creating data links\"\n+  utils/create_data_link.pl $(for x in $(seq $num_archives); do echo $dir/egs.$x.ark; done)\n+  for x in $(seq $num_archives_intermediate); do\n+    utils/create_data_link.pl $(for y in $(seq $nj); do echo $dir/egs_orig.$y.$x.ark; done)\n+  done\n+fi\n+\n+egs_opts=\"--left-context=$left_context --right-context=$right_context --compress=$compress\"\n+\n+[ -z $valid_left_context ] &&  valid_left_context=$left_context;\n+[ -z $valid_right_context ] &&  valid_right_context=$right_context;\n+valid_egs_opts=\"--left-context=$valid_left_context --right-context=$valid_right_context --compress=$compress\"\n+\n+echo $left_context > $dir/info/left_context\n+echo $right_context > $dir/info/right_context\n+\n+for n in `seq $nj`; do\n+  utils/filter_scp.pl $sdata/$n/utt2spk $targets_scp > $dir/targets.$n.scp\n+done\n+\n+targets_scp_split=$dir/targets.JOB.scp\n+\n+if [ $target_type == \"dense\" ]; then\n+  num_targets=$(feat-to-dim \"scp:$targets_scp\" - 2>/dev/null) || exit 1\n+fi\n+\n+if [ -z \"$num_targets\" ]; then\n+  echo \"$0: num-targets is not set\" \n+  exit 1\n+fi\n+\n+case $target_type in\n+  \"dense\") \n+    get_egs_program=\"nnet3-get-egs-dense-targets --num-targets=$num_targets\"\n+\n+    targets=\"ark:utils/filter_scp.pl --exclude $dir/valid_uttlist $targets_scp_split | copy-feats scp:- ark:- |\"\n+    valid_targets=\"ark:utils/filter_scp.pl $dir/valid_uttlist $targets_scp | copy-feats scp:- ark:- |\"\n+    train_subset_targets=\"ark:utils/filter_scp.pl $dir/train_subset_uttlist $targets_scp | copy-feats scp:- ark:- |\"\n+    ;;\n+  \"sparse\")\n+    get_egs_program=\"nnet3-get-egs --num-pdfs=$num_targets\"\n+    targets=\"ark:utils/filter_scp.pl --exclude $dir/valid_uttlist $targets_scp_split | ali-to-post scp:- ark:- |\"\n+    valid_targets=\"ark:utils/filter_scp.pl $dir/valid_uttlist $targets_scp | ali-to-post scp:- ark:- |\" \n+    train_subset_targets=\"ark:utils/filter_scp.pl $dir/train_subset_uttlist $targets_scp | ali-to-post scp:- ark:- |\"\n+    ;;\n+  default)\n+    echo \"$0: Unknown --target-type $target_type. Choices are dense and sparse\"\n+    exit 1\n+esac\n+\n+if [ $stage -le 3 ]; then\n+  echo \"$0: Getting validation and training subset examples.\"\n+  rm -f $dir/.error 2>/dev/null\n+  echo \"$0: ... extracting validation and training-subset alignments.\"\n+\n+  $cmd $dir/log/create_valid_subset.log \\\n+    $get_egs_program \\\n+    $valid_ivector_opt $valid_egs_opts \"$valid_feats\" \\\n+    \"$valid_targets\" \\\n+    \"ark:$dir/valid_all.egs\" || touch $dir/.error &\n+  $cmd $dir/log/create_train_subset.log \\\n+    $get_egs_program \\\n+    $train_subset_ivector_opt $valid_egs_opts \"$train_subset_feats\" \\\n+    \"$train_subset_targets\" \\\n+    \"ark:$dir/train_subset_all.egs\" || touch $dir/.error &\n+  wait;\n+  [ -f $dir/.error ] && echo \"Error detected while creating train/valid egs\" && exit 1\n+  echo \"... Getting subsets of validation examples for diagnostics and combination.\"\n+  $cmd $dir/log/create_valid_subset_combine.log \\\n+    nnet3-subset-egs --n=$num_valid_frames_combine ark:$dir/valid_all.egs \\\n+    ark:$dir/valid_combine.egs || touch $dir/.error &\n+  $cmd $dir/log/create_valid_subset_diagnostic.log \\\n+    nnet3-subset-egs --n=$num_frames_diagnostic ark:$dir/valid_all.egs \\\n+    ark:$dir/valid_diagnostic.egs || touch $dir/.error &\n+\n+  $cmd $dir/log/create_train_subset_combine.log \\\n+    nnet3-subset-egs --n=$num_train_frames_combine ark:$dir/train_subset_all.egs \\\n+    ark:$dir/train_combine.egs || touch $dir/.error &\n+  $cmd $dir/log/create_train_subset_diagnostic.log \\\n+    nnet3-subset-egs --n=$num_frames_diagnostic ark:$dir/train_subset_all.egs \\\n+    ark:$dir/train_diagnostic.egs || touch $dir/.error &\n+  wait\n+  sleep 5  # wait for file system to sync.\n+  cat $dir/valid_combine.egs $dir/train_combine.egs > $dir/combine.egs\n+\n+  for f in $dir/{combine,train_diagnostic,valid_diagnostic}.egs; do\n+    [ ! -s $f ] && echo \"No examples in file $f\" && exit 1;\n+  done\n+  rm -f $dir/valid_all.egs $dir/train_subset_all.egs $dir/{train,valid}_combine.egs\n+fi\n+\n+if [ $stage -le 4 ]; then\n+  # create egs_orig.*.*.ark; the first index goes to $nj,\n+  # the second to $num_archives_intermediate.\n+\n+  egs_list=\n+  for n in $(seq $num_archives_intermediate); do\n+    egs_list=\"$egs_list ark:$dir/egs_orig.JOB.$n.ark\"\n+  done\n+  echo \"$0: Generating training examples on disk\"\n+  # The examples will go round-robin to egs_list.\n+  $cmd JOB=1:$nj $dir/log/get_egs.JOB.log \\\n+    $get_egs_program \\\n+    $ivector_opt $egs_opts --num-frames=$frames_per_eg \"$feats\" \"$targets\" \\\n+    ark:- \\| \\\n+    nnet3-copy-egs --random=true --srand=JOB ark:- $egs_list || exit 1;\n+fi\n+\n+if [ $stage -le 5 ]; then\n+  echo \"$0: recombining and shuffling order of archives on disk\"\n+  # combine all the \"egs_orig.*.JOB.scp\" (over the $nj splits of the data) and\n+  # shuffle the order, writing to the egs.JOB.ark\n+\n+  # the input is a concatenation over the input jobs.\n+  egs_list=\n+  for n in $(seq $nj); do\n+    egs_list=\"$egs_list $dir/egs_orig.$n.JOB.ark\"\n+  done\n+\n+  if [ $archives_multiple == 1 ]; then # normal case.\n+    $cmd --max-jobs-run $nj JOB=1:$num_archives_intermediate $dir/log/shuffle.JOB.log \\\n+      nnet3-shuffle-egs --srand=JOB \"ark:cat $egs_list|\" ark:$dir/egs.JOB.ark  || exit 1;\n+  else\n+    # we need to shuffle the 'intermediate archives' and then split into the\n+    # final archives.  we create soft links to manage this splitting, because\n+    # otherwise managing the output names is quite difficult (and we don't want\n+    # to submit separate queue jobs for each intermediate archive, because then\n+    # the --max-jobs-run option is hard to enforce).\n+    output_archives=\"$(for y in $(seq $archives_multiple); do echo ark:$dir/egs.JOB.$y.ark; done)\"\n+    for x in $(seq $num_archives_intermediate); do\n+      for y in $(seq $archives_multiple); do\n+        archive_index=$[($x-1)*$archives_multiple+$y]\n+        # egs.intermediate_archive.{1,2,...}.ark will point to egs.archive.ark\n+        ln -sf egs.$archive_index.ark $dir/egs.$x.$y.ark || exit 1\n+      done\n+    done\n+    $cmd --max-jobs-run $nj JOB=1:$num_archives_intermediate $dir/log/shuffle.JOB.log \\\n+      nnet3-shuffle-egs --srand=JOB \"ark:cat $egs_list|\" ark:- \\| \\\n+      nnet3-copy-egs ark:- $output_archives || exit 1;\n+  fi\n+\n+fi\n+\n+if [ $stage -le 6 ]; then\n+  echo \"$0: removing temporary archives\"\n+  for x in $(seq $nj); do\n+    for y in $(seq $num_archives_intermediate); do\n+      file=$dir/egs_orig.$x.$y.ark\n+      [ -L $file ] && rm $(readlink -f $file)\n+      rm $file\n+    done\n+  done\n+  if [ $archives_multiple -gt 1 ]; then\n+    # there are some extra soft links that we should delete.\n+    for f in $dir/egs.*.*.ark; do rm $f; done\n+  fi\n+  echo \"$0: removing temporary alignments and transforms\"","path":"egs/wsj/s5/steps/nnet3/get_egs_targets.sh","position":405,"original_position":405,"commit_id":"2cfc989789b7e77d85a03aa1fbb99bbdd6d43590","original_commit_id":"2cfc989789b7e77d85a03aa1fbb99bbdd6d43590","user":{"login":"vijayaditya","id":3895456,"avatar_url":"https://avatars.githubusercontent.com/u/3895456?v=3","gravatar_id":"","url":"https://api.github.com/users/vijayaditya","html_url":"https://github.com/vijayaditya","followers_url":"https://api.github.com/users/vijayaditya/followers","following_url":"https://api.github.com/users/vijayaditya/following{/other_user}","gists_url":"https://api.github.com/users/vijayaditya/gists{/gist_id}","starred_url":"https://api.github.com/users/vijayaditya/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vijayaditya/subscriptions","organizations_url":"https://api.github.com/users/vijayaditya/orgs","repos_url":"https://api.github.com/users/vijayaditya/repos","events_url":"https://api.github.com/users/vijayaditya/events{/privacy}","received_events_url":"https://api.github.com/users/vijayaditya/received_events","type":"User","site_admin":false},"body":"update the message","created_at":"2016-03-03T08:37:28Z","updated_at":"2016-03-03T08:37:28Z","html_url":"https://github.com/kaldi-asr/kaldi/pull/539#discussion_r54849356","pull_request_url":"https://api.github.com/repos/kaldi-asr/kaldi/pulls/539","_links":{"self":{"href":"https://api.github.com/repos/kaldi-asr/kaldi/pulls/comments/54849356"},"html":{"href":"https://github.com/kaldi-asr/kaldi/pull/539#discussion_r54849356"},"pull_request":{"href":"https://api.github.com/repos/kaldi-asr/kaldi/pulls/539"}}},"pull_request":{"url":"https://api.github.com/repos/kaldi-asr/kaldi/pulls/539","id":61360336,"html_url":"https://github.com/kaldi-asr/kaldi/pull/539","diff_url":"https://github.com/kaldi-asr/kaldi/pull/539.diff","patch_url":"https://github.com/kaldi-asr/kaldi/pull/539.patch","issue_url":"https://api.github.com/repos/kaldi-asr/kaldi/issues/539","number":539,"state":"open","locked":false,"title":"Raw nnet3 training","user":{"login":"vimalmanohar","id":4199255,"avatar_url":"https://avatars.githubusercontent.com/u/4199255?v=3","gravatar_id":"","url":"https://api.github.com/users/vimalmanohar","html_url":"https://github.com/vimalmanohar","followers_url":"https://api.github.com/users/vimalmanohar/followers","following_url":"https://api.github.com/users/vimalmanohar/following{/other_user}","gists_url":"https://api.github.com/users/vimalmanohar/gists{/gist_id}","starred_url":"https://api.github.com/users/vimalmanohar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vimalmanohar/subscriptions","organizations_url":"https://api.github.com/users/vimalmanohar/orgs","repos_url":"https://api.github.com/users/vimalmanohar/repos","events_url":"https://api.github.com/users/vimalmanohar/events{/privacy}","received_events_url":"https://api.github.com/users/vimalmanohar/received_events","type":"User","site_admin":false},"body":"Scripts for raw nnet3 training supporting quadratic objective function and dense targets.\r\nLet me know if this looks ok. \r\n@vijayaditya @danpovey \r\n","created_at":"2016-03-02T07:17:55Z","updated_at":"2016-03-03T08:37:28Z","closed_at":null,"merged_at":null,"merge_commit_sha":"eb273d39aa23d8242db1403d6657318fbc5946d8","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/kaldi-asr/kaldi/pulls/539/commits","review_comments_url":"https://api.github.com/repos/kaldi-asr/kaldi/pulls/539/comments","review_comment_url":"https://api.github.com/repos/kaldi-asr/kaldi/pulls/comments{/number}","comments_url":"https://api.github.com/repos/kaldi-asr/kaldi/issues/539/comments","statuses_url":"https://api.github.com/repos/kaldi-asr/kaldi/statuses/2cfc989789b7e77d85a03aa1fbb99bbdd6d43590","head":{"label":"vimalmanohar:raw_nnet","ref":"raw_nnet","sha":"2cfc989789b7e77d85a03aa1fbb99bbdd6d43590","user":{"login":"vimalmanohar","id":4199255,"avatar_url":"https://avatars.githubusercontent.com/u/4199255?v=3","gravatar_id":"","url":"https://api.github.com/users/vimalmanohar","html_url":"https://github.com/vimalmanohar","followers_url":"https://api.github.com/users/vimalmanohar/followers","following_url":"https://api.github.com/users/vimalmanohar/following{/other_user}","gists_url":"https://api.github.com/users/vimalmanohar/gists{/gist_id}","starred_url":"https://api.github.com/users/vimalmanohar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vimalmanohar/subscriptions","organizations_url":"https://api.github.com/users/vimalmanohar/orgs","repos_url":"https://api.github.com/users/vimalmanohar/repos","events_url":"https://api.github.com/users/vimalmanohar/events{/privacy}","received_events_url":"https://api.github.com/users/vimalmanohar/received_events","type":"User","site_admin":false},"repo":{"id":39757201,"name":"kaldi","full_name":"vimalmanohar/kaldi","owner":{"login":"vimalmanohar","id":4199255,"avatar_url":"https://avatars.githubusercontent.com/u/4199255?v=3","gravatar_id":"","url":"https://api.github.com/users/vimalmanohar","html_url":"https://github.com/vimalmanohar","followers_url":"https://api.github.com/users/vimalmanohar/followers","following_url":"https://api.github.com/users/vimalmanohar/following{/other_user}","gists_url":"https://api.github.com/users/vimalmanohar/gists{/gist_id}","starred_url":"https://api.github.com/users/vimalmanohar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vimalmanohar/subscriptions","organizations_url":"https://api.github.com/users/vimalmanohar/orgs","repos_url":"https://api.github.com/users/vimalmanohar/repos","events_url":"https://api.github.com/users/vimalmanohar/events{/privacy}","received_events_url":"https://api.github.com/users/vimalmanohar/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/vimalmanohar/kaldi","description":"This is now the official location of the Kaldi project.","fork":true,"url":"https://api.github.com/repos/vimalmanohar/kaldi","forks_url":"https://api.github.com/repos/vimalmanohar/kaldi/forks","keys_url":"https://api.github.com/repos/vimalmanohar/kaldi/keys{/key_id}","collaborators_url":"https://api.github.com/repos/vimalmanohar/kaldi/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/vimalmanohar/kaldi/teams","hooks_url":"https://api.github.com/repos/vimalmanohar/kaldi/hooks","issue_events_url":"https://api.github.com/repos/vimalmanohar/kaldi/issues/events{/number}","events_url":"https://api.github.com/repos/vimalmanohar/kaldi/events","assignees_url":"https://api.github.com/repos/vimalmanohar/kaldi/assignees{/user}","branches_url":"https://api.github.com/repos/vimalmanohar/kaldi/branches{/branch}","tags_url":"https://api.github.com/repos/vimalmanohar/kaldi/tags","blobs_url":"https://api.github.com/repos/vimalmanohar/kaldi/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/vimalmanohar/kaldi/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/vimalmanohar/kaldi/git/refs{/sha}","trees_url":"https://api.github.com/repos/vimalmanohar/kaldi/git/trees{/sha}","statuses_url":"https://api.github.com/repos/vimalmanohar/kaldi/statuses/{sha}","languages_url":"https://api.github.com/repos/vimalmanohar/kaldi/languages","stargazers_url":"https://api.github.com/repos/vimalmanohar/kaldi/stargazers","contributors_url":"https://api.github.com/repos/vimalmanohar/kaldi/contributors","subscribers_url":"https://api.github.com/repos/vimalmanohar/kaldi/subscribers","subscription_url":"https://api.github.com/repos/vimalmanohar/kaldi/subscription","commits_url":"https://api.github.com/repos/vimalmanohar/kaldi/commits{/sha}","git_commits_url":"https://api.github.com/repos/vimalmanohar/kaldi/git/commits{/sha}","comments_url":"https://api.github.com/repos/vimalmanohar/kaldi/comments{/number}","issue_comment_url":"https://api.github.com/repos/vimalmanohar/kaldi/issues/comments{/number}","contents_url":"https://api.github.com/repos/vimalmanohar/kaldi/contents/{+path}","compare_url":"https://api.github.com/repos/vimalmanohar/kaldi/compare/{base}...{head}","merges_url":"https://api.github.com/repos/vimalmanohar/kaldi/merges","archive_url":"https://api.github.com/repos/vimalmanohar/kaldi/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/vimalmanohar/kaldi/downloads","issues_url":"https://api.github.com/repos/vimalmanohar/kaldi/issues{/number}","pulls_url":"https://api.github.com/repos/vimalmanohar/kaldi/pulls{/number}","milestones_url":"https://api.github.com/repos/vimalmanohar/kaldi/milestones{/number}","notifications_url":"https://api.github.com/repos/vimalmanohar/kaldi/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/vimalmanohar/kaldi/labels{/name}","releases_url":"https://api.github.com/repos/vimalmanohar/kaldi/releases{/id}","deployments_url":"https://api.github.com/repos/vimalmanohar/kaldi/deployments","created_at":"2015-07-27T05:46:16Z","updated_at":"2016-02-03T06:27:42Z","pushed_at":"2016-03-03T05:40:17Z","git_url":"git://github.com/vimalmanohar/kaldi.git","ssh_url":"git@github.com:vimalmanohar/kaldi.git","clone_url":"https://github.com/vimalmanohar/kaldi.git","svn_url":"https://github.com/vimalmanohar/kaldi","homepage":"http://kaldi-asr.org","size":107073,"stargazers_count":4,"watchers_count":4,"language":"C++","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":3,"mirror_url":null,"open_issues_count":0,"forks":3,"open_issues":0,"watchers":4,"default_branch":"master"}},"base":{"label":"kaldi-asr:master","ref":"master","sha":"775fe3fe7f06ce4ddcdcf1c34d77c17e364eb3ef","user":{"login":"kaldi-asr","id":11650082,"avatar_url":"https://avatars.githubusercontent.com/u/11650082?v=3","gravatar_id":"","url":"https://api.github.com/users/kaldi-asr","html_url":"https://github.com/kaldi-asr","followers_url":"https://api.github.com/users/kaldi-asr/followers","following_url":"https://api.github.com/users/kaldi-asr/following{/other_user}","gists_url":"https://api.github.com/users/kaldi-asr/gists{/gist_id}","starred_url":"https://api.github.com/users/kaldi-asr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kaldi-asr/subscriptions","organizations_url":"https://api.github.com/users/kaldi-asr/orgs","repos_url":"https://api.github.com/users/kaldi-asr/repos","events_url":"https://api.github.com/users/kaldi-asr/events{/privacy}","received_events_url":"https://api.github.com/users/kaldi-asr/received_events","type":"Organization","site_admin":false},"repo":{"id":34275045,"name":"kaldi","full_name":"kaldi-asr/kaldi","owner":{"login":"kaldi-asr","id":11650082,"avatar_url":"https://avatars.githubusercontent.com/u/11650082?v=3","gravatar_id":"","url":"https://api.github.com/users/kaldi-asr","html_url":"https://github.com/kaldi-asr","followers_url":"https://api.github.com/users/kaldi-asr/followers","following_url":"https://api.github.com/users/kaldi-asr/following{/other_user}","gists_url":"https://api.github.com/users/kaldi-asr/gists{/gist_id}","starred_url":"https://api.github.com/users/kaldi-asr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kaldi-asr/subscriptions","organizations_url":"https://api.github.com/users/kaldi-asr/orgs","repos_url":"https://api.github.com/users/kaldi-asr/repos","events_url":"https://api.github.com/users/kaldi-asr/events{/privacy}","received_events_url":"https://api.github.com/users/kaldi-asr/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/kaldi-asr/kaldi","description":"This is now the official location of the Kaldi project.","fork":false,"url":"https://api.github.com/repos/kaldi-asr/kaldi","forks_url":"https://api.github.com/repos/kaldi-asr/kaldi/forks","keys_url":"https://api.github.com/repos/kaldi-asr/kaldi/keys{/key_id}","collaborators_url":"https://api.github.com/repos/kaldi-asr/kaldi/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/kaldi-asr/kaldi/teams","hooks_url":"https://api.github.com/repos/kaldi-asr/kaldi/hooks","issue_events_url":"https://api.github.com/repos/kaldi-asr/kaldi/issues/events{/number}","events_url":"https://api.github.com/repos/kaldi-asr/kaldi/events","assignees_url":"https://api.github.com/repos/kaldi-asr/kaldi/assignees{/user}","branches_url":"https://api.github.com/repos/kaldi-asr/kaldi/branches{/branch}","tags_url":"https://api.github.com/repos/kaldi-asr/kaldi/tags","blobs_url":"https://api.github.com/repos/kaldi-asr/kaldi/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/kaldi-asr/kaldi/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/kaldi-asr/kaldi/git/refs{/sha}","trees_url":"https://api.github.com/repos/kaldi-asr/kaldi/git/trees{/sha}","statuses_url":"https://api.github.com/repos/kaldi-asr/kaldi/statuses/{sha}","languages_url":"https://api.github.com/repos/kaldi-asr/kaldi/languages","stargazers_url":"https://api.github.com/repos/kaldi-asr/kaldi/stargazers","contributors_url":"https://api.github.com/repos/kaldi-asr/kaldi/contributors","subscribers_url":"https://api.github.com/repos/kaldi-asr/kaldi/subscribers","subscription_url":"https://api.github.com/repos/kaldi-asr/kaldi/subscription","commits_url":"https://api.github.com/repos/kaldi-asr/kaldi/commits{/sha}","git_commits_url":"https://api.github.com/repos/kaldi-asr/kaldi/git/commits{/sha}","comments_url":"https://api.github.com/repos/kaldi-asr/kaldi/comments{/number}","issue_comment_url":"https://api.github.com/repos/kaldi-asr/kaldi/issues/comments{/number}","contents_url":"https://api.github.com/repos/kaldi-asr/kaldi/contents/{+path}","compare_url":"https://api.github.com/repos/kaldi-asr/kaldi/compare/{base}...{head}","merges_url":"https://api.github.com/repos/kaldi-asr/kaldi/merges","archive_url":"https://api.github.com/repos/kaldi-asr/kaldi/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/kaldi-asr/kaldi/downloads","issues_url":"https://api.github.com/repos/kaldi-asr/kaldi/issues{/number}","pulls_url":"https://api.github.com/repos/kaldi-asr/kaldi/pulls{/number}","milestones_url":"https://api.github.com/repos/kaldi-asr/kaldi/milestones{/number}","notifications_url":"https://api.github.com/repos/kaldi-asr/kaldi/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/kaldi-asr/kaldi/labels{/name}","releases_url":"https://api.github.com/repos/kaldi-asr/kaldi/releases{/id}","deployments_url":"https://api.github.com/repos/kaldi-asr/kaldi/deployments","created_at":"2015-04-20T17:23:16Z","updated_at":"2016-03-03T08:16:35Z","pushed_at":"2016-03-03T05:40:18Z","git_url":"git://github.com/kaldi-asr/kaldi.git","ssh_url":"git@github.com:kaldi-asr/kaldi.git","clone_url":"https://github.com/kaldi-asr/kaldi.git","svn_url":"https://github.com/kaldi-asr/kaldi","homepage":"http://kaldi-asr.org","size":115276,"stargazers_count":470,"watchers_count":470,"language":"C++","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":293,"mirror_url":null,"open_issues_count":33,"forks":293,"open_issues":33,"watchers":470,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/kaldi-asr/kaldi/pulls/539"},"html":{"href":"https://github.com/kaldi-asr/kaldi/pull/539"},"issue":{"href":"https://api.github.com/repos/kaldi-asr/kaldi/issues/539"},"comments":{"href":"https://api.github.com/repos/kaldi-asr/kaldi/issues/539/comments"},"review_comments":{"href":"https://api.github.com/repos/kaldi-asr/kaldi/pulls/539/comments"},"review_comment":{"href":"https://api.github.com/repos/kaldi-asr/kaldi/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/kaldi-asr/kaldi/pulls/539/commits"},"statuses":{"href":"https://api.github.com/repos/kaldi-asr/kaldi/statuses/2cfc989789b7e77d85a03aa1fbb99bbdd6d43590"}}}}