title: ffmpeg and nvidia acceleration, need better results
body: <p>I've been trying to figure out the best way to use the nvenc hardware acceleration in ffmpeg to convert some older, larger avi files to MP4.  My issue is the default x264 settings generate a good file size with settings that don't seem to replicate when you use the h264_nvenc codec.</p>
  <p>First we have the <strong>original AVI</strong>...</p>
  <pre><code>General
  Complete name                            : input.avi
  Format                                   : AVI
  Format/Info                              : Audio Video Interleave
  Format profile                           : OpenDML
  File size                                : 1.55 GiB
  Duration                                 : 2 h 14 min
  Overall bit rate                         : 1 643 kb/s
  Writing application                      : Lavf52.91.0
  
  Video
  ID                                       : 0
  Format                                   : MPEG-4 Visual
  Format profile                           : Simple@L1
  Format settings, BVOP                    : No
  Format settings, QPel                    : No
  Format settings, GMC                     : No warppoints
  Format settings, Matrix                  : Default (H.263)
  Codec ID                                 : FMP4
  Duration                                 : 2 h 14 min
  Bit rate                                 : 1 500 kb/s
  Width                                    : 640 pixels
  Height                                   : 480 pixels
  Display aspect ratio                     : 4:3
  Frame rate                               : 29.970 (30000/1001) FPS
  Color space                              : YUV
  Chroma subsampling                       : 4:2:0
  Bit depth                                : 8 bits
  Scan type                                : Progressive
  Compression mode                         : Lossy
  Bits/(Pixel*Frame)                       : 0.163
  Stream size                              : 1.41 GiB (91%)
  Writing library                          : Lavc52.101.0
  
  Audio
  ID                                       : 1
  Format                                   : MPEG Audio
  Format version                           : Version 1
  Format profile                           : Layer 3
  Format settings                          : Joint stereo / MS Stereo
  Codec ID                                 : 55
  Codec ID/Hint                            : MP3
  Duration                                 : 2 h 14 min
  Bit rate mode                            : Constant
  Bit rate                                 : 128 kb/s
  Channel(s)                               : 2 channels
  Sampling rate                            : 44.1 kHz
  Compression mode                         : Lossy
  Stream size                              : 124 MiB (8%)
  Alignment                                : Aligned on interleaves
  Interleave, duration                     : 26  ms (0.78 video frame)
  </code></pre>
  <p>If we use the <strong>default x264</strong> settings and no hardware acceleration for encoding</p>
  <pre><code>ffmpeg.exe -i .\input.avi  -c:v libx264 -c:a aac &quot;output x264.mp4&quot;
  </code></pre>
  <pre><code>General
  Complete name                            : output x264.mp4
  Format                                   : MPEG-4
  Format profile                           : Base Media
  Codec ID                                 : isom (isom/iso2/avc1/mp41)
  File size                                : 886 MiB
  Duration                                 : 2 h 14 min
  Overall bit rate                         : 918 kb/s
  Writing application                      : Lavf59.27.100
  
  Video
  ID                                       : 1
  Format                                   : AVC
  Format/Info                              : Advanced Video Codec
  Format profile                           : High@L3
  Format settings                          : CABAC / 4 Ref Frames
  Format settings, CABAC                   : Yes
  Format settings, Reference frames        : 4 frames
  Codec ID                                 : avc1
  Codec ID/Info                            : Advanced Video Coding
  Duration                                 : 2 h 14 min
  Bit rate                                 : 780 kb/s
  Width                                    : 640 pixels
  Height                                   : 480 pixels
  Display aspect ratio                     : 4:3
  Frame rate mode                          : Constant
  Frame rate                               : 29.970 (30000/1001) FPS
  Color space                              : YUV
  Chroma subsampling                       : 4:2:0
  Bit depth                                : 8 bits
  Scan type                                : Progressive
  Bits/(Pixel*Frame)                       : 0.085
  Stream size                              : 753 MiB (85%)
  Writing library                          : x264 core 164 r3095 baee400
  Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
  Codec configuration box                  : avcC
  
  Audio
  ID                                       : 2
  Format                                   : AAC LC
  Format/Info                              : Advanced Audio Codec Low Complexity
  Codec ID                                 : mp4a-40-2
  Duration                                 : 2 h 14 min
  Source duration                          : 2 h 14 min
  Source_Duration_LastFrame                : -17 ms
  Bit rate mode                            : Constant
  Bit rate                                 : 132 kb/s
  Channel(s)                               : 2 channels
  Channel layout                           : L R
  Sampling rate                            : 44.1 kHz
  Frame rate                               : 43.066 FPS (1024 SPF)
  Compression mode                         : Lossy
  Stream size                              : 126 MiB (14%)
  Source stream size                       : 126 MiB (14%)
  Default                                  : Yes
  Alternate group                          : 1
  </code></pre>
  <p>Then we try with <strong>default h264_nvenc</strong> codec and the file is larger than the original, but also noticing there are different default encoding settings.</p>
  <pre><code>ffmpeg.exe -hwaccel auto -i .\input.avi  -c:v h264_nvenc -c:a aac &quot;output x264 nvenc.mp4&quot;
  </code></pre>
  <pre><code>General
  Complete name                            : output x264 nvenc.mp4
  Format                                   : MPEG-4
  Format profile                           : Base Media
  Codec ID                                 : isom (isom/iso2/avc1/mp41)
  File size                                : 2.01 GiB
  Duration                                 : 2 h 14 min
  Overall bit rate mode                    : Variable
  Overall bit rate                         : 2 134 kb/s
  Writing application                      : Lavf58.29.100
  
  Video
  ID                                       : 1
  Format                                   : AVC
  Format/Info                              : Advanced Video Codec
  Format profile                           : High@L3
  Format settings                          : CABAC / 1 Ref Frames
  Format settings, CABAC                   : Yes
  Format settings, Reference frames        : 1 frame
  Codec ID                                 : avc1
  Codec ID/Info                            : Advanced Video Coding
  Duration                                 : 2 h 14 min
  Bit rate mode                            : Variable
  Bit rate                                 : 1 999 kb/s
  Maximum bit rate                         : 6 000 kb/s
  Width                                    : 640 pixels
  Height                                   : 480 pixels
  Display aspect ratio                     : 4:3
  Frame rate mode                          : Constant
  Frame rate                               : 29.970 (30000/1001) FPS
  Color space                              : YUV
  Chroma subsampling                       : 4:2:0
  Bit depth                                : 8 bits
  Scan type                                : Progressive
  Bits/(Pixel*Frame)                       : 0.217
  Stream size                              : 1.88 GiB (94%)
  Codec configuration box                  : avcC
  
  Audio
  ID                                       : 2
  Format                                   : AAC LC
  Format/Info                              : Advanced Audio Codec Low Complexity
  Codec ID                                 : mp4a-40-2
  Duration                                 : 2 h 14 min
  Duration_LastFrame                       : -17 ms
  Bit rate mode                            : Constant
  Bit rate                                 : 129 kb/s
  Channel(s)                               : 2 channels
  Channel layout                           : L R
  Sampling rate                            : 44.1 kHz
  Frame rate                               : 43.066 FPS (1024 SPF)
  Compression mode                         : Lossy
  Stream size                              : 124 MiB (6%)
  Default                                  : Yes
  Alternate group                          : 1
  </code></pre>
  <p>So the question is how can I get the same results (or better) while using the acceleration from nvidia codecs?  My GPU can decode this file around 600 FPS, where as the CPU on standard x264 is under 100 FPS.</p>