title: How should I use g++'s -finput-charset compiler option correctly in order to compile a non-UTF-8 source file?
body: <p>I'm trying to compile a UTF-16BE C++ source file in g++ with -finput-charset compiler option but I'm always getting a bunch of errors.  More details follow.</p>
  
  <h2>My environment(in CentOS Linux):</h2>
  
  <ul>
  <li>g++: 4.1.2</li>
  <li>iconv: 2.5</li>
  <li>Linux language(in Terminal): LANG="en_US.UTF-8"</li>
  </ul>
  
  <h2>My sample source file(stored in UTF-16BE encoding):</h2>
  
  <pre><code>// main.cpp:
  
  #include &lt;iostream&gt;
  
  int main()
  {
      std::cout &lt;&lt; "Hello, UTF-16" &lt;&lt; std::endl;
      return 0;
  }
  </code></pre>
  
  <h2>My steps:</h2>
  
  <ul>
  <li><strong>I read the manual of g++ about the -finput-charset option.</strong>  The g++ manual says:</li>
  </ul>
  
  <blockquote>
    <p><strong>-finput-charset=charset</strong>
        Set the input character set, used for translation from the character set of the input file to the source character set used by
    GCC.  If the locale does not specify, or GCC cannot get this
    information from the locale, <strong>the default is UTF-8</strong>.  This can be
    overridden by either the locale or this command line option. 
    Currently the command line option takes precedence if there’s a
    conflict.  charset can be any encoding supported by the system’s
    "<strong>iconv</strong>" library routine.</p>
  </blockquote>
  
  <ul>
  <li><strong>Thus I entered the command as follows:</strong></li>
  </ul>
  
  <blockquote>
    <p>g++ -finput-charset=UTF-16BE main.cpp</p>
  </blockquote>
  
  <p>and I got these errors:</p>
  
  <blockquote>
    <p>In file included from main.cpp:1:</p>
    
    <p>/usr/lib/gcc/i386-redhat-linux/4.1.2/../../../../include/c++/4.1.2/iostream:1:
    error: stray ‘\342’ in program</p>
    
    <p>/usr/lib/gcc/i386-redhat-linux/4.1.2/../../../../include/c++/4.1.2/iostream:1:
    error: stray ‘\274’ in program </p>
    
    <p>...(repeatedly, A LOT, around 4000+)...</p>
    
    <p>/usr/lib/gcc/i386-redhat-linux/4.1.2/../../../../include/c++/4.1.2/iostream:1:
    error: stray ‘\257’ in program</p>
    
    <p>main.cpp: In function ‘int main()’:</p>
    
    <p>main.cpp:5: error: ‘cout’ is not a member of ‘std’ </p>
    
    <p>main.cpp:5: error: ‘endl’ is not a member of ‘std’</p>
  </blockquote>
  
  <ul>
  <li><strong>The manual text suggests that the charset can be any encoding supported by 'iconv' routine, thus I guessed the compilation errors might be caused by my iconv library.  I then tested the iconv:</strong></li>
  </ul>
  
  <blockquote>
    <p>iconv --from-code=UTF-16BE --to-code=UTF-8 --output=main_utf8.cpp main.cpp</p>
  </blockquote>
  
  <p>A "main_utf8.cpp" file is generated as expected.  I then tried to compile it:</p>
  
  <blockquote>
    <p>g++ -finput-charset=UTF-8 main_utf8.cpp</p>
  </blockquote>
  
  <p>Note that I specified the input-charset explicitly to see if I did anything wrong, but this time a "a.out" was generated without any errors.  When I ran it, it could produce the correct output.</p>
  
  <h2>Finally...</h2>
  
  <p>I couldn't figure out where I did wrong.  I searched in the web trying to find out some examples for this compiler option but I couldn't.</p>
  
  <p>Please advise!  Thanks!</p>
  
  <h2>Further edits:</h2>
  
  <p>Thanks, guys!  Your replies are quick!  Some updates:</p>
  
  <ol>
  <li>When I said "UTF-16" I meant "UTF-16 + BOM".  In fact I used UTF-16BE.  I have updated the text above.</li>
  <li>Some answers say the errors are caused by the non-UTF-16 header files.  Here are my thoughts if this is the case: We'll always include some standard header files when writing a C/C++ project, right?  Such as stdio.h or iostream.  If the G++ compiler only deals with the encoding of the source files created by us but never with the source files in the standard library, then what does this -finput-charset option exist for??</li>
  </ol>
  
  <h2>Final edit:</h2>
  
  <p>At last, my solution is like this:</p>
  
  <ol>
  <li>At the beginning, I changed the encoding of my source files to GB2312, as "Mr Lister" said below.  This worked fine for a while, but later I found it not suitable for my situation because most of the other parts in the system still use UTF-8 for communication and interfaces, thus I must convert the encoding in many places... Not only an overhead of my work, it may also result in some performance decrease in my program.</li>
  <li>Later I tried to convert all my source files to UTF-8 + BOM.  In this way, Visual Studio in Windows could compile them happily but GCC in Linux would complain.  I then wrote a shell script to remove the BOM, and before I want to compile my code with GCC, I run this script first.</li>
  <li>Luckily, I don't have to build the code in Linux manually because TeamCity the continuous integration tool is used in my project to generate the build automatically.  I could change the build steps in TeamCity to help me run this script before the daily build starts.</li>
  <li>With this UTF-8 + BOM + script method, I decide not to edit my source code in Linux, because if I want to do so, I must make sure my code could build successfully before I commit it, which means I must run the script to remove the BOM before I build the code, which means SVN would report EVERY file is modified(BOM removed) thus make it very easy to mistakenly commit a wrong file.  To solve this problem, I wrote another shell script to add the BOM back to the source files.  Though I still don't edit my code very often in Linux, but when I really need to, I don't have to face the terribly long change list in the commit dialog.</li>
  </ol>