title: Weird JAVA UTF-8 encoding behaviour, new String(bytes,"UTF-8") gives different results on mostly similar setup
body: <p>My application needs to support Japanese character, so we have used UTF-8 as default encoding on the entire stack. We are facing a wierd issue where the new String ( bytes, "UTF-8") gives a different results.</p>
  
  <p>Input from user: 東京<br>
  Base64 encoded string generated from Browser and sent to API: 5p2x5Lqs<br>
  Both system generate same byte array.<br>
  But only on system 1 decoded string comes as 東京<br>
  On System 2 the decoded string comes as ??  </p>
  
  <p>System 1:<br>
  Container: Tomee 7.1.0<br>
  JDK: 1.8.0_201-b09<br>
  OS Version:            3.10.0-957.12.2.el7.x86_64<br>
  Architecture:          amd64<br>
  Locale:</p>
  
  <pre><code>[logs]$ locale
  LANG=en_GB.UTF-8
  LC_CTYPE="en_GB.UTF-8"
  LC_NUMERIC="en_GB.UTF-8"
  LC_TIME="en_GB.UTF-8"
  LC_COLLATE="en_GB.UTF-8"
  LC_MONETARY="en_GB.UTF-8"
  LC_MESSAGES="en_GB.UTF-8"
  LC_PAPER="en_GB.UTF-8"
  LC_NAME="en_GB.UTF-8"
  LC_ADDRESS="en_GB.UTF-8"
  LC_TELEPHONE="en_GB.UTF-8"
  LC_MEASUREMENT="en_GB.UTF-8"
  LC_IDENTIFICATION="en_GB.UTF-8"
  LC_ALL=
  [logs]$ locale status
  locale: unknown name "status"
  
  </code></pre>
  
  <p>System 2:<br>
  Container: Tomee 7.1.0<br>
  JDK: 1.8.0_201-b09<br>
  OS Version:            2.6.32-696.18.7.el6.x86_64<br>
  Architecture:          amd64<br>
  Locale:</p>
  
  <pre><code>[ logs]$ locale
  LANG=en_GB
  LC_CTYPE="en_GB"
  LC_NUMERIC="en_GB"
  LC_TIME="en_GB"
  LC_COLLATE="en_GB"
  LC_MONETARY="en_GB"
  LC_MESSAGES="en_GB"
  LC_PAPER="en_GB"
  LC_NAME="en_GB"
  LC_ADDRESS="en_GB"
  LC_TELEPHONE="en_GB"
  LC_MEASUREMENT="en_GB"
  LC_IDENTIFICATION="en_GB"
  LC_ALL=
  [ logs]$ locale status
  locale: unknown name "status"
  
  </code></pre>
  
  <p>Java code being used</p>
  
  <pre><code>LogUtil.logMessage("searchString before decoding="+searchString);
                   //s =  new String (Base64.decodeBase64(searchString),utf8_test);
                   byte[] decodedBytes=Base64.getDecoder().decode(searchString);
                   byte[] decodedBaytesFromapache=org.apache.commons.codec.binary.Base64.decodeBase64(searchString);
                   System.out.println("java native array :: ");
                   for(byte b:decodedBytes)
                   {
                       System.out.print(b);
                   }
                   System.out.println("\njava apache array :: \n");
                   for(byte b:decodedBaytesFromapache)
                   {
                       System.out.print(b);
                   }
                   s=new String(decodedBytes,"UTF-8"); //Charset.forName("UTF-8") was also tried here
                   System.out.println("\n String post decode:: "+s);
                   System.out.println("");
              //String s = 
                   System.out.println("loaded charset is utf-8:: "+Charset.isSupported("UTF-8"));
                   Set&lt;String&gt; listOfCharsets=Charset.availableCharsets().keySet();
                   System.out.println("Listing supported charsets:: ");
                   for(String item: listOfCharsets)
                   {System.out.println(item); }
  </code></pre>
  
  <p>Output on <strong>System1</strong></p>
  
  <pre><code>searchString before decoding=5p2x5Lqs
  java native array ::
  -26-99-79-28-70-84
  java apache array ::
  
  -26-99-79-28-70-84
   String post decode:: 東京
  
  loaded charset is utf-8:: true
  Listing supported charsets::
  Big5
  Big5-HKSCS
  CESU-8
  EUC-JP
  EUC-KR
  GB18030
  GB2312
  GBK
  IBM-Thai
  IBM00858
  IBM01140
  IBM01141
  IBM01142
  IBM01143
  IBM01144
  IBM01145
  IBM01146
  IBM01147
  IBM01148
  IBM01149
  IBM037
  IBM1026
  IBM1047
  IBM273
  IBM277
  IBM278
  IBM280
  IBM284
  IBM285
  IBM290
  IBM297
  IBM420
  IBM424
  IBM437
  IBM500
  IBM775
  IBM850
  IBM852
  IBM855
  IBM857
  IBM860
  IBM861
  IBM862
  IBM863
  IBM864
  IBM865
  IBM866
  IBM868
  IBM869
  IBM870
  IBM871
  IBM918
  ISO-2022-CN
  ISO-2022-JP
  ISO-2022-JP-2
  ISO-2022-KR
  ISO-8859-1
  ISO-8859-13
  ISO-8859-15
  ISO-8859-2
  ISO-8859-3
  ISO-8859-4
  ISO-8859-5
  ISO-8859-6
  ISO-8859-7
  ISO-8859-8
  ISO-8859-9
  JIS_X0201
  JIS_X0212-1990
  KOI8-R
  KOI8-U
  Shift_JIS
  TIS-620
  US-ASCII
  UTF-16
  UTF-16BE
  UTF-16LE
  UTF-32
  UTF-32BE
  UTF-32LE
  UTF-8
  windows-1250
  windows-1251
  windows-1252
  windows-1253
  windows-1254
  windows-1255
  windows-1256
  windows-1257
  windows-1258
  windows-31j
  x-Big5-HKSCS-2001
  x-Big5-Solaris
  x-COMPOUND_TEXT
  x-euc-jp-linux
  x-EUC-TW
  x-eucJP-Open
  x-IBM1006
  x-IBM1025
  x-IBM1046
  x-IBM1097
  x-IBM1098
  x-IBM1112
  x-IBM1122
  x-IBM1123
  x-IBM1124
  x-IBM1166
  x-IBM1364
  x-IBM1381
  x-IBM1383
  x-IBM300
  x-IBM33722
  x-IBM737
  x-IBM833
  x-IBM834
  x-IBM856
  x-IBM874
  x-IBM875
  x-IBM921
  x-IBM922
  x-IBM930
  x-IBM933
  x-IBM935
  x-IBM937
  x-IBM939
  x-IBM942
  x-IBM942C
  x-IBM943
  x-IBM943C
  x-IBM948
  x-IBM949
  x-IBM949C
  x-IBM950
  x-IBM964
  x-IBM970
  x-ISCII91
  x-ISO-2022-CN-CNS
  x-ISO-2022-CN-GB
  x-iso-8859-11
  x-JIS0208
  x-JISAutoDetect
  x-Johab
  x-MacArabic
  x-MacCentralEurope
  x-MacCroatian
  x-MacCyrillic
  x-MacDingbat
  x-MacGreek
  x-MacHebrew
  x-MacIceland
  x-MacRoman
  x-MacRomania
  x-MacSymbol
  x-MacThai
  x-MacTurkish
  x-MacUkraine
  x-MS932_0213
  x-MS950-HKSCS
  x-MS950-HKSCS-XP
  x-mswin-936
  x-PCK
  x-SJIS_0213
  x-UTF-16LE-BOM
  X-UTF-32BE-BOM
  X-UTF-32LE-BOM
  x-windows-50220
  x-windows-50221
  x-windows-874
  x-windows-949
  x-windows-950
  x-windows-iso2022jp
  searchString after decoding=東京
  </code></pre>
  
  <p>Output on <strong>System 2</strong></p>
  
  <pre><code>searchString before decoding=5p2x5Lqs
  java native array ::
  -26-99-79-28-70-84
  java apache array ::
  
  -26-99-79-28-70-84
   String post decode:: ??
  
  loaded charset is utf-8:: true
  Listing supported charsets::
  Big5
  Big5-HKSCS
  CESU-8
  EUC-JP
  EUC-KR
  GB18030
  GB2312
  GBK
  IBM-Thai
  IBM00858
  IBM01140
  IBM01141
  IBM01142
  IBM01143
  IBM01144
  IBM01145
  IBM01146
  IBM01147
  IBM01148
  IBM01149
  IBM037
  IBM1026
  IBM1047
  IBM273
  IBM277
  IBM278
  IBM280
  IBM284
  IBM285
  IBM290
  IBM297
  IBM420
  IBM424
  IBM437
  IBM500
  IBM775
  IBM850
  IBM852
  IBM855
  IBM857
  IBM860
  IBM861
  IBM862
  IBM863
  IBM864
  IBM865
  IBM866
  IBM868
  IBM869
  IBM870
  IBM871
  IBM918
  ISO-2022-CN
  ISO-2022-JP
  ISO-2022-JP-2
  ISO-2022-KR
  ISO-8859-1
  ISO-8859-13
  ISO-8859-15
  ISO-8859-2
  ISO-8859-3
  ISO-8859-4
  ISO-8859-5
  ISO-8859-6
  ISO-8859-7
  ISO-8859-8
  ISO-8859-9
  JIS_X0201
  JIS_X0212-1990
  KOI8-R
  KOI8-U
  Shift_JIS
  TIS-620
  US-ASCII
  UTF-16
  UTF-16BE
  UTF-16LE
  UTF-32
  UTF-32BE
  UTF-32LE
  UTF-8
  windows-1250
  windows-1251
  windows-1252
  windows-1253
  windows-1254
  windows-1255
  windows-1256
  windows-1257
  windows-1258
  windows-31j
  x-Big5-HKSCS-2001
  x-Big5-Solaris
  x-COMPOUND_TEXT
  x-euc-jp-linux
  x-EUC-TW
  x-eucJP-Open
  x-IBM1006
  x-IBM1025
  x-IBM1046
  x-IBM1097
  x-IBM1098
  x-IBM1112
  x-IBM1122
  x-IBM1123
  x-IBM1124
  x-IBM1166
  x-IBM1364
  x-IBM1381
  x-IBM1383
  x-IBM300
  x-IBM33722
  x-IBM737
  x-IBM833
  x-IBM834
  x-IBM856
  x-IBM874
  x-IBM875
  x-IBM921
  x-IBM922
  x-IBM930
  x-IBM933
  x-IBM935
  x-IBM937
  x-IBM939
  x-IBM942
  x-IBM942C
  x-IBM943
  x-IBM943C
  x-IBM948
  x-IBM949
  x-IBM949C
  x-IBM950
  x-IBM964
  x-IBM970
  x-ISCII91
  x-ISO-2022-CN-CNS
  x-ISO-2022-CN-GB
  x-iso-8859-11
  x-JIS0208
  x-JISAutoDetect
  x-Johab
  x-MacArabic
  x-MacCentralEurope
  x-MacCroatian
  x-MacCyrillic
  x-MacDingbat
  x-MacGreek
  x-MacHebrew
  x-MacIceland
  x-MacRoman
  x-MacRomania
  x-MacSymbol
  x-MacThai
  x-MacTurkish
  x-MacUkraine
  x-MS932_0213
  x-MS950-HKSCS
  x-MS950-HKSCS-XP
  x-mswin-936
  x-PCK
  x-SJIS_0213
  x-UTF-16LE-BOM
  X-UTF-32BE-BOM
  X-UTF-32LE-BOM
  x-windows-50220
  x-windows-50221
  x-windows-874
  x-windows-949
  x-windows-950
  x-windows-iso2022jp
  searchString after decoding=??
  </code></pre>
  
  <p>The ?? are not due to terminal window, as both were taken from same putty terminal with all matched settings.
  The ?? is then passed to jdbctemplate which returns 0 results on System 2, while on System 1 we are getting the expected result.
  What is the possible solution to make the decoding consistent on all systems?</p>