{"action":"created","issue":{"url":"https://api.github.com/repos/guxd/deep-code-search/issues/16","repository_url":"https://api.github.com/repos/guxd/deep-code-search","labels_url":"https://api.github.com/repos/guxd/deep-code-search/issues/16/labels{/name}","comments_url":"https://api.github.com/repos/guxd/deep-code-search/issues/16/comments","events_url":"https://api.github.com/repos/guxd/deep-code-search/issues/16/events","html_url":"https://github.com/guxd/deep-code-search/issues/16","id":389901116,"node_id":"MDU6SXNzdWUzODk5MDExMTY=","number":16,"title":"Evaluation Benchmark on the trained model","user":{"login":"matanpugach","id":27849930,"node_id":"MDQ6VXNlcjI3ODQ5OTMw","avatar_url":"https://avatars2.githubusercontent.com/u/27849930?v=4","gravatar_id":"","url":"https://api.github.com/users/matanpugach","html_url":"https://github.com/matanpugach","followers_url":"https://api.github.com/users/matanpugach/followers","following_url":"https://api.github.com/users/matanpugach/following{/other_user}","gists_url":"https://api.github.com/users/matanpugach/gists{/gist_id}","starred_url":"https://api.github.com/users/matanpugach/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/matanpugach/subscriptions","organizations_url":"https://api.github.com/users/matanpugach/orgs","repos_url":"https://api.github.com/users/matanpugach/repos","events_url":"https://api.github.com/users/matanpugach/events{/privacy}","received_events_url":"https://api.github.com/users/matanpugach/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-12-11T18:43:21Z","updated_at":"2018-12-25T04:31:25Z","closed_at":null,"author_association":"NONE","body":"Hi, i am trying to reproduce the papers results. I re-trained the Keras model for 500 epochs (as written in the paper) to replicate the originally returned code snippets. I followed the readme instructions and still couldn't reach the paper's results. I then tried updating the reload param in configs.py to be 500 and got the following responses for queries from the benchmark: \r\n\r\nFor the query:\r\niterate through a   hashmap\r\nI got the responses:\r\n```\r\n('void startBlock ( final int buildingCount ) { SwingUtilities . invokeLater ( new Runnable ( ) { @ Override public void run ( ) { statusLabel . setText ( \"Computing-blockades\" ) ; if ( buildingCount == 0 ) { blockadeProgress . setMaximum ( 1 ) ; blockadeProgress . setValue ( 1 ) ; } else { blockadeProgress . setMaximum ( buildingCount ) ; blockadeProgress . setValue ( 0 ) ; block = 0 ; } } } ) ; } \\n', 0.035571914)\r\n\r\n('public static void main ( String [ ] args ) { long maxMemory = runtime . maxMemory ( ) ; long allocatedMemory = runtime . totalMemory ( ) ; long freeMemory = runtime . freeMemory ( ) ; System . out . println ( \"free-memory:-\" + freeMemory / 1024 ) ; System . out . println ( \"allocated-memory:-\" + allocatedMemory / 1024 ) ; System . out . println ( \"max-memory:-\" + maxMemory / 1024 ) ; System . out . println ( \"total-free-memory:-\" + ( freeMemory + ( maxMemory - allocatedMemory ) ) / 1024 ) ; } \\n', 0.033702463)\r\n\r\n('@ Override public boolean equals ( Object obj ) { if ( ! ( obj instanceof ByteOrderMark ) ) { return false ; } ByteOrderMark bom = ( ByteOrderMark ) obj ; if ( bytes . length != bom . length ( ) ) { return false ; } for ( int i = 0 ; i < bytes . length ; i ++ ) { if ( bytes [ i ] != bom . get ( i ) ) { return false ; } } return true ; } \\n', 0.03228843)\r\n\r\n('static public RtspSession create ( String sessionId ) { if ( sessions . get ( sessionId ) != null ) { log . error ( \"Session-key-conflit!!\" ) ; return null ; } RtspSession session = new RtspSession ( sessionId ) ; sessions . put ( sessionId , session ) ; log . debug ( \"New-session-created---id=\" + sessionId ) ; return session ; } \\n', 0.03165227)\r\n\r\n('public List < Resource > getResources ( ) { if ( resources == null ) { resources = new ArrayList < Resource > ( ) ; } return this . resources ; } \\n', 0.031491578)\r\n\r\n('public List < String > getSingleCommentDelimiters ( ) { return new ArrayList < String > ( singleCommentDelimiters ) ; } \\n', 0.031074077)\r\n\r\n('public String getCanonicalHostName ( ) { try { return getHostByAddrImpl ( this ) . hostName ; } catch ( UnknownHostException ex ) { return getHostAddress ( ) ; } } \\n', 0.030964658)\r\n\r\n('public static < K , V > Map < K , V > createNewMap ( ) { return new HashMap < K , V > ( ) ; } \\n', 0.030577734)\r\n\r\n('public static String separatorsToWindows ( String path ) { if ( path == null || path . indexOf ( UNIX_SEPARATOR ) == - 1 ) { return path ; } return path . replace ( UNIX_SEPARATOR , WINDOWS_SEPARATOR ) ; } \\n', 0.030175861)\r\n\r\n('public synchronized void shutdown ( ) { dataQueue . clear ( ) ; shutdown = true ; } \\n', 0.030141229)\r\n\r\n('@ Override public void run ( ) { while ( ! destroyed ) { StreamOutput processStream = StreamOutput . EMPTY ; try { CapturedStreamOutput capturedProcessStream = null ; while ( ! destroyed && capturedProcessStream == null ) { synchronized ( toCapture ) { if ( toCapture . containsKey ( key ) ) { capturedProcessStream = toCapture . remove ( key ) ; } else { toCapture . wait ( ) ; } } } if ( ! destroyed ) { processStream = capturedProcessStream ; capturedProcessStream . readAndClose ( ) ; } } catch ( InterruptedException e ) { logger . info ( \"OutputCapture-interrupted,-exiting\" ) ; break ; } catch ( IOException e ) { logger . error ( \"Error-reading-process-output\" , e ) ; } finally { synchronized ( fromCapture ) { fromCapture . put ( key , processStream ) ; fromCapture . notify ( ) ; } } } } \\n', 0.029915098)\r\n\r\n('public static Document parse ( File file ) throws IOException , ParserConfigurationException , SAXException { DocumentBuilderFactory factory = DocumentBuilderFactory . newInstance ( ) ; DocumentBuilder builder = factory . newDocumentBuilder ( ) ; return builder . parse ( file ) ; } \\n', 0.02982778)\r\n\r\n('private static String _latexFromHtml ( Collection col , String latex ) { latex = latex . replaceAll ( \"<br(-/)?>|<div>\" , \"-\" ) ; latex = Utils . stripHTML ( latex ) ; return latex ; } \\n', 0.029724797)\r\n\r\n('public synchronized boolean clearExpired ( final Date date ) { if ( date == null ) { return false ; } boolean removed = false ; for ( Iterator < Cookie > it = cookies . iterator ( ) ; it . hasNext ( ) ; ) { if ( it . next ( ) . isExpired ( date ) ) { it . remove ( ) ; removed = true ; } } return removed ; } \\n', 0.029710343)\r\n\r\n('private String getAssetSource ( String assetPath ) { if ( StringUtils . isNotBlank ( assetPath ) ) { File asset = new File ( rootDir + File . separator + assetPath ) ; if ( asset . exists ( ) && asset . canRead ( ) ) { try { return FileUtils . readFileToString ( asset ) ; } catch ( IOException e ) { LOG . error ( \"Error-reading-asset-source:-\" + asset . getPath ( ) , e ) ; } } } return null ; } \\n', 0.02968416)\r\n```\r\nWhere only one was related to hash map (and no iteration)\r\n\r\nfor the query:\r\nconvert an inputstream to a string\r\nI got the responses:\r\n```\r\n('public static void registerTarget ( javax . rmi . CORBA . Tie tie , java . rmi . Remote target ) { if ( utilDelegate != null ) { utilDelegate . registerTarget ( tie , target ) ; } } \\n', 0.059240054)\r\n\r\n('public Quaternion normalise ( Quaternion dest ) { return normalise ( this , dest ) ; } \\n', 0.05841601)\r\n\r\n('public static IntFunction chain ( final IntFunction g , final IntFunction h ) { return new IntFunction ( ) { public final int apply ( int a ) { return g . apply ( h . apply ( a ) ) ; } } ; } \\n', 0.057828866)\r\n\r\n('void focusWheel ( ) { setFocusType ( 1 ) ; } \\n', 0.05775229)\r\n\r\n('public ServerSocket createServerSocket ( int port ) throws IOException { _logger . debug ( \"PortalSocketFactory.createServerPort()---port=\" + port ) ; ServerSocket srvsocket = new ServerSocket ( port ) ; _logger . debug ( \"PortalSocketFactory-returned-new-ServerSocket(port)...\" ) ; return ( srvsocket ) ; } \\n', 0.05753772)\r\n\r\n('@ Override public final long evaluateAdj ( final int [ ] adj ) { int a , old_i , cur_i ; long totalDist ; final double [ ] m ; double phi1 , phi2 , chi1 , chi2 , twoth1 , twoth2 , t ; old_i = 1 ; totalDist = 0l ; m = this . m_coords ; a = ( ( old_i - 1 ) * 3 ) ; phi2 = m [ a ++ ] ; chi2 = m [ a ++ ] ; twoth2 = m [ a ++ ] ; for ( ; ; ) { cur_i = adj [ old_i - 1 ] ; a = ( ( cur_i - 1 ) * 3 ) ; phi1 = m [ a ++ ] ; chi1 = m [ a ++ ] ; twoth1 = m [ a ++ ] ; t = Math . abs ( phi1 - phi2 ) ; totalDist += ( ( int ) ( 0.5d + ( Math . max ( Math . min ( t , Math . abs ( t - ( 360d ) ) ) , Math . max ( Math . abs ( chi1 - chi2 ) , Math . abs ( twoth1 - twoth2 ) ) ) ) ) ) ; phi2 = phi1 ; chi2 = chi1 ; twoth2 = twoth1 ; if ( cur_i == 1 ) { return totalDist ; } old_i = cur_i ; } } \\n', 0.05679653)\r\n\r\n('public boolean equals ( Object o ) { return m_Root . equals ( ( ( Trie ) o ) . getRoot ( ) ) ; } \\n', 0.056255654)\r\n\r\n('@ Override public P2pActivityCorrections fetchByPrimaryKey ( Serializable primaryKey ) throws SystemException { return fetchByPrimaryKey ( ( ( Long ) primaryKey ) . longValue ( ) ) ; } \\n', 0.056060914)\r\n\r\n('public int getWidth ( ) { init ( ) ; return width ; } \\n', 0.056045968)\r\n\r\n('public void dispose ( ) { logDebug ( \"Disposing.\" ) ; mSetupDone = false ; if ( mServiceConn != null ) { logDebug ( \"Unbinding-from-service.\" ) ; if ( mContext != null ) mContext . unbindService ( mServiceConn ) ; } mDisposed = true ; mContext = null ; mServiceConn = null ; mService = null ; mPurchaseListener = null ; } \\n', 0.055970065)\r\n\r\n('private List < PrivacyListItem > tranformPrivacyItemsToPrivacyListItems ( List < PrivacyItem > items ) { List < PrivacyListItem > rItems = new ArrayList < PrivacyListItem > ( ) ; for ( int i = 0 ; i < items . size ( ) ; i ++ ) { rItems . add ( new PrivacyListItem ( items . get ( i ) . getType ( ) . ordinal ( ) , items . get ( i ) . getValue ( ) ) ) ; } return rItems ; } \\n', 0.055890348)\r\n\r\n('public BigInteger multiply ( BigInteger val ) { if ( val . signum == 0 || signum == 0 ) return ZERO ; int [ ] result = multiplyToLen ( mag , mag . length , val . mag , val . mag . length , null ) ; result = trustedStripLeadingZeroInts ( result ) ; return new BigInteger ( result , signum == val . signum ? 1 : - 1 ) ; } \\n', 0.05568312)\r\n\r\n('public java . util . Iterator getAllMembers ( ) throws GroupsException { return primGetAllMembers ( new HashSet ( ) ) . iterator ( ) ; } \\n', 0.055669665)\r\n\r\n('public void checkTextOnly ( ) { for ( int i = 0 ; i < size ; i ++ ) { ( ( CharSequence ) parts [ i ] ) . getClass ( ) ; } } \\n', 0.055539)\r\n\r\n('public void drawBarsOnGraph ( Graphics2D g2d , ArrayList < ComparableLabel > orderedDateSet , HashMap < ComparableLabel , Integer [ ] > barDataPoints , long yMaxMark ) { int sectionWidth = this . graphWidth / orderedDateSet . size ( ) ; int xOffset = sectionWidth / 2 ; int yValue ; float yOffsetPerc ; int numberOfBars = barDataPoints . get ( orderedDateSet . get ( 0 ) ) . length ; int barWidth = sectionWidth / ( numberOfBars + 1 ) ; for ( int datePos = 0 ; datePos < orderedDateSet . size ( ) ; datePos ++ ) { for ( int barNumber = 0 ; barNumber < numberOfBars ; barNumber ++ ) { yValue = barDataPoints . get ( orderedDateSet . get ( datePos ) ) [ barNumber ] ; yOffsetPerc = yValue / ( float ) yMaxMark ; int xLeftBar = this . graphLeft + datePos * sectionWidth + barWidth / 2 + barWidth * barNumber ; drawBar ( g2d , Math . round ( this . graphHeight * yOffsetPerc ) , barWidth , xLeftBar , this . graphBottom , this . barColors [ barNumber ] ) ; if ( Math . round ( this . graphHeight * yOffsetPerc ) == 0 && yValue != 0 ) { g2d . setColor ( this . barColors [ barNumber ] ) ; g2d . drawLine ( xLeftBar , this . graphBottom , xLeftBar + barWidth , this . graphBottom ) ; } } } } \\n', 0.0555174)\r\n```\r\nAnd similiar quality results which don't fit the result table for the other queries."},"comment":{"url":"https://api.github.com/repos/guxd/deep-code-search/issues/comments/449799617","html_url":"https://github.com/guxd/deep-code-search/issues/16#issuecomment-449799617","issue_url":"https://api.github.com/repos/guxd/deep-code-search/issues/16","id":449799617,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0OTc5OTYxNw==","user":{"login":"guxd","id":6091014,"node_id":"MDQ6VXNlcjYwOTEwMTQ=","avatar_url":"https://avatars0.githubusercontent.com/u/6091014?v=4","gravatar_id":"","url":"https://api.github.com/users/guxd","html_url":"https://github.com/guxd","followers_url":"https://api.github.com/users/guxd/followers","following_url":"https://api.github.com/users/guxd/following{/other_user}","gists_url":"https://api.github.com/users/guxd/gists{/gist_id}","starred_url":"https://api.github.com/users/guxd/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/guxd/subscriptions","organizations_url":"https://api.github.com/users/guxd/orgs","repos_url":"https://api.github.com/users/guxd/repos","events_url":"https://api.github.com/users/guxd/events{/privacy}","received_events_url":"https://api.github.com/users/guxd/received_events","type":"User","site_admin":false},"created_at":"2018-12-25T04:31:25Z","updated_at":"2018-12-25T04:31:25Z","author_association":"OWNER","body":"We tested the code and everything is just normal. \r\nThe similarities are around 0.4.\r\nDid you normalize the code vectors by run ```$python norm_code_repr.py```? "}}
{"actor":{"display_login":"guxd"}}