{"action":"closed","issue":{"url":"https://api.github.com/repos/IQSS/dataverse/issues/4558","repository_url":"https://api.github.com/repos/IQSS/dataverse","labels_url":"https://api.github.com/repos/IQSS/dataverse/issues/4558/labels{/name}","comments_url":"https://api.github.com/repos/IQSS/dataverse/issues/4558/comments","events_url":"https://api.github.com/repos/IQSS/dataverse/issues/4558/events","html_url":"https://github.com/IQSS/dataverse/issues/4558","id":309880564,"node_id":"MDU6SXNzdWUzMDk4ODA1NjQ=","number":4558,"title":"unable to convert (2017) into YYYY format and couldn't index it (dsDescriptionDate)","user":{"login":"andrewSC","id":1331119,"node_id":"MDQ6VXNlcjEzMzExMTk=","avatar_url":"https://avatars3.githubusercontent.com/u/1331119?v=4","gravatar_id":"","url":"https://api.github.com/users/andrewSC","html_url":"https://github.com/andrewSC","followers_url":"https://api.github.com/users/andrewSC/followers","following_url":"https://api.github.com/users/andrewSC/following{/other_user}","gists_url":"https://api.github.com/users/andrewSC/gists{/gist_id}","starred_url":"https://api.github.com/users/andrewSC/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andrewSC/subscriptions","organizations_url":"https://api.github.com/users/andrewSC/orgs","repos_url":"https://api.github.com/users/andrewSC/repos","events_url":"https://api.github.com/users/andrewSC/events{/privacy}","received_events_url":"https://api.github.com/users/andrewSC/received_events","type":"User","site_admin":false},"labels":[{"id":633808813,"node_id":"MDU6TGFiZWw2MzM4MDg4MTM=","url":"https://api.github.com/repos/IQSS/dataverse/labels/Help%20Wanted:%20Code","name":"Help Wanted: Code","color":"128A0C","default":false},{"id":633809132,"node_id":"MDU6TGFiZWw2MzM4MDkxMzI=","url":"https://api.github.com/repos/IQSS/dataverse/labels/Mentor:%20pdurbin","name":"Mentor: pdurbin","color":"128A0C","default":false},{"id":112723874,"node_id":"MDU6TGFiZWwxMTI3MjM4NzQ=","url":"https://api.github.com/repos/IQSS/dataverse/labels/Status:%20QA","name":"Status: QA","color":"e6e6e6","default":false},{"id":112724030,"node_id":"MDU6TGFiZWwxMTI3MjQwMzA=","url":"https://api.github.com/repos/IQSS/dataverse/labels/Type:%20Bug","name":"Type: Bug","color":"fef2c0","default":false}],"state":"closed","locked":false,"assignee":{"login":"kcondon","id":4983816,"node_id":"MDQ6VXNlcjQ5ODM4MTY=","avatar_url":"https://avatars2.githubusercontent.com/u/4983816?v=4","gravatar_id":"","url":"https://api.github.com/users/kcondon","html_url":"https://github.com/kcondon","followers_url":"https://api.github.com/users/kcondon/followers","following_url":"https://api.github.com/users/kcondon/following{/other_user}","gists_url":"https://api.github.com/users/kcondon/gists{/gist_id}","starred_url":"https://api.github.com/users/kcondon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kcondon/subscriptions","organizations_url":"https://api.github.com/users/kcondon/orgs","repos_url":"https://api.github.com/users/kcondon/repos","events_url":"https://api.github.com/users/kcondon/events{/privacy}","received_events_url":"https://api.github.com/users/kcondon/received_events","type":"User","site_admin":false},"assignees":[{"login":"kcondon","id":4983816,"node_id":"MDQ6VXNlcjQ5ODM4MTY=","avatar_url":"https://avatars2.githubusercontent.com/u/4983816?v=4","gravatar_id":"","url":"https://api.github.com/users/kcondon","html_url":"https://github.com/kcondon","followers_url":"https://api.github.com/users/kcondon/followers","following_url":"https://api.github.com/users/kcondon/following{/other_user}","gists_url":"https://api.github.com/users/kcondon/gists{/gist_id}","starred_url":"https://api.github.com/users/kcondon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kcondon/subscriptions","organizations_url":"https://api.github.com/users/kcondon/orgs","repos_url":"https://api.github.com/users/kcondon/repos","events_url":"https://api.github.com/users/kcondon/events{/privacy}","received_events_url":"https://api.github.com/users/kcondon/received_events","type":"User","site_admin":false}],"milestone":null,"comments":14,"created_at":"2018-03-29T19:45:14Z","updated_at":"2018-09-27T16:42:09Z","closed_at":"2018-09-27T16:42:09Z","author_association":"CONTRIBUTOR","body":"After clearing the Solr index and kicking off a full-reindex I noticed multiple log messages similar to: \r\n```\r\n[2018-03-29T14:41:40.296-0400] [glassfish 4.1] [INFO] [] [edu.harvard.iq.dataverse.search.IndexAllServiceBean] [tid: _ThreadID=160 _ThreadName=__ejb-thread-pool10] [timeMillis: 1522348900296] [levelValue: 800] [[\r\nindexing dataset 618 of 975 (id=4684, persistentId=<omitted>)]]\r\n\r\n[2018-03-29T14:41:40.346-0400] [glassfish 4.1] [INFO] [] [edu.harvard.iq.dataverse.search.IndexServiceBean] [tid: _ThreadID=160 _ThreadName=__ejb-thread-pool10] [timeMillis: 1522348900346] [levelValue: 800] [[\r\nunable to convert (2017-02-22) into YYYY format and couldn't index it (dsDescriptionDate)]]\r\n\r\n[2018-03-29T14:41:40.777-0400] [glassfish 4.1] [INFO] [] [edu.harvard.iq.dataverse.search.IndexAllServiceBean] [tid: _ThreadID=160 _ThreadName=__ejb-thread-pool10] [timeMillis: 1522348900777] [levelValue: 800] [[\r\nindexing dataset 619 of 975 (id=4686, persistentId=<omitted>)]]\r\n```\r\n\r\nAfter running `asadmin set-log-levels edu.harvard.iq.dataverse.search.IndexServiceBean=FINE` I was able to capture some more descriptive output (of a different dataset failing) from the logging in [IndexServiceBean.java#L713](https://github.com/IQSS/dataverse/blob/develop/src/main/java/edu/harvard/iq/dataverse/search/IndexServiceBean.java#L713) which provided:\r\n```\r\n15906-[2018-03-29T15:14:02.360-0400] [glassfish 4.1] [FINE] [] [edu.harvard.iq.dataverse.search.IndexServiceBean] [tid: _ThreadID=158 _ThreadName=__ejb-thread-pool8] [timeMillis: 1522350842360] [levelValue: 500] [CLASSNAME: edu.harvard.iq.dataverse.search.IndexServiceBean] [METHODNAME: addOrUpdateDataset] [[\r\n15907-  indexing dsDescriptionDate:[(2017)] into dsDescriptionDate and maybe dsDescriptionDate_ss]]\r\n15908-\r\n15909-[2018-03-29T15:14:02.360-0400] [glassfish 4.1] [FINE] [] [edu.harvard.iq.dataverse.search.IndexServiceBean] [tid: _ThreadID=158 _ThreadName=__ejb-thread-pool8] [timeMillis: 1522350842360] [levelValue: 500] [CLASSNAME: edu.harvard.iq.dataverse.search.IndexServiceBean] [METHODNAME: addOrUpdateDataset] [[\r\n15910-  date as string: (2017)]]\r\n15911-\r\n15912-[2018-03-29T15:14:02.360-0400] [glassfish 4.1] [FINE] [] [edu.harvard.iq.dataverse.search.IndexServiceBean] [tid: _ThreadID=158 _ThreadName=__ejb-thread-pool8] [timeMillis: 1522350842360] [levelValue: 500] [CLASSNAME: edu.harvard.iq.dataverse.search.IndexServiceBean] [METHODNAME: addOrUpdateDataset] [[\r\n15913:  Trying to convert (2017) to a YYYY date from dataset 5135]]\r\n15914-\r\n15915-[2018-03-29T15:14:02.360-0400] [glassfish 4.1] [INFO] [] [edu.harvard.iq.dataverse.search.IndexServiceBean] [tid: _ThreadID=158 _ThreadName=__ejb-thread-pool8] [timeMillis: 1522350842360] [levelValue: 800] [[\r\n15916-  unable to convert (2017) into YYYY format and couldn't index it (dsDescriptionDate)]]\r\n```\r\n\r\nWhich seems to imply that the conversion is failing due to parenthesis existing around the date at the time of conversion. I did some quick spelunking and it doesn't seem immediately obvious what's adding them? Here's what I found in the codebase:\r\n```\r\n$ ag '\\\"\\(\\\"\\ \\+'\r\nmain/java/edu/harvard/iq/dataverse/mydata/MyDataFilterParams.java\r\n273:            valStr = \"(\" + valStr + \")\";\r\n294:            valStr = \"(\" + valStr + \")\";\r\n297:        return  \"(\" + SearchFields.PUBLICATION_STATUS + \":\" + valStr + \")\";\r\n\r\nmain/java/edu/harvard/iq/dataverse/mydata/MyDataFinder.java\r\n354:            return \"(\" + entityIdClause + \" OR \" + parentIdClause + \")\";\r\n\r\nmain/java/edu/harvard/iq/dataverse/mydata/RoleTagRetriever.java\r\n395:            //msg(\"dv id: \" + dvId + \"(\" + this.idToDvObjectType.get(dvId) + \") | roleId: \"\r\n396:            //        + roleId + \"(\" +  this.rolePermissionHelper.getRoleName(roleId)+\")\");\r\n496:           //msgt(\"dvIdForCard: \" + dvIdForCard + \"(\" + this.idToDvObjectType.get(dvIdForCard) + \")\");\r\n\r\nmain/java/edu/harvard/iq/dataverse/mydata/SolrQueryFormatter.java\r\n53:                            String qPart = \"(\" + paramName  + \":(\" + orClause + \"))\";\r\n55:             qPart = \"(\" + paramName  + \":(\" + orClause + \") AND \" + SearchFields.TYPE + \":(\" +  dvObjectType + \"))\";\r\n\r\nmain/java/edu/harvard/iq/dataverse/authorization/AuthenticationServiceBean.java\r\n849:                            String error = \"at \" + stacktrace.getClassName() + \".\" + stacktrace.getMethodName() + \"(\" + stacktrace.getFileName() + \":\" + lineNumber + \") \";\r\n\r\nmain/java/edu/harvard/iq/dataverse/harvest/server/OAISetServiceBean.java\r\n222:        query = \"(\" + query + \")\";\r\n\r\nmain/java/edu/harvard/iq/dataverse/api/Index.java\r\n191:                            String error = \"at \" + stacktrace.getClassName() + \".\" + stacktrace.getMethodName() + \"(\" + stacktrace.getFileName() + \":\" + lineNumber + \") \";\r\n279:                            String error = \"at \" + stacktrace.getClassName() + \".\" + stacktrace.getMethodName() + \"(\" + stacktrace.getFileName() + \":\" + lineNumber + \") \";\r\n\r\nmain/java/edu/harvard/iq/dataverse/PermissionServiceBean.java\r\n503:                    indirectParentIds = \"(\" + Integer.toString(dvIdAsInt);\r\n525:                        indirectDatasetParentIds = \"(\" + Integer.toString(dvIdAsInt);\r\n\r\nmain/java/edu/harvard/iq/dataverse/dataaccess/ImageThumbConverter.java\r\n347:        logger.fine(\"image dimensions: \" + width + \"x\" + height + \"(\" + storageIO.getDataFile().getStorageIdentifier() + \")\");\r\n\r\nmain/java/edu/harvard/iq/dataverse/search/IndexServiceBean.java\r\n133:        logger.fine(\"indexDataverse called on dataverse id \" + dataverse.getId() + \"(\" + dataverse.getAlias() + \")\");\r\n```"}}
{"actor":{"display_login":"kcondon"}}