{"action":"created","issue":{"url":"https://api.github.com/repos/Rblp/Rblpapi/issues/253","repository_url":"https://api.github.com/repos/Rblp/Rblpapi","labels_url":"https://api.github.com/repos/Rblp/Rblpapi/issues/253/labels{/name}","comments_url":"https://api.github.com/repos/Rblp/Rblpapi/issues/253/comments","events_url":"https://api.github.com/repos/Rblp/Rblpapi/issues/253/events","html_url":"https://github.com/Rblp/Rblpapi/issues/253","id":280867796,"number":253,"title":"Race in bds_Impl?","user":{"login":"randomee","id":14268573,"avatar_url":"https://avatars0.githubusercontent.com/u/14268573?v=4","gravatar_id":"","url":"https://api.github.com/users/randomee","html_url":"https://github.com/randomee","followers_url":"https://api.github.com/users/randomee/followers","following_url":"https://api.github.com/users/randomee/following{/other_user}","gists_url":"https://api.github.com/users/randomee/gists{/gist_id}","starred_url":"https://api.github.com/users/randomee/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/randomee/subscriptions","organizations_url":"https://api.github.com/users/randomee/orgs","repos_url":"https://api.github.com/users/randomee/repos","events_url":"https://api.github.com/users/randomee/events{/privacy}","received_events_url":"https://api.github.com/users/randomee/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":27,"created_at":"2017-12-11T01:23:15Z","updated_at":"2018-02-21T08:20:21Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"Converting some Rbbg (java) code to Rblpapi, I'm hitting a persistent problem with bds. \r\n\r\nThe exact segfault is always a little different, but it's always in bds_Impl or Rblpapi_bds_Impl.\r\n\r\n```\r\n *** caught segfault ***\r\naddress (nil), cause 'unknown'\r\n\r\nTraceback:\r\n 1: .Call(\"Rblpapi_bds_Impl\", PACKAGE = \"Rblpapi\", con_, securities,     field, options_, overrides_, verbose, identity_)\r\n 2: bds_Impl(con, security, field, options, overrides, verbose, identity)\r\n 3: bds(paste(idx, \"Index\"), \"INDX_MWEIGHT_HIST\", overrides = c(END_DT = as.character(format(fdt,     format = \"%Y%m%d\"))), identity = blpID)\r\n```\r\n```\r\n *** caught segfault ***\r\naddress 0x39a000003a9, cause 'memory not mapped'\r\n\r\nTraceback:\r\n 1: .Call(\"Rblpapi_bds_Impl\", PACKAGE = \"Rblpapi\", con_, securities,     field, options_, overrides_, verbose, identity_)\r\n 2: bds_Impl(con, security, field, options, overrides, verbose, identity)\r\n 3: bds(paste(idx, \"Index\"), \"INDX_MWEIGHT_HIST\", overrides = c(END_DT = as.character(format(fdt,     format = \"%Y%m%d\"))), identity = blpID)\r\n```\r\n```\r\n *** caught segfault ***\r\naddress 0x2f9, cause 'memory not mapped'\r\n```\r\n\r\nDoes not happen at the same point in the data job (timeslice), but in the same call.  I tried to run it under strace and ltrace to see what the c++ end of things was/is doing, and it never fails when under strace/ltrace.  \r\n\r\nI've tried it on an AWS Ubuntu machine (R 3.2.3), and on a Centos local machine (R 3.4.1); talking to different SAPI backend servers.  Fails in the same way.  Both running current Rblpapi:\r\n`Rblpapi version 0.3.6 using Blpapi headers 3.8.18.1 and run-time 3.8.18.1.`\r\n\r\nSimplified code to reproduce it (sometimes takes multiple loops, sometimes dies on the 1st call.)\r\n\r\n```\r\nidx <- \"SPR\"\r\nperiod<-30\r\ndt.interval <- period*24*60*60\r\n\r\nst.date <- \"2002-01-02\"\r\nst.dates <- seq(as.POSIXct(st.date), as.POSIXct(Sys.time()), by=dt.interval)\r\n\r\nfor (dt in 1:length(st.dates)) {\r\n    fdt <- format(st.dates[dt], format=\"%Y%m%d\")\r\n    cat(paste(fdt, idx, \"... \\n\"))\r\n    foo <- bds(paste(idx, \"Index\"), \"INDX_MWEIGHT_HIST\",\r\n               overrides=c(\"END_DT\"=as.character(format(fdt, format=\"%Y%m%d\"))),\r\n               identity=blpID)\r\n}\r\n```\r\n\r\nHere's a backtrace from the core file\r\n```\r\n(gdb) bt full\r\n#0  0x00007f726a2a0a3a in R_gc_internal () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#1  0x00007f726a2a3751 in Rf_allocVector3 () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#2  0x00007f725ae0aceb in Vector<unsigned long> (size=140736377739056,\r\n    this=0x7fffbdcd55d0)\r\n    at /usr/lib64/R/library/Rcpp/include/Rcpp/vector/Vector.h:124\r\nNo locals.\r\n#3  allocateDataFrameColumn (fieldT=<optimized out>, n=n@entry=1505)\r\n    at bds.cpp:129\r\n        ans = <optimized out>\r\n#4  0x00007f725ae0c402 in assertColumnDefined (\r\n    lazy_frame=std::map with 0 elements, e=..., n=1505) at bds.cpp:195\r\n        column = <optimized out>\r\n        iter = {\r\n          first = <error reading variable: Cannot access memory at address 0xffffffffffffffe8>, second = }\r\n#5  0x00007f725ae0c62c in bulkArrayToDf (fieldData=...) at bds.cpp:211\r\n        e = {d_handle_p = 0x3bf7b70}\r\n        iter = <optimized out>\r\n        j = 0\r\n        i = 0\r\n        lazy_frame = std::map with 0 elements\r\n#6  0x00007f725ae0d03f in BulkDataResponseToDF (event=...,\r\n    requested_field=\"INDX_MWEIGHT_HIST\",\r\n    response_type=\"ReferenceDataResponse\", verbose=verbose@entry=false)\r\n    at bds.cpp:243\r\n        e = {d_handle_p = 0x1d22a60}\r\n        i = 0\r\n        msgIter = {d_impl_p = 0x3e29020, d_current_p = 0x7f72540297f0}\r\n        msg = {d_handle = <optimized out>, d_elements = {\r\n            d_handle_p = <optimized out>}, d_isCloned = false}\r\n        securityData = {d_handle_p = 0x2966390}\r\n        ans_names = std::vector of length 1, capacity 1 = {\"SPR Index\"}\r\n        ans = {<Rcpp::PreserveStorage<Rcpp::Vector<19, Rcpp::PreserveStorage> >> = {\r\n            data = 0x402a0a8}, <Rcpp::SlotProxyPolicy<Rcpp::Vector<19, Rcpp::PreserveStorage> >> = {<No data fields>}, <Rcpp::AttributeProxyPolicy<Rcpp::Vector<19, Rcpp::PreserveStorage> >> = {<No data fields>}, <Rcpp::NamesProxyPolicy<Rcpp::Vector<19, Rcpp::PreserveStorage> >> = {<No data fields>}, <Rcpp::RObjectMethods<Rcpp::Vector<19, Rcpp::PreserveStorage> >> = {<No data fields>}, <Rcpp::VectorBase<19, true, Rcpp::Vector<19, Rcpp::PreserveStorage> >> = {<Rcpp::traits::expands_to_logical__impl<19>> = {<No data fields>}, <No data fields>}, cache = {\r\n            p = 0x7fffbdcd5a70}}\r\n#7  0x00007f725ae0d87e in bds_Impl (con_=con_@entry=0x359e488,\r\n    securities=std::vector of length 1, capacity 1 = {...},\r\n    field=\"INDX_MWEIGHT_HIST\", options_=options_@entry=0x1976838,\r\n    overrides_=overrides_@entry=0x402a108, verbose=verbose@entry=false,\r\n    identity_=identity_@entry=0x35a1a20) at bds.cpp:282\r\n        msgIter = {d_impl_p = 0x3ca8a20, d_current_p = 0x0}\r\n        event = {d_impl_p = 0x7f725400ee40}\r\n        rdsrv = \"//blp/refdata\"\r\n        refDataService = {d_handle = 0x7f72540b68e0}\r\n        request = {d_handle = 0x2967dc0, d_elements = {d_handle_p = 0x3a2a6b0}}\r\n        session = 0x2edcab0\r\n        field_types = std::vector of length 0, capacity 0\r\n#8  0x00007f725ae0090b in Rblpapi_bds_Impl (con_SEXP=0x359e488,\r\n    securitiesSEXP=<optimized out>, fieldSEXP=<optimized out>,\r\n    options_SEXP=0x1976838, overrides_SEXP=0x402a108,\r\n    verboseSEXP=<optimized out>, identity_SEXP=0x35a1a20) at RcppExports.cpp:72\r\n        securities = <optimized out>\r\n        overrides_ = {x = 0x402a108}\r\n        identity_ = {x = 0x35a1a20}\r\n        rcpp_result_gen = {<Rcpp::PreserveStorage<Rcpp::RObject_Impl<Rcpp::PreserveStorage> >> = {\r\n            data = 0x1976838}, <Rcpp::SlotProxyPolicy<Rcpp::RObject_Impl<Rcpp::PreserveStorage> >> = {<No data fields>}, <Rcpp::AttributeProxyPolicy<Rcpp::RObject_Impl<Rcpp::PreserveStorage> >> = {<No data fields>}, <Rcpp::RObjectMethods<Rcpp::RObject_Impl<Rcpp::PreserveStorage> >> = {<No data fields>}, <No data fields>}\r\n        con_ = {x = 0x359e488}\r\n        field = <optimized out>\r\n        options_ = {x = 0x1976838}\r\n        verbose = <optimized out>\r\n        rcpp_output_type = 0\r\n        rcpp_output_condition = 0x1976838\r\n#9  0x00007f726a22c804 in R_doDotCall () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#10 0x00007f726a22cedd in do_dotcall () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#11 0x00007f726a270831 in Rf_eval () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#12 0x00007f726a272fac in do_begin () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#13 0x00007f726a270605 in Rf_eval () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#14 0x00007f726a27228f in R_execClosure () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#15 0x00007f726a2703d4 in Rf_eval () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#16 0x00007f726a27409e in do_set () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#17 0x00007f726a270605 in Rf_eval () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#18 0x00007f726a272fac in do_begin () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#19 0x00007f726a270605 in Rf_eval () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#20 0x00007f726a27228f in R_execClosure () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#21 0x00007f726a268a35 in bcEval () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#22 0x00007f726a2765ae in R_compileAndExecute () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#23 0x00007f726a276abe in do_for () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#24 0x00007f726a270605 in Rf_eval () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#25 0x00007f726a299e42 in Rf_ReplIteration () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#26 0x00007f726a29a221 in R_ReplConsole () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#27 0x00007f726a29a2df in run_Rmainloop () from /usr/lib64/R/lib/libR.so\r\nNo symbol table info available.\r\n#28 0x000000000040080b in main ()\r\nNo symbol table info available.\r\n(gdb)\r\n```\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/Rblp/Rblpapi/issues/comments/367246988","html_url":"https://github.com/Rblp/Rblpapi/issues/253#issuecomment-367246988","issue_url":"https://api.github.com/repos/Rblp/Rblpapi/issues/253","id":367246988,"user":{"login":"joel23888","id":13341626,"avatar_url":"https://avatars2.githubusercontent.com/u/13341626?v=4","gravatar_id":"","url":"https://api.github.com/users/joel23888","html_url":"https://github.com/joel23888","followers_url":"https://api.github.com/users/joel23888/followers","following_url":"https://api.github.com/users/joel23888/following{/other_user}","gists_url":"https://api.github.com/users/joel23888/gists{/gist_id}","starred_url":"https://api.github.com/users/joel23888/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joel23888/subscriptions","organizations_url":"https://api.github.com/users/joel23888/orgs","repos_url":"https://api.github.com/users/joel23888/repos","events_url":"https://api.github.com/users/joel23888/events{/privacy}","received_events_url":"https://api.github.com/users/joel23888/received_events","type":"User","site_admin":false},"created_at":"2018-02-21T08:20:21Z","updated_at":"2018-02-21T08:20:21Z","author_association":"CONTRIBUTOR","body":"@mayeromain you may want to redirect Rscript to a file so you can review this more easily. If you are using Windows then try something like:\r\n`Rscript.exe --verbose script.R > log.txt 2>&1`"}}
{"actor":{"display_login":"joel23888"}}