{"action":"created","issue":{"url":"https://api.github.com/repos/robjhyndman/forecast/issues/682","repository_url":"https://api.github.com/repos/robjhyndman/forecast","labels_url":"https://api.github.com/repos/robjhyndman/forecast/issues/682/labels{/name}","comments_url":"https://api.github.com/repos/robjhyndman/forecast/issues/682/comments","events_url":"https://api.github.com/repos/robjhyndman/forecast/issues/682/events","html_url":"https://github.com/robjhyndman/forecast/issues/682","id":318252501,"number":682,"title":"forecast with object, xreg, and arima xreg model fails","user":{"login":"blakeflei","id":19981173,"avatar_url":"https://avatars2.githubusercontent.com/u/19981173?v=4","gravatar_id":"","url":"https://api.github.com/users/blakeflei","html_url":"https://github.com/blakeflei","followers_url":"https://api.github.com/users/blakeflei/followers","following_url":"https://api.github.com/users/blakeflei/following{/other_user}","gists_url":"https://api.github.com/users/blakeflei/gists{/gist_id}","starred_url":"https://api.github.com/users/blakeflei/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/blakeflei/subscriptions","organizations_url":"https://api.github.com/users/blakeflei/orgs","repos_url":"https://api.github.com/users/blakeflei/repos","events_url":"https://api.github.com/users/blakeflei/events{/privacy}","received_events_url":"https://api.github.com/users/blakeflei/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-04-27T01:58:16Z","updated_at":"2018-04-27T20:13:50Z","closed_at":null,"author_association":"NONE","body":"Calling forecast on a time series object, xreg, and arima xreg model seems to fail. The goal is:\r\n1 - Split all data into into separate train and test data. \r\n2 - Fit the train set to create train model.\r\n3 - Use the test set to determine error (skipped here for brevity).\r\n4 - Apply the train model to all data to create forecast.\r\n\r\nExample:\r\n```\r\n> library(forecast)\r\n> sessionInfo()\r\nR version 3.5.0 (2018-04-23)\r\nPlatform: x86_64-w64-mingw32/x64 (64-bit)\r\nRunning under: Windows >= 8 x64 (build 9200)\r\n\r\nMatrix products: default\r\n\r\nlocale:\r\n[1] LC_COLLATE=English_United States.1252 \r\n[2] LC_CTYPE=English_United States.1252   \r\n[3] LC_MONETARY=English_United States.1252\r\n[4] LC_NUMERIC=C                          \r\n[5] LC_TIME=English_United States.1252    \r\n\r\nattached base packages:\r\n[1] stats     graphics  grDevices utils     datasets \r\n[6] methods   base     \r\n\r\nother attached packages:\r\n[1] forecast_8.4\r\n\r\nloaded via a namespace (and not attached):\r\n [1] Rcpp_0.12.16      magrittr_1.5      uroot_2.0-9      \r\n [4] munsell_0.4.3     colorspace_1.3-2  lattice_0.20-35  \r\n [7] rlang_0.2.0       quadprog_1.5-5    TTR_0.23-3       \r\n[10] plyr_1.8.4        tools_3.5.0       xts_0.10-2       \r\n[13] nnet_7.3-12       parallel_3.5.0    quantmod_0.4-13  \r\n[16] grid_3.5.0        nlme_3.1-137      timeDate_3043.102\r\n[19] gtable_0.2.0      urca_1.3-0        tseries_0.10-44  \r\n[22] lazyeval_0.2.1    lmtest_0.9-36     tibble_1.4.2     \r\n[25] ggplot2_2.2.1     curl_3.2          fracdiff_1.4-2   \r\n[28] compiler_3.5.0    pillar_1.2.1      scales_0.5.0     \r\n[31] zoo_1.8-1  \r\n```\r\n\r\nGenerate synthetic data. Forecasts using arima with xreg will be attempted on the first column:\r\n```\r\n> num_rows <- 100\r\n> num_cols <- 3\r\n> \r\n> dat <- matrix(abs(rnorm(num_rows)))\r\n> dat[2:length(dat),1] <- dat[2:length(dat)] + dat[1:(length(dat)-1)]*2 \r\n> \r\n> df_in <- matrix(data=NA, nrow=num_rows, ncol=num_cols)\r\n> df_in[,1] <- dat\r\n> df_in[,2] <- c(tail(dat, length(dat) - 2), NA, NA) + rnorm(num_rows, 0, 0.2) \r\n> df_in[,3] <- c(tail(dat, length(dat) - 3), NA, NA, NA)  + rnorm(num_rows, 0, 0.2)\r\n> df_in <- df_in[apply(df_in, 1, function(x) !any(is.na(x))), ] # Drop NA\r\n\r\n> forecast_window <- 5  \r\n```\r\n\r\nSplit into train and test data:\r\n```\r\n> val_test_perc <- 0.2\r\n> \r\n> split_ind <- floor((1-val_test_perc) * dim(df_in)[1])\r\n> train <- df_in[seq_len(split_ind),]\r\n> test <- df_in[-seq_len(split_ind),]\r\n```\r\n\r\nDetermine arima with xreg model using train data:\r\n```\r\n> fit_arima_xreg <- auto.arima(train[,1], xreg=train[,-1])\r\n```\r\n\r\nForecast all exogenous variables using train models on all data:\r\n```\r\n# First create exog train models using train data:\r\n> exog_models <- list()\r\n> for (i in 2:num_cols){\r\n+     exog_models[[i]] <- auto.arima(train[,i])\r\n+ }\r\n> \r\n> # Then forecast exogs using all data and exog train models:\r\n> exog_forcs <- data.frame(matrix(data=NA, nrow=forecast_window, ncol=num_cols))\r\n> for (i in 2:num_cols){\r\n+     exog_curr_forecast <- forecast(df_in[,i], \r\n+                                    model=exog_models[[i]], \r\n+                                    h=forecast_window) \r\n+     exog_forcs[,i] <- exog_curr_forecast[['mean']]\r\n+ }\r\n```\r\n\r\nAttempt to forecast using all data:\r\n```\r\n> forecast(df_in[,1], model=fit_arima_xreg, xreg=exog_forcs[,-1], h=forecast_window)\r\nError in stats::arima(x = x, order = order, seasonal = seasonal, xreg = xreg,  : \r\n  lengths of 'x' and 'xreg' do not match\r\n```\r\n \r\nOnly the model seems to work:\r\n```\r\nforecast(fit_arima_xreg, xreg=exog_forcs[,-1], h=forecast_window)\r\n   Point Forecast      Lo 80    Hi 80      Lo 95    Hi 95\r\n78       2.136355  0.8962271 3.376483  0.2397431 4.032967\r\n79       3.249901  1.7886203 4.711181  1.0150650 5.484737\r\n80       2.897088  1.4199463 4.374230  0.6379947 5.156182\r\n81       1.710781  0.1407135 3.280849 -0.6904302 4.111992\r\n82       1.397953 -0.2342081 3.030114 -1.0982220 3.894128\r\n```\r\n\r\nAttempt to forecast train data (should be same as above):\r\n```\r\n> forecast(train[,1], model=fit_arima_xreg, xreg=exog_forcs[,-1], h=forecast_window)\r\nError in stats::arima(x = x, order = order, seasonal = seasonal, xreg = xreg,  : \r\n  lengths of 'x' and 'xreg' do not match\r\n```"},"comment":{"url":"https://api.github.com/repos/robjhyndman/forecast/issues/comments/385081858","html_url":"https://github.com/robjhyndman/forecast/issues/682#issuecomment-385081858","issue_url":"https://api.github.com/repos/robjhyndman/forecast/issues/682","id":385081858,"user":{"login":"blakeflei","id":19981173,"avatar_url":"https://avatars2.githubusercontent.com/u/19981173?v=4","gravatar_id":"","url":"https://api.github.com/users/blakeflei","html_url":"https://github.com/blakeflei","followers_url":"https://api.github.com/users/blakeflei/followers","following_url":"https://api.github.com/users/blakeflei/following{/other_user}","gists_url":"https://api.github.com/users/blakeflei/gists{/gist_id}","starred_url":"https://api.github.com/users/blakeflei/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/blakeflei/subscriptions","organizations_url":"https://api.github.com/users/blakeflei/orgs","repos_url":"https://api.github.com/users/blakeflei/repos","events_url":"https://api.github.com/users/blakeflei/events{/privacy}","received_events_url":"https://api.github.com/users/blakeflei/received_events","type":"User","site_admin":false},"created_at":"2018-04-27T20:13:50Z","updated_at":"2018-04-27T20:13:50Z","author_association":"NONE","body":"That was probably way too complicated. \r\n\r\nA simpler example:\r\n```\r\n> library(forecast)\r\n> dat <- matrix( c(1.56, 0.12, 0.44, 3.24, 0.64, 0.79, 0.46, \r\n+                  0.41, 0.91, 0.71, 0.66, 2.97, 0.56, 3.25, 3.62),\r\n+                nrow=5, ncol=3)\r\n> \r\n> ex <- matrix( c(2.84, 1.41, 2.78, 2.08, 2.41, 2.73, 2.57, \r\n+                 2.73), \r\n+               nrow=4, ncol=2)\r\n> \r\n> fit <- auto.arima(dat[,1], xreg=dat[,-1])\r\n> \r\n> forecast(dat[,1], model=fit, xreg=ex, h=4)\r\nError in stats::arima(x = x, order = order, seasonal = seasonal, xreg = xreg,  : \r\n  lengths of 'x' and 'xreg' do not match\r\n```"}}
{"actor":{"display_login":"blakeflei"}}