<a href="https://colab.research.google.com/github/YenLinWu/Time_Series_Model/blob/main/Materials/Introduction_to_Time_Series_Models.ipynb" target="_parent"></a>

# Introduction to Time Series Models

# <font color="#00dd00">**Autoregressive (AR) Models**</font> 

$$ y_t = \beta + \epsilon_t + \sum_{i=1}^{p} \phi_i y_{t-i} $$  
where $\beta$ is a constant, $\epsilon_t$ is the white noise at time t, and $p$ is the number of time lags to regress on. We refer to this as an AR($p$) model, an autoregressive model of order $p$.     

* Forecast the variable of interest using a linear combination of past values of the variable.  
* The term "Autoregression" indicates that it is a regression of the variable against itself.    
* We normally restrict autoregressive models to <font color="#00dd00">**stationary data**</font>, in which case some constraints on the values of the parameters are required.   
  * AR($1$) : $-1< \phi_1 < 1$.
  * AR($2$) : $-1< \phi_2 < 1$, $\phi_1+\phi_2 < 1$, $\phi_2-\phi_1 < 1$.   

Reference : [Autoregressive models](https://otexts.com/fpp2/AR.html#AR)

# <font color="#00dd00">**Moving average (MA) Models**</font>   


$$ y_t = \beta + \epsilon_t + \sum_{i=1}^{q} \theta_i \epsilon_{t-i} $$   
where $\beta$ is a constant, $\epsilon_t$ is the white noise at time t, and $q$ is the number of time lags of the error term to regress on. We refer to this as an MA($q$) model, a moving average model of order $q$. 
* Each value of $y_t$ can be thought of as a weighted moving average of the past few forecast errors.   
* It is possible to write any stationary AR($p$) model as an MA($\infty$) model. For example, AR($1$) model  
\begin{equation}  
\begin{split}
  y_t &= \epsilon_t + \phi_1 y_{t-1} \\
      &= \epsilon_t + \phi_1 (\epsilon_{t-1} + \phi_1 y_{t-2}) \\ 
      &= \epsilon_t + \phi_1^2 y_{t-2} + \phi_1 \epsilon_{t-1} \\ 
      &= \epsilon_t + \phi_1^3 y_{t-3} + \phi_1^2 y_{t-2} + \phi_1 \epsilon_{t-1} \\ 
      &= \text{...... ,} \\   \\
\end{split}
\end{equation} 
if $|\phi_1|<1$, then we obtain MA($\infty$)
$$y_t=\epsilon_t+\phi_1 \epsilon_{t-1}+\phi_1^2 \epsilon_{t-2} + \phi_1^3 \epsilon_{t_3} + \dots \text{ .}$$    
* On the other hand, if we impose some constraints on the MA parameters (MA model is called **invertible**), we can write any invertible MA($q$) process as an AR($\infty$) process. 

Reference : [Moving average models](https://otexts.com/fpp2/MA.html#MA)

# <font color="#00dd00">**Non-seasonal ARIMA models**</font>

ARIMA is an acronym for AutoRegressive Integrated Moving Average.  
$$y_{t}^{'} = \beta+\phi_1 y_{t-1}^{'}+\dots+\phi_p y_{t-p}^{'}+\theta_1 \epsilon_{t-1}+\dots+\theta_p \epsilon_{t-q}+\epsilon_{t} $$  
where $y_t^{'}$ is the differenced series (it may have been differenced more than once). We call this an ARIMA($p$, $d$, $q$) model, where $p$ is the order of the autoregressive part, $d$ is the degree of first differencing involved, and $q$ is the order of the moving average part.

* The same stationarity and invertibility conditions that are used for autoregressive and moving average models also apply to an ARIMA model.  
* The constant $\beta$ has an important effect on the long-term forecasts obtained from these models.  
  * if $\beta=0$ and $d=0$, the long-term forecasts will go to zero. 
  * if $\beta=0$ and $d=1$, the long-term forecasts will go to a non-zero constant.  
  * if $\beta=0$ and $d=2$, the long-term forecasts will follow a straight line.  
  * if $\beta\neq0$ and $d=0$, the long-term forecasts will go to the mean of the data.  
  * if $\beta\neq0$ and $d=1$, the long-term forecasts will follow a straight line.  
  * if $\beta\neq0$ and $d=2$, the long-term forecasts will follow a quadratic trend.   
* <font color="#dddd00">The value of $d$ also has an effect on the prediction intervals</font> — the higher the value of $d$, the more rapidly the prediction intervals increase in size.   
* <font color="#dddd00">The value of $p$ is important if the data show cycles.</font> To obtain cyclic forecasts, it is necessary to have $p\geq2$, along with some additional conditions on the parameters.  
* It is sometimes possible to use the <font color="#00dd00">**ACF**</font> plot, and the closely related <font color="#00dd00">**PACF**</font> plot, to determine appropriate values for $p$ and $q$.  

Reference : [Non-seasonal ARIMA models](https://otexts.com/fpp2/non-seasonal-arima.html#non-seasonal-arima)
  

# <font color="#00dd00">**SARIMAX Models**</font>  

SARIMAX (Seasonal Auto-Regressive Integrated Moving Average with eXogenous factors) is an updated version of the ARIMA model. <font color="#dddd00">SARIMAX includes **seasonal effects** (季節效應) and **eXogenous factors** (外部因素) with the autoregressive and moving average component in the model.</font> SARIMAX model takes seasonality into account by essentially applying an ARIMA model to lags that are integer multiples of seasonality. Once the seasonality is modeled, an ARIMA model is applied to the leftover to capture non-seasonal structure. One shorthand notation for SARIMAX models is: 
$$ \text{SARIMAX( }p, d, q\text{ )}\times\text{( }P, D, Q, S\text{ )}$$
where $p$ is the order of the non-seasonal AR part,   
&emsp;&emsp;&ensp; $d$ is the non-seasonal differencing,   
&emsp;&emsp;&ensp; $q$ is the order of the non-seasonal MA part,   
&emsp;&emsp;&ensp; $P$ is the order of the seasonal AR part,   
&emsp;&emsp;&ensp; $D$ is the seasonal differencing,   
&emsp;&emsp;&ensp; $Q$ is the order of the seasonal MA part, and   
&emsp;&emsp;&ensp; $S$ is the length of repeating seasonal pattern.    
</br>    
* The $S$ parameter influences the $P$, $D$, and $Q$ parameters. For example, <font color="#dddd00">an $S$ of 12 for monthly data suggests a yearly seasonal cycle</font>.   
* The SARIMAX model can subsume (包含) the AR, MA, and ARIMA models via model configuration parameters.  
* The trend and seasonal hyperparameters of the model can be configured by analyzing [<font color="#00dd00">autocorrelation</font>](https://colab.research.google.com/github/YenLinWu/Time_Series_Model/blob/main/Materials/Stationarity_Differencing_and_Autocorrelation.ipynb) and [<font color="#00dd00">partial autocorrelation plots</font>](https://colab.research.google.com/github/YenLinWu/Time_Series_Model/blob/main/Materials/Stationarity_Differencing_and_Autocorrelation.ipynb), and this can take some expertise.   
<font color="#dddd00">An alternative approach is to <font color="#00dd00">**grid search**</font> a suite of model configurations and discover which configurations work best for a specific univariate time series.</font> This approach can be faster on modern computers than an analysis process and can reveal surprising findings that might not be obvious and result in lower forecast error.  
> *Seasonal ARIMA models can potentially have a large number of parameters and combinations of terms. Therefore, it is appropriate to try out a wide range of models when fitting to data and choose a best fitting model using an appropriate criterion …*   
Quote : Pages 143-144, [Introductory Time Series with R](https://link.springer.com/book/10.1007/978-0-387-88698-5), Paul S.P. Cowpertwait and Andrew V. Metcalfe, 2009.   




07c565a73a040fa17363c6570ecdc0de4d03f8d5
ipynb
Jupyter Notebook
Materials/Introduction_to_Time_Series_Models.ipynb
YenLinWu/Time_Series_Model
c8bbbc2e5121be4f646b29742ebd525683ab90ff
MIT
Materials/Introduction_to_Time_Series_Models.ipynb
YenLinWu/Time_Series_Model
c8bbbc2e5121be4f646b29742ebd525683ab90ff
MIT
Materials/Introduction_to_Time_Series_Models.ipynb
YenLinWu/Time_Series_Model
c8bbbc2e5121be4f646b29742ebd525683ab90ff
MIT
Qwen/Qwen-72B
1. YES
2. YES