```python
import numpy as np
from matplotlib import pyplot as plt
from scipy.io import loadmat
import sys, os
from utils import *
import math
%matplotlib inline
```

## Building Models of Spike Train Data

### _Overview_
#### In this tutorial, we will be analyzing the firing activity of a retinal ganglion cell in cuture, exposed to low-light and high-light  environments. The aims of the tutorial follow:

* To visualize spike-train data
* To compute and interpret summary statistics
* To build models of the inter-spike-interval distribution as a function of the light level
* To use these models to interpret new data

### Background

#### The scientific question at hand

Neurons in the retina respond to patterns of light. When retinal ganglion cells are grown in culture and held under constant environmental conditions, these neurons exhibit spontaneous firing activity. A researcher extracted the retina from goldfish and sustained them in culture and performed electrophysiological recordings under two conditions: low ambient light levels and high ambient light levels. The aim of the study was to characterize the variability of the goldfish ganglion cell firing due to the common noise source, the background light level. [1]


#### Neural spike trains as point processes

Point processes can be considered, approximately, to be a binary time series. They can be described in terms of the event times,  inter-event intervals or the number of events that occur. When describing a set of spikes according to the inter-event intervals, we define the interspike intervals as the length of time between one event and the next. These processes can exhibit a variety of history dependent behaviors. The simplest type of point process is a poisson process.

***Poisson processes*** are point processes that are memoryless: the probability of an event happening at a particular time is independent on the occurrence or timing of past events. Homogeneous Poisson processes describe irregular sequences of events that lack a temporal structure. There is a theorem that states that a point process is a homogeneous Poisson process with rate $\lambda$ if and only if the inter-event waiting times are independent, identically distributed exponential variables with rate $\lambda$. We will begin this tutorial by considering the exponential distribution as a plausible model for the interspike intervals. 

We will see, however, that the temporal structure of inter-events is relevant because we are often interested in measuring how spiking activity changes with respect to a stimulus. For this reason, the history or previous neural activity influences the spiking activity over time. We will explore this by examining how well the inverse gaussian distribution describes the data.

For a more detailed description of point processes and their use for analyzing neural data, check out Dr. Uri Eden's<div><a href = "http://www.stat.columbia.edu/~liam/teaching/neurostat-fall18/uri-eden-point-process-notes.pdf">point process notes</a></div>.


## Application

In order to assess whether there is a difference between the interspike intervals for the retinal ganglion cells between these two conditions, we will model the distribution of the interspike intervals. 

### Loading the data

The file 'Ch8-spikes-1.mat' in the directory 'Case-Studies-Kramer-Eden/Chapter8' contains a set of spike-times for two retinal ganglion cells, one exposed to low-level ambient light and another to high-level ambient light. 

For this tutorial, we have defined a library called utils with general purpose functions to help assist the learning process. In order to load the data, run the following line of code. You will see that ``retinal_ganglion_data_dict`` is a python dictionary with keys ``SpikesLow`` and ``SpikesHigh``. Both of these lists have units in seconds.


```python
retinal_ganglion_data_path = '../Case-Studies-Kramer-Eden/Chapter8/Ch8-spikes-1.mat'
retinal_ganglion_data_dict = load_retinal_ganglion_data(retinal_ganglion_data_path)
print(retinal_ganglion_data_dict.keys())
spike_train_low = retinal_ganglion_data_dict['SpikesLow']
spike_train_high= retinal_ganglion_data_dict['SpikesHigh']
```

    dict_keys(['SpikesLow', 'SpikesHigh'])


### Exploratory Data Analysis

#### Comparing firing rates

One of the fundamental tenants of neuroscience is that neurons respond to a stimulus by increasing their 'firing rate', the number of spikes that occur per unit time:
<h3></h3>
\begin{align*}FR = \frac{\text{number of spikes}}{\Delta t}\end{align*}
<h3></h3>

Given that we have a dataset for which the neuron was exposed to a higher level of light, we can start by exploring whether the firing rate of this neuron is higher under the high light condition than under the low light condition.

#### Computing the average firing rate. 

In order to compute the average firing rate, we need to find the number of events that occurred over the duration of the experiment. The lists ``spike_train_low``  and ``spike_train_high`` contain the times where these events occurred. Write a function to obtain the number of spikes that occurred for the low and high conditions. Use this function to compute the average firing rate for the duration of the experiment (30 seconds).


```python
def num_spikes(spike_train):
    pass

def mean_FR(spike_train):
    pass
```


```python
mean_FR_high= mean_FR(spike_train_high)
mean_FR_low = mean_FR(spike_train_low)

#print('Average Firing Rate for the low light condition: ', mean_FR_low  , 'spikes per second')
#print('Average Firing Rate for the high light condition:', mean_FR_high , 'spikes per second')

```

#### Visualizing spike trains

The firing rate has a downfall: its value depends on the interval for which it is computed and on the time interval used for its calculation. This can be seen by looking at the raster plots for the neural activity. A raster plot shows the occurrence of a spike for a sequence of times, the temporal duration of the experiment, for instance. This sequence is called a ***spike train***. A neural spike train represents the time series of spikes that occurred during a recording session. 

The following sections show raster plots for the spiking activity for the two conditions. Each row has as x axis the time in seconds, and each vertical blue line represents a spike. 

Take a look at the spike trains below. What do these spike trains suggests as possible hypotheses worth exploring in the data? 

What information is lost in the average firing rate that is suggested by the spike train?


```python
make_raster_plot(spike_train_low, 'Raster plot for the low background light condition')
```


```python
make_raster_plot(spike_train_high, 'Raster plot for the high background light condition')
```

### The interspike intervals 

One possible hypothesis worth exploring is whether there is a difference in the inter-event times. That is, the spike trains suggest that, for the high light condition, there are some segments where the neuron seems to have a higher firing rate, and other segments where the neuron has a lower firing rate. We can explore this by computing the inter-event times, which for neural data are called the interspike intervals. 

In order to compare the inter-event times between the low ambient light condition and the high ambient light condition, we first need to compute the inter-spike intervals. The inter-spike intervals are the waiting times between the spike times. 

**Write a function that takes as input the list of spike times and returns as output the amount of time between each of the spike times**


```python
def compute_inter_spike_intervals(spike_times):
    pass
```

Use the function ``plot_ISIs`` to visualize the interspike intervals for the low and high conditions. This function takes as input ``ISIs``, ``n_bins``, and ``plot_title``, which are respectively the interspike intervals, the number of bins (an integer), and a title (a string). 

Uncomment the following code segment to visualize the empirical distribution of interspike times for each of the light conditions. What do these histograms suggest? 


```python
spike_trains_high = retinal_ganglion_data_dict['SpikesHigh']
spike_trains_low  = retinal_ganglion_data_dict['SpikesLow']
#n_bins = 50
#plot_title1 = 'Inter-Spike-Intervals for the High Light Condition'
#plot_title2 = 'Inter-Spike-Intervals for the Low Light Condition'

#low_ISIs    = compute_inter_spike_intervals(spike_trains_low)
#high_ISIs   = compute_inter_spike_intervals(spike_trains_high)

#plot_ISIs(low_ISIs,  n_bins, plot_title2)
#plot_ISIs(high_ISIs, n_bins, plot_title1)
```




### Building Statistical Models of the Inter-Spike-Intervals

In order to investigate whether there is a difference in the inter-spike-intervals for the two conditions, we need a statistical model. We will investigate which of the following distributions best describes the data. We will explore the **exponential** and **inverse Gaussian** distributions.   

In order to do so, we will compare the model probability density estimates to the normalized histograms of the Inter-Spike Intervals. Let's start with the exponential distribution.

### The exponential distribution

An **exponential random variable** is a continuous random variable on the interval $[0, \infty)$ and has probability density function given by

\begin{align}
f_x(x) = P(0\leq X \leq \infty) & = \int_0^\infty \! \lambda e^{-\lambda x} \, \mathrm{d}x.  
\end{align}

Its cumulative density function is given by

\begin{align}
F_x(x) = P(X \leq x) & = \int_0^x \! 1 - \lambda e^{-\lambda x} \, \mathrm{d}x.  
\end{align}

It is a useful distribution for describing waiting times that are **memoryless**, where the next waiting time can be thought of as being independent of the previous waiting times. Furthermore, it has a **rate parameter** $\lambda > 0$. 
  
The following functions will allow you to visualize the exponential PDF and CDF for different $\lambda$ values. Take a look at how the shape of the distribution changes for different parameters $\lambda$. This parameter is the ***rate*** parameter and must be greater than 0.

**Create a vector of different $\lambda$ parameters and an array of x values. Then, use tue function ``exponential_pdf`` to visualize the PDF and the function ``exponential_cdf`` to visualize the CDF for the corresponding $\lambda$ parameters. Use the function ``plot_exponential_dist`` to visualize the CDF and the PDF by uncommenting the last two lines of code in the next block.**

***Hint: the function <a href = "https://docs.scipy.org/doc/numpy/reference/generated/numpy.arange.html">np.arange</a> is useful for creating arrays within a particular range and at specific increments***


```python


exponential_pdf = lambda x, lam: lam * np.exp(-x*lam)
exponential_cdf = lambda x, lam: 1 - np.exp(-x*lam)

#plot_exponential_dist( exponential_pdf, x_vals, lambdas, 1) 
#plot_exponential_dist( exponential_cdf, x_vals, lambdas, 0)
```

Our goal now is to find an appropriate rate parameter $\lambda$ for which to model the interspike intervals for the two light conditions. In order to explore which $\lambda$ parameters might be reasonable, we can overlay the exponential PDF over the **empirical probability distribution**. To compute the empirical probability distribution, we will define small time bins (1ms) and will count the number of times that a given interspike interval falls within each of these time bins. 

**Compute the empirical probability distribution for 1000ms time bins and overlay the probability density function for the exponential distribution for different $\lambda$s. Which $\lambda$ best fits the observed ISI distribution?**

***Hint: create a vector that goes from 0 to 0.5 in 0.001 increments. Look at the documentation for <a href= "https://docs.scipy.org/doc/numpy-1.15.1/reference/generated/numpy.histogram.html">np.histogram</a> in order to compute the number of times that the interspike intervals fall within each of these bins***


```python
#Your code here
```

### The method of maximum likelihood maximizes the likelihood function

If we make the assumption that the interspike intervals are independent, and that they are exponentially distributed, then the joint probability distribution of the observed data is given by:

\begin{align}
f(x_1,x_2,\cdots,x_n)
& = f(x_1)f(x_2) \cdots f(x_n) \\
& = \lambda e^{-\lambda x_1 } \lambda e^{-\lambda x_2} \cdots \lambda e^{-\lambda x_n }  \\
& = \lambda^n e^{ -\lambda \sum_{i=1}^{n}{x_i} }
\end{align}

Here, we are considering the PDF as a function of the parameter $\lambda$, rather than as a function of the random variable $x$. In this scenario, $f(x|\lambda)$ is considered the **likelihood function** and we will denote it by $L(\lambda)$. The value of $\lambda$ that maximizes the probability of the data is termed the **maximum likelihood estimator** of $\lambda$. Notationally, we write it as $\hat{\lambda}$.  

We can determine the value of $\lambda$ that maximizes the probability of the data through **Maximum Likelihood** methods. That is, we take the derivative of the likelihood function, set it to zero, and solve for $\lambda$. Since often times, the values for $L(\lambda)$ are quite large, it is convenient to take the logarithm and maximize the log-likelihood function $log(L(\lambda)) = \ell(\lambda)$. 

Let's compute the value of $\hat{\lambda}$ analytically:

\begin{align}
\ell (\lambda)
& = log( L(\lambda) ) \\
& = log( \lambda^n e^{ -\lambda \sum_{i=1}^{n}{x_i} } ) \\
& = n log(\lambda) - \lambda \sum_{i=1}^{n}{x_i} \\
\frac{\partial \ell (\lambda) }{\partial \lambda}
& = \frac{n}{\lambda} - \sum_{i=1}^{n}{x_i} = 0 \\
\Rightarrow
\hat{\lambda} &= \frac{n}{\sum_{i=1}^{n}{x_i} } \\
\end{align}

**Create a function called ``log_lik_fun`` that takes as input $\lambda$, a numpy array of lambda values, and $x$, the data, and computes the log-likelihood for the exponential distribution. Then, plot the log-likelihood function $\ell (\lambda)$ for different values of $\lambda$ and compute the maximum likelihood estimate $\hat{\lambda}_{ML}$ for the low-light and the high-light conditions. Compare the analytical estimate to the maximum value of the log-likelihood plot.**

***Hint: for getting the maximum value of the line below, use the function ``np.argmax(log_lik)``, which will return the index corresponding to the maximum value of ``lambda_vals``, where ``lambda_vals`` is a vector of the lambdas for which you are plotting the log-likelihood function***


```python
def log_lik_fun(lambdas, ISIs):
    pass
```

**Now, overlay the probability density function for the exponential distribution with the $\hat{\lambda}_{ML}$ estimates and the binned counts from the previous section. How do these distributions differ?**


```python
#Your code here
```


```python
#Your code here
```

### Scientific inference requires having a notion of precision over the estimates.

Confidence intervals allow for quantifying our certainty about our estimate. 

<p><div align = "center"> "A 95% confidence interval (CI) for a parameter $\theta$ is an interval of the form $(L, U)$, (L for lower, U for upper), where $L = L(X_1, \cdots , X_n)$ and $U = U(X_1, \cdots, X_n)$ are random variables derived from the data and $P(L \le \theta \le U) = 0.95\%$" -Analysis of Neural Data, Brown <it> et. al</it>.</div></p>

This statement implies that if $\theta$ were the true value, then the probability that the interval $(L, U)$ contains the $\theta$ is approximately 95%. This does **not** imply that, based on the data, the probability that $(L, U)$ contains $\theta$ is 95%. Think about this statement. Can you see why the two are different? 

Another way of interpreting confidence intervals is to draw ***N*** random samples independently and to compute the CIs for each sample. If we do this many times, at the limit, approximately 95% of those CIs would contain $\theta$, if $\theta$ is the true value of the estimate. 

Now that we have computed $\hat{\theta}_{ML}$, we want to determine our level of uncertainty over this estimate. We can do so through the **parametric bootstrap**. The parametric bootstrap involves generating pseudo-data from an estimated parametric distribution. 

The algorithm follows.

Let $X = x_1, \dots, x_n$ be a set of data with $N$ datapoints.   

Choose $G$ to be the number of samples of pseudo-data to generate. We want G to be large (i.e. pick 10,000).  

**do**  
For $g = 1:G$

   Draw a random sample $U^{(g)} = u_1, \cdots, U_n$ from the distribution with pdf $f(x_i|\hat{\theta})$.

   Compute $\hat{\theta}^{(g)}_{ML}$ based on the bootstrap sample $U^{(g)}$.

   Set $W^{(g)} = \hat{\lambda}^{(g)}$

**end**
    
Compute $\bar{W} = \frac{1}{G}\sum_{i=1}^{G}{W^{(g)}}$  

Sort $W^{(g)}$ in ascending order

Compute the 95% Confidence intervals as the 25th and 95th element of the sorted vector of $W^{(g)}$

**Perform a parametric boostrap procedure to compute $\hat{\theta}_{ML}$ for the exponential distribution, along with its 95% confidence intervals by implementing the algorithm defined above. Then, plot a histogram of the bootstrap estimates for $\hat{\lambda}$ along with the 95% confidence intervals. Can you think of a way to test whether the model parameters are different for the low light condition versus the high light condition?**


```python
alpha   = 0.05
N_boots = 10000
#X_low   = len(low_ISIs)
#X_high  = len(high_ISIs)

#uci_idx = int(((1-alpha)*N_boots))
#lci_idx = int((alpha * N_boots))

#Your code here
```

### Is the difference between the $\hat{\lambda}_{ML}$ estimates for the low-light and high-light conditions statistically significant?


Is the difference between the $\hat{\lambda}_{ML}$ estimates for the low-light and high-light conditions statistically significant? In order to answer this question, we can perform a **parametric bootstrap procedure**. That is, we will estimate $\lambda$ by drawing N random samples with replacement from both estimated distributions. Each random draw will be a vector of the size of the data. We will compute the ML estimate for each random draw. This will give us a distribution of the ML estimates over the data. We can then compare the two distributions for the ISIs under high light condition and under low light condition. To do so, we can use the numpy function <a href = "https://docs.scipy.org/doc/numpy/reference/generated/numpy.random.choice.html"> np.random.choice </a>, which takes as input a vector, the number of samples, and a boolean corresponding to whether to sample with replacement or not. We want to sample with replacement.

Now that we have a distribution for the ML estimate for each of the conditions, we can compare these distributions to determine whether the difference between the two parameters is statistically significant. 


**Compute the difference between these samples. What are the confidence intervals for the difference between the two distributions? Do the confidence intervals for the difference include 0?** 


```python
#Your code here
```

### Goodness of fit: how well does the model describe the data?

A way of checking whether a set of data comes from, approximately, a particular probability model, is to generate **Quantile-Quantile Plots**. Quantile quantile plots allow for comparing the observed quantiles of the data to the theoretical quantiles of the probability distribution. To do so, we sort

Let $X$ be an array containing the data. To compute the observed quantiles, we sort the data in ascending order, $x_{(1)}, \dots, x_{(r)}$ and then say that $x_{(r)}$  is the quantile of order $\frac{r}{(n+1}$, also called the $\frac{100r}{(n+1)}$ percentile. We want to see whether the data are a sample from a distribution having a CDF $F(x)$ then we may define the set of quantile $y_(r) = F^{-1}\frac{r}{(n+1)}$, for $r = 1, \dots , n$, where $n$ is the number of elements in the data set. 

**Write down the formula for the inverse CDF for the exponential distribution and write a function that takes as input a list and returns the theoretical quantiles for the exponential distribution. Then, plot the sorted items of the data against the theoretical quantiles. Check your understanding of the process by comparing your plot to the line of code below. Is the exponential distribution a good model for the data? Why, or why not?** 


```python
#Your code here

#Complete the following code below to compare it to numpy's probplot function. 
from scipy.stats import probplot
from scipy import stats
#lam_ml_low_isis is the ML estimate for the low interspike intervals. 
#fig = plt.figure()
#ax  = fig.add_subplot(1,1,1)
#res = stats.probplot(low_ISIs, dist=stats.expon, sparams=(lam_ml_low_isis,), plot=ax)
```

### The inverse Gaussian distribution

We know from physiology that neurons have an absolute refractory period. That is, once a neuron spikes, it cannot do so again for a certain amount of time. This means that the spikes are **not** independent events. A distribution that can be used for modelling this is the inverse Gaussian distribution. 

The inverse gaussian distribution has PDF


Am **inverse gaussian random variable** is a continuous random variable on the interval $[0, \infty)$ and has probability density function given by

\begin{align}
f_x(x) = P(0\leq X \leq \infty) & = \int_0^\infty \! \frac{\lambda}{2 \pi x ^3}x^{k-1}e^{\frac{-\lambda(x-\mu)^2}{2 \mu^2 x}}\, \mathrm{d}x.  
\end{align}

Its cumulative density function is given by

\begin{align}
F_x(x) = P(X \leq x) & = \int_0^x \! \Phi\Bigg(\sqrt{\frac{\lambda}{x}} \frac{x}{\mu}-1\Bigg) + e^{\frac{2\lambda}{\mu}} \Phi \Bigg(-\sqrt{ \frac{\lambda}{x} } \Bigg), \mathrm{d}x.  
\end{align}

where $\Phi$ is the <a href = "http://mathworld.wolfram.com/NormalDistributionFunction.html">standard normal gaussian CDF </a>, $\mu > 0$ , $\lambda > 0$ and $x \in (0, \infty)$.


**Generate different $\lambda$ and $\mu$ parameters and visualize the shape of the PDF for the Inverse Gaussian Distribution and for its CDF. How does the shape compare to that of the exponential distribution? Overlay the Probability Density Functions for the two distributions. Which one do you think will describe the data best?**


```python
inv_gauss_pdf = lambda x, lam, mu: (lam/ (2*math.pi*(x**3))) ** 0.5 * np.exp( np.divide( (-lam * ((x - mu)**2)), (2*x*(mu**2))))
inv_gauss_cdf = lambda x, lam, mu: norm.cdf(np.sqrt(np.divide(lam,x)) * (np.divide(x,mu)-1)) + (np.exp(np.divide(2*lam,mu)) * norm.cdf(-np.sqrt(np.divide(lam,x))*(np.divide(x,mu)+1)))
lambdas = [1,0.2,3,1,0.2]
mu_vals = [1,1,1,3,3]
x_vals  = np.linspace(0.000001,3,100)
plot_igauss_dist(inv_gauss_pdf, x_vals, lambdas, mu_vals, 1, [0, 3])
plot_igauss_dist(inv_gauss_cdf, x_vals, lambdas, mu_vals, 0, [0, 3])
```

We can see that the inverse gaussian distribution has two parameters that determine its shape, $\lambda$ and $\mu$. As with the exponential distribution, we can compute the maximum likelihood solution for the model, which will be the pair of parameter estimates $\hat{\mu}$ and $\hat{\lambda}$ that maximize the likelihood of the data.

If we assume that the interspike intervals are independent of eachother, the likelihood function is given by:

$L(\mu, \lambda) = \Pi_{i=1}^{N}\sqrt{\frac{\lambda}{2 \pi x_{i}^3}}\exp{\frac{-\lambda(x_i-\mu)^2}{2x_{i}\mu^2}}$  

The log-likelihood function is then given by

$\ell (\mu, \lambda) = \frac{N}{2} log \frac{\lambda}{2 \pi} = \frac{3}{2} \sum_{i=1}^{N}log(x_i) - \sum_{i=1}^{N} \frac{\lambda (x_i - \mu)^2}{2x_i \mu^2}$

We can now repeat the steps performed above to assess whether the inverse gaussian distribution fits the data well.


1. Derive the maximum likelihood estimators for $\hat{\mu}$ and $\hat{\lambda}$ by taking the partial derivatives of the log-likelihood function with respect of each of these parameters.


2. Use the function above ``inv_gauss_pdf`` to create a model for the inverse gaussian distribution, using the analytical solution to the maximum-likelihood estimates.



3. Assess whether the model fits the data well by ploting the theoretical quantiles for the inverse CDF against the sorted values for both datasets.   
  
  
4. Which of the two models is a better fit to the data?



### Inference

Based on the model that best describes the data, perform the following steps to compare the interspike intervals between the two conditions:

1. Compute the 95% confidence intervals for the maximum likelihood estimates through the parametric bootstrap procedure.


2. Compare the ML estimates for the high and low conditions using the bootstrap procedure implemented for the exponential distribution.

### Decision

Based on the results from the model that best fits the data, what can we conclude about the interspike intervals for the high-light condition? Is their mean firing rate higher or lower than that for the low-light condition? What about the shape of the distribution? Does it suggest that the cell is likelier to fire in bursts with short ISIs in the high-light condition, or is it the opposite?


### Acknowledgements

I am grateful to Dr. Brown for allowing me to base this tutorial on the course notes for <div><a href = "https://ocw.mit.edu/courses/brain-and-cognitive-sciences/9-07-statistics-for-brain-and-cognitive-science-fall-2016/"> Statistics for Brain and Cognitive Sciences.</a></div>

I am grateful to Dr. Kramer for allowing me to use the data from his book Case Studies in neural data analysis. 

I am also grateful to Dr. Sassanfar and Dr. Ordoñez for the oportunity to teach this tutorial. 

### References
  
[1] Ginsburg, K. S., Johnsen, J. A., & Levine, M. W. (1984). Common noise in the firing of neighbouring ganglion cells in goldfish retina. The Journal of physiology, 351(1), 433-450.  
  
[2] Iyengar, S., & Liao, Q. (1997). Modeling neural activity using the generalized inverse Gaussian distribution. Biological Cybernetics, 77(4), 289-295.

  
[3] Levine, M. W. (1991). The distribution of the intervals between neural impulses in the maintained discharges of retinal ganglion cells. Biological cybernetics, 65(6), 459-467.  

[4] Eden, Uri T., Kramer, Mark A. (2016). Case studies in neural data analysis. MIT Press.

[5] Kass, R. E., Eden, U. T., & Brown, E. N. (2014). Analysis of neural data (Vol. 491). New York: Springer.

3f0ce696b48dbf854bf346d51e2327652f944acc
ipynb
Jupyter Notebook
modeling_interspike_intervals.ipynb
josefinacmenendez/IQ_BIO_Workshop_2019
c3377c891102f2ffa8e3bd899ffcb8310ab6faef
MIT
modeling_interspike_intervals.ipynb
josefinacmenendez/IQ_BIO_Workshop_2019
c3377c891102f2ffa8e3bd899ffcb8310ab6faef
MIT
modeling_interspike_intervals.ipynb
josefinacmenendez/IQ_BIO_Workshop_2019
c3377c891102f2ffa8e3bd899ffcb8310ab6faef
MIT
2019-07-19T15:51:28.000Z
2019-07-19T15:51:28.000Z
Qwen/Qwen-72B
1. YES
2. YES