# Primitive model of electrolytes: NaCl in continuum solvent

In this example we explore the possibility for setting a background dielectric constant for simulating (soft) particles in an implicit solvent.

- Can $\epsilon_r$ be set for PME/Ewald? Nothing found in the documentation so far except for scaling charges
- Use a custom nonbonded ff instead?
- GBSA forces are automatically added when a `<GBSAOBCForce>` section is encountered in .xml file
- For ideas for rigid bodies, see openmm forum: https://simtk.org/forums/viewtopic.php?f=161&t=5841&p=14836&hilit=rigid#p14836

```python
%matplotlib inline
from __future__ import print_function
from simtk.openmm import app
import simtk.openmm as mm
from simtk import unit
from sys import stdout, exit
import numpy as np
import math
import mdtraj as mdtraj

cutoff           = 50*unit.angstrom
temperature      = 300*unit.kelvin
useMinimize      = False
epsilon_r        = 80.
steps_eq         = 5000
steps_production = 4e5

pdb = app.PDBFile('nacl.pdb')
forcefield = app.ForceField('cg.xml')

system = forcefield.createSystem(pdb.topology, nonbondedMethod=app.Ewald,
                nonbondedCutoff=cutoff, ewaldErrorTolerance=0.1)

# scale charges
def scaleCharges(system, scale=1.0):
    """
    Scale charges in all appropriate force classes of system. This can
    be used to simulate a system with a different background dielectric
    by setting scale=1/sqrt(epsilon_r), for example.
    """
    for force in system.getForces():
        if 'getParticleParameters' in dir(force):
            for i in range(force.getNumParticles()):
                charge, sigma, epsilon = force.getParticleParameters(i)
                force.setParticleParameters(i, scale*charge, sigma, epsilon)
        if 'getExceptionParameters' in dir(force):
            for i in range(force.getNumExceptions()):
                p1, p2, chargeProd, sigma, epsilon = force.getExceptionParameters(i)
                force.setExceptionParameters(i, p1, p2, chargeProd*(scale**2), sigma, epsilon)

scaleCharges(system, scale=1/math.sqrt(epsilon_r))

# set dielectric constant (no effect on pairwise-interactions)
for i in system.getForces():
  if isinstance(i, mm.NonbondedForce):
    i.setReactionFieldDielectric(epsilon_r)
    print ('RF dielectric = ', i.getReactionFieldDielectric())

integrator = mm.LangevinIntegrator(
    temperature,
    1.0/unit.picoseconds,
    20.0*unit.femtoseconds)

integrator.setConstraintTolerance(0.00001)

simulation = app.Simulation(pdb.topology, system, integrator)
simulation.context.setPositions(pdb.positions)

if useMinimize:
  print('Minimizing...')
  simulation.minimizeEnergy()

print('Equilibrating...')
simulation.context.setVelocitiesToTemperature(300*unit.kelvin)
simulation.step(steps_eq)

steps_total = steps_eq + steps_production
simulation.reporters.append(mdtraj.reporters.HDF5Reporter('trajectory.h5', 100))
simulation.reporters.append(app.PDBReporter('trajectory.pdb', 1000))
simulation.reporters.append(app.StateDataReporter(stdout, int(steps_total/10), step=True,
    potentialEnergy=True, temperature=True, progress=True, remainingTime=False,
    speed=True, totalSteps=steps_total, volume=True, separator='\t'))

print('Production...')
simulation.step(steps_production)

print('Done!')
```

    RF dielectric =  80.0
    Equilibrating...
    Production...
    #"Progress (%)"	"Step"	"Potential Energy (kJ/mole)"	"Temperature (K)"	"Box Volume (nm^3)"	"Speed (ns/day)"
    10.0%	40500	-9.41419052737	389.559018631	1000.0	0
    20.0%	81000	-5.38719238931	323.348955062	1000.0	5.51e+03
    30.0%	121500	-13.2359821502	358.396576464	1000.0	5.5e+03
    40.0%	162000	-10.0617166891	256.047483629	1000.0	5.49e+03
    50.0%	202500	-9.76001213709	312.319746817	1000.0	5.39e+03
    60.0%	243000	-10.6731361359	308.124138624	1000.0	5.41e+03
    70.0%	283500	-9.8400834653	432.585410388	1000.0	5.42e+03
    80.0%	324000	-17.2790620144	255.991047233	1000.0	5.43e+03
    90.0%	364500	-14.271373764	273.278255592	1000.0	5.43e+03
    100.0%	405000	-19.5669405017	305.142845317	1000.0	5.43e+03
    Done!

### Plot and compare with Debye-Hückel

Calculate the radial distribution function, $g(r)=\exp\left(-w(r)/k_BT\right)$, between Na+ and Cl- and compare
the result with the Debye-Hückel potential for the interaction free energy between two ions in an aqueous salt solution,

\begin{equation}
w(r) / k_BT = \frac{\lambda_Bz_1z_2}{r}e^{-r/\lambda_D}
\end{equation}

where
$\lambda_B=e^2/4\pi\epsilon_0\epsilon_rk_BT$ and
$\lambda_D=(4\pi\lambda_B\sum \rho_iz_i^2)^{-1/2}$ are the Bjerrum and Debye lengths, respectively. $\rho_i$ is the number density of the $i$th ion.

```python
import matplotlib
import matplotlib.pyplot as plt

# volume, conc., bjerrum & debye length
epsilon_0 = 8.8541878176e-12*unit.farad/unit.meter
N         = simulation.system.getNumParticles()
V         = simulation.topology.getUnitCellDimensions()[0]**3
lB        = (unit.elementary_charge**2 /
            (4*math.pi*epsilon_0*epsilon_r*unit.BOLTZMANN_CONSTANT_kB*temperature))\
            .in_units_of(unit.nanometer)
lD        = 1/unit.sqrt(4*math.pi*lB*N/V)

print('Salt concentration = ', 0.5*(N/unit.AVOGADRO_CONSTANT_NA/V).in_units_of(unit.molar))
print('Bjerrum length     = ', lB)
print('Debye length       = ', lD)

# open trajectory and plot rdf
simulation.reporters[0].close()
traj = mdtraj.load('trajectory.h5')
r,g = mdtraj.compute_rdf(traj, traj.topology.select_pairs('name Na', 'name Cl'),\
                         r_range=[0.2,5], bin_width=0.025)
r=r*unit.nanometer

# plot simulated Na-Cl pmf (red) vs. Debye-Huckel (blue)
plt.xlabel('$r$/nm')
plt.ylabel('$w(r)/k_BT$')
plt.plot(r, -np.log(g), 'r.')
plt.plot(r, -lB / r * np.exp( -r/lD ), 'b-')
```

```python

```


##### Copyright 2018 The TensorFlow Probability Authors.

Licensed under the Apache License, Version 2.0 (the "License");

```
#@title Licensed under the Apache License, Version 2.0 (the "License"); { display-mode: "form" }
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
```

# Gaussian Process Regression in TensorFlow Probability

    View on TensorFlow.org

    Run in Google Colab

    View source on GitHub

    Download notebook

In this colab, we explore Gaussian process regression using
TensorFlow and TensorFlow Probability. We generate some noisy observations from
some known functions and fit GP models to those data. We then sample from the GP
posterior and plot the sampled function values over grids in their domains.

## Background
Let $\mathcal{X}$ be any set. A *Gaussian process*
(GP) is a collection of random variables indexed by $\mathcal{X}$ such that if
$\{X_1, \ldots, X_n\} \subset \mathcal{X}$ is any finite subset, the marginal density
$p(X_1 = x_1, \ldots, X_n = x_n)$ is multivariate Gaussian. Any Gaussian
distribution is completely specified by its first and second central moments
(mean and covariance), and GP's are no exception. We can specify a GP completely
in terms of its mean function $\mu : \mathcal{X} \to \mathbb{R}$ and covariance function
$k : \mathcal{X} \times \mathcal{X} \to \mathbb{R}$. Most of the expressive power of GP's is encapsulated
in the choice of covariance function. For various reasons, the covariance
function is also referred to as a *kernel function*. It is required only to be
symmetric and positive-definite (see [Ch. 4 of Rasmussen & Williams](
http://www.gaussianprocess.org/gpml/chapters/RW4.pdf)). Below we make use of the
ExponentiatedQuadratic covariance kernel. Its form is

$$
k(x, x') := \sigma^2 \exp \left( \frac{\|x - x'\|^2}{\lambda^2} \right)
$$

where $\sigma^2$ is called the 'amplitude' and $\lambda$ the *length scale*.
The kernel parameters can be selected via a maximum likelihood optimization
procedure.

A full sample from a GP comprises a real-valued function over the entire space
$\mathcal{X}$ and is in practice impractical to realize; often one chooses a set of
points at which to observe a sample and draws function values at these points.
This is achieved by sampling from an appropriate (finite-dimensional)
multi-variate Gaussian.

Note that, according to the above definition, any finite-dimensional
multivariate Gaussian distribution is also a Gaussian process. Usually, when
one refers to a GP, it is implicit that the index set is some $\mathbb{R}^n$
and we will indeed make this assumption here.

A common application of Gaussian processes in machine learning is Gaussian
process regression. The idea is that we wish to estimate an unknown function
given noisy observations $\{y_1, \ldots, y_N\}$ of the function at a finite
number of points $\{x_1, \ldots x_N\}.$ We imagine a generative process

$$
\begin{align}
f \sim \: & \textsf{GaussianProcess}\left(
    \text{mean_fn}=\mu(x),
    \text{covariance_fn}=k(x, x')\right) \\
y_i \sim \: & \textsf{Normal}\left(
    \text{loc}=f(x_i),
    \text{scale}=\sigma\right), i = 1, \ldots, N
\end{align}
$$

As noted above, the sampled function is impossible to compute, since we would
require its value at an infinite number of points. Instead, one considers a
finite sample from a multivariate Gaussian.

$$
  \begin{gather}
    \begin{bmatrix}
      f(x_1) \\
      \vdots \\
      f(x_N)
    \end{bmatrix}
    \sim
    \textsf{MultivariateNormal} \left( \:
      \text{loc}=
      \begin{bmatrix}
        \mu(x_1) \\
        \vdots \\
        \mu(x_N)
      \end{bmatrix} \:,\:
      \text{scale}=
      \begin{bmatrix}
        k(x_1, x_1) & \cdots & k(x_1, x_N) \\
        \vdots & \ddots & \vdots \\
        k(x_N, x_1) & \cdots & k(x_N, x_N) \\
      \end{bmatrix}^{1/2}
    \: \right)
  \end{gather} \\
  y_i \sim \textsf{Normal} \left(
      \text{loc}=f(x_i),
      \text{scale}=\sigma
  \right)
$$

Note the exponent $\frac{1}{2}$ on the covariance matrix: this denotes a
Cholesky decomposition. Comptuing the Cholesky is necessary because the MVN is
a location-scale family distribution. Unfortunately the Cholesky decomposition
is computationally expensive, taking $O(N^3)$ time and $O(N^2)$ space. Much of
the GP literature is focused on dealing with this seemingly innocuous little
exponent.

It is common to take the prior mean function to be constant, often zero. Also,
some notational conventions are convenient. One often writes $\mathbf{f}$ for the
finite vector of sampled function values. A number of interesting notations are
used for the covariance matrix resulting from the application of $k$ to pairs of
inputs. Following [(Quiñonero-Candela, 2005)][QuinoneroCandela2005], we note
that the components of the matrix are covariances of function values at
particular input points. Thus we can denote the covariance matrix as $K_{AB}$
where $A$ and $B$ are some indicators of the collection of function values along
the given matrix dimensions.

[QuinoneroCandela2005]: http://www.jmlr.org/papers/volume6/quinonero-candela05a/quinonero-candela05a.pdf

For example, given observed data $(\mathbf{x}, \mathbf{y})$ with implied latent function
values $\mathbf{f}$, we can write

$$
K_{\mathbf{f},\mathbf{f}} = \begin{bmatrix}
  k(x_1, x_1) & \cdots & k(x_1, x_N) \\
  \vdots & \ddots & \vdots \\
  k(x_N, x_1) & \cdots & k(x_N, x_N) \\
\end{bmatrix}
$$

Similarly, we can mix sets of inputs, as in

$$
K_{\mathbf{f},*} = \begin{bmatrix}
  k(x_1, x^*_1) & \cdots & k(x_1, x^*_T) \\
  \vdots & \ddots & \vdots \\
  k(x_N, x^*_1) & \cdots & k(x_N, x^*_T) \\
\end{bmatrix}
$$

where we suppose there are $N$ training inputs, and $T$ test inputs. The above
generative process may then be written compactly as

$$
\begin{align}
\mathbf{f} \sim \: & \textsf{MultivariateNormal} \left(
        \text{loc}=\mathbf{0},
        \text{scale}=K_{\mathbf{f},\mathbf{f}}^{1/2}
    \right) \\
y_i \sim \: & \textsf{Normal} \left(
    \text{loc}=f_i,
    \text{scale}=\sigma \right), i = 1, \ldots, N
\end{align}
$$

The sampling operation in the first line yields a finite set of $N$ function
values from a multivariate Gaussian -- *not an entire function as in the above
GP draw notation*. The second line describes a collection of $N$ draws from
*univariate* Gaussians centered at the various function values, with fixed
observation noise $\sigma^2$.

With the above generative model in place, we can proceed to consider the
posterior inference problem. This yields a posterior distribution over function
values at a new set of test points, conditioned on the observed noisy data from
the process above.

With the above notation in place, we can compactly write the posterior
predictive distribution over future (noisy) observations conditional on
corresponding inputs and training data as follows (for more details, see §2.2 of
[Rasmussen & Williams](http://www.gaussianprocess.org/gpml/)).

$$
\mathbf{y}^* \mid \mathbf{x}^*, \mathbf{x}, \mathbf{y} \sim \textsf{Normal} \left(
    \text{loc}=\mathbf{\mu}^*,
    \text{scale}=(\Sigma^*)^{1/2}
\right),
$$

where

$$
\mathbf{\mu}^* = K_{*,\mathbf{f}}\left(K_{\mathbf{f},\mathbf{f}} + \sigma^2 I \right)^{-1} \mathbf{y}
$$

and

$$
\Sigma^* = K_{*,*} - K_{*,\mathbf{f}}
    \left(K_{\mathbf{f},\mathbf{f}} + \sigma^2 I \right)^{-1} K_{\mathbf{f},*}
$$

## Imports

```
import time

import numpy as np
import matplotlib.pyplot as plt
import tensorflow.compat.v2 as tf
import tensorflow_probability as tfp
tfb = tfp.bijectors
tfd = tfp.distributions
tfk = tfp.math.psd_kernels
tf.enable_v2_behavior()

from mpl_toolkits.mplot3d import Axes3D
%pylab inline
# Configure plot defaults
plt.rcParams['axes.facecolor'] = 'white'
plt.rcParams['grid.color'] = '#666666'
%config InlineBackend.figure_format = 'png'
```

    Populating the interactive namespace from numpy and matplotlib

## Example: Exact GP Regression on Noisy Sinusoidal Data
Here we generate training data from a noisy sinusoid, then sample a bunch of
curves from the posterior of the GP regression model. We use
[Adam](https://arxiv.org/abs/1412.6980) to optimize the kernel hyperparameters
(we minimize the negative log likelihood of the data under the prior). We
plot the training curve, followed by the true function and the posterior
samples.

```
def sinusoid(x):
  return np.sin(3 * np.pi * x[..., 0])

def generate_1d_data(num_training_points, observation_noise_variance):
  """Generate noisy sinusoidal observations at a random set of points.

  Returns:
     observation_index_points, observations
  """
  index_points_ = np.random.uniform(-1., 1., (num_training_points, 1))
  index_points_ = index_points_.astype(np.float64)
  # y = f(x) + noise
  observations_ = (sinusoid(index_points_) +
                   np.random.normal(loc=0,
                                    scale=np.sqrt(observation_noise_variance),
                                    size=(num_training_points)))
  return index_points_, observations_
```

```
# Generate training data with a known noise level (we'll later try to recover
# this value from the data).
NUM_TRAINING_POINTS = 100
observation_index_points_, observations_ = generate_1d_data(
    num_training_points=NUM_TRAINING_POINTS,
    observation_noise_variance=.1)
```

We'll put priors on the kernel hyperparameters, and write the joint distribution of the hyperparameters and observed data using `tfd.JointDistributionNamed`.

```
def build_gp(amplitude, length_scale, observation_noise_variance):
  """Defines the conditional dist. of GP outputs, given kernel parameters."""

  # Create the covariance kernel, which will be shared between the prior (which we
  # use for maximum likelihood training) and the posterior (which we use for
  # posterior predictive sampling)
  kernel = tfk.ExponentiatedQuadratic(amplitude, length_scale)

  # Create the GP prior distribution, which we will use to train the model
  # parameters.
  return tfd.GaussianProcess(
      kernel=kernel,
      index_points=observation_index_points_,
      observation_noise_variance=observation_noise_variance)

gp_joint_model = tfd.JointDistributionNamed({
    'amplitude': tfd.LogNormal(loc=0., scale=np.float64(1.)),
    'length_scale': tfd.LogNormal(loc=0., scale=np.float64(1.)),
    'observation_noise_variance': tfd.LogNormal(loc=0., scale=np.float64(1.)),
    'observations': build_gp,
})
```

We can sanity-check our implementation by verifying that we can sample from the prior, and compute the log-density of a sample.

```
x = gp_joint_model.sample()
lp = gp_joint_model.log_prob(x)

print("sampled {}".format(x))
print("log_prob of sample: {}".format(lp))
```

    sampled {'observation_noise_variance': <tf.Tensor: shape=(), dtype=float64, numpy=2.067952217184325>, 'length_scale': <tf.Tensor: shape=(), dtype=float64, numpy=1.154435715487831>, 'amplitude': <tf.Tensor: shape=(), dtype=float64, numpy=5.383850737703549>, 'observations': <tf.Tensor: shape=(100,), dtype=float64, numpy=
    array([-2.37070577, -2.05363838, -0.95152824,  3.73509388, -0.2912646 ,
            0.46112342, -1.98018513, -2.10295857, -1.33589756, -2.23027226,
           -2.25081374, -0.89450835, -2.54196452,  1.46621647,  2.32016193,
            5.82399989,  2.27241034, -0.67523432, -1.89150197, -1.39834474,
           -2.33954116,  0.7785609 , -1.42763627, -0.57389025, -0.18226098,
           -3.45098732,  0.27986652, -3.64532398, -1.28635204, -2.42362875,
            0.01107288, -2.53222176, -2.0886136 , -5.54047694, -2.18389607,
           -1.11665628, -3.07161217, -2.06070336, -0.84464262,  1.29238438,
           -0.64973999, -2.63805504, -3.93317576,  0.65546645,  2.24721181,
           -0.73403676,  5.31628298, -1.2208384 ,  4.77782252, -1.42978168,
           -3.3089274 ,  3.25370494,  3.02117591, -1.54862932, -1.07360811,
            1.2004856 , -4.3017773 , -4.95787789, -1.95245901, -2.15960839,
           -3.78592731, -1.74096185,  3.54891595,  0.56294143,  1.15288455,
           -0.77323696,  2.34430694, -1.05302007, -0.7514684 , -0.98321063,
           -3.01300144, -3.00033274,  0.44200837,  0.45060886, -1.84497318,
           -1.89616746, -2.15647664, -2.65672581, -3.65493379,  1.70923375,
           -3.88695218, -0.05151283,  4.51906677, -2.28117003,  3.03032793,
           -1.47713194, -0.35625273,  3.73501587, -2.09328047, -0.60665614,
           -0.78177188, -0.67298545,  2.97436033, -0.29407932,  2.98482427,
           -1.54951178,  2.79206821,  4.2225733 ,  2.56265198,  2.80373284])>}
    log_prob of sample: -194.96442183797524

Now let's optimize to find the parameter values with highest posterior probability. We'll define a variable for each parameter, and constrain their values to be positive.

```
# Create the trainable model parameters, which we'll subsequently optimize.
# Note that we constrain them to be strictly positive.

constrain_positive = tfb.Shift(np.finfo(np.float64).tiny)(tfb.Exp())

amplitude_var = tfp.util.TransformedVariable(
    initial_value=1.,
    bijector=constrain_positive,
    name='amplitude',
    dtype=np.float64)

length_scale_var = tfp.util.TransformedVariable(
    initial_value=1.,
    bijector=constrain_positive,
    name='length_scale',
    dtype=np.float64)

observation_noise_variance_var = tfp.util.TransformedVariable(
    initial_value=1.,
    bijector=constrain_positive,
    name='observation_noise_variance_var',
    dtype=np.float64)

trainable_variables = [v.trainable_variables[0] for v in
                       [amplitude_var,
                       length_scale_var,
                       observation_noise_variance_var]]
```

To condition the model on our observed data, we'll define a `target_log_prob` function, which takes the (still to be inferred) kernel hyperparameters.

```
def target_log_prob(amplitude, length_scale, observation_noise_variance):
  return gp_joint_model.log_prob({
      'amplitude': amplitude,
      'length_scale': length_scale,
      'observation_noise_variance': observation_noise_variance,
      'observations': observations_
  })
```

```
# Now we optimize the model parameters.
num_iters = 1000
optimizer = tf.optimizers.Adam(learning_rate=.01)

# Use `tf.function` to trace the loss for more efficient evaluation.
@tf.function(autograph=False, jit_compile=False)
def train_model():
  with tf.GradientTape() as tape:
    loss = -target_log_prob(amplitude_var, length_scale_var,
                            observation_noise_variance_var)
  grads = tape.gradient(loss, trainable_variables)
  optimizer.apply_gradients(zip(grads, trainable_variables))
  return loss

# Store the likelihood values during training, so we can plot the progress
lls_ = np.zeros(num_iters, np.float64)
for i in range(num_iters):
  loss = train_model()
  lls_[i] = loss

print('Trained parameters:')
print('amplitude: {}'.format(amplitude_var._value().numpy()))
print('length_scale: {}'.format(length_scale_var._value().numpy()))
print('observation_noise_variance: {}'.format(observation_noise_variance_var._value().numpy()))
```

    Trained parameters:
    amplitude: 0.9176153445125278
    length_scale: 0.18444082442910079
    observation_noise_variance: 0.0880273312850989

```
# Plot the loss evolution
plt.figure(figsize=(12, 4))
plt.plot(lls_)
plt.xlabel("Training iteration")
plt.ylabel("Log marginal likelihood")
plt.show()
```

```
# Having trained the model, we'd like to sample from the posterior conditioned
# on observations. We'd like the samples to be at points other than the training
# inputs.
predictive_index_points_ = np.linspace(-1.2, 1.2, 200, dtype=np.float64)
# Reshape to [200, 1] -- 1 is the dimensionality of the feature space.
predictive_index_points_ = predictive_index_points_[..., np.newaxis]

optimized_kernel = tfk.ExponentiatedQuadratic(amplitude_var, length_scale_var)
gprm = tfd.GaussianProcessRegressionModel(
    kernel=optimized_kernel,
    index_points=predictive_index_points_,
    observation_index_points=observation_index_points_,
    observations=observations_,
    observation_noise_variance=observation_noise_variance_var,
    predictive_noise_variance=0.)

# Create op to draw  50 independent samples, each of which is a *joint* draw
# from the posterior at the predictive_index_points_. Since we have 200 input
# locations as defined above, this posterior distribution over corresponding
# function values is a 200-dimensional multivariate Gaussian distribution!
num_samples = 50
samples = gprm.sample(num_samples)
```

```
# Plot the true function, observations, and posterior samples.
plt.figure(figsize=(12, 4))
plt.plot(predictive_index_points_, sinusoid(predictive_index_points_),
         label='True fn')
plt.scatter(observation_index_points_[:, 0], observations_,
            label='Observations')
for i in range(num_samples):
  plt.plot(predictive_index_points_, samples[i, :], c='r', alpha=.1,
           label='Posterior Sample' if i == 0 else None)
leg = plt.legend(loc='upper right')
for lh in leg.legendHandles:
    lh.set_alpha(1)
plt.xlabel(r"Index points ($\mathbb{R}^1$)")
plt.ylabel("Observation space")
plt.show()
```

*Note: if you run the above code several times, sometimes it looks great and
other times it looks terrible! The maximum likelihood training of the parameters
is quite sensitive and sometimes converges to poor models. The best approach
is to use MCMC to marginalize the model hyperparameters.*

## Marginalizing hyperparameters with HMC

Instead of optimizing the hyperparameters, let's try integrating them out with Hamiltonian Monte Carlo. We'll first define and run a sampler to approximately draw from the posterior distribution over kernel hyperparameters, given the observations.

```
num_results = 100
num_burnin_steps = 50

sampler = tfp.mcmc.TransformedTransitionKernel(
    tfp.mcmc.NoUTurnSampler(
        target_log_prob_fn=target_log_prob,
        step_size=tf.cast(0.1, tf.float64)),
    bijector=[constrain_positive, constrain_positive, constrain_positive])

adaptive_sampler = tfp.mcmc.DualAveragingStepSizeAdaptation(
    inner_kernel=sampler,
    num_adaptation_steps=int(0.8 * num_burnin_steps),
    target_accept_prob=tf.cast(0.75, tf.float64))

initial_state = [tf.cast(x, tf.float64) for x in [1., 1., 1.]]
```

```
# Speed up sampling by tracing with `tf.function`.
@tf.function(autograph=False, jit_compile=False)
def do_sampling():
  return tfp.mcmc.sample_chain(
      kernel=adaptive_sampler,
      current_state=initial_state,
      num_results=num_results,
      num_burnin_steps=num_burnin_steps,
      trace_fn=lambda current_state, kernel_results: kernel_results)

t0 = time.time()
samples, kernel_results = do_sampling()
t1 = time.time()
print("Inference ran in {:.2f}s.".format(t1-t0))
```

    Inference ran in 9.00s.

Let's sanity-check the sampler by examining the hyperparameter traces.

```
(amplitude_samples,
 length_scale_samples,
 observation_noise_variance_samples) = samples

f = plt.figure(figsize=[15, 3])
for i, s in enumerate(samples):
  ax = f.add_subplot(1, len(samples) + 1, i + 1)
  ax.plot(s)
```

Now instead of constructing a single GP with the optimized hyperparameters, we construct the *posterior predictive distribution* as a mixture of GPs, each defined by a sample from the posterior distribution over hyperparameters. This approximately integrates over the posterior parameters via Monte Carlo sampling to compute the marginal predictive distribution at unobserved locations.

```
# The sampled hyperparams have a leading batch dimension, `[num_results, ...]`,
# so they construct a *batch* of kernels.
batch_of_posterior_kernels = tfk.ExponentiatedQuadratic(
    amplitude_samples, length_scale_samples)

# The batch of kernels creates a batch of GP predictive models, one for each
# posterior sample.
batch_gprm = tfd.GaussianProcessRegressionModel(
    kernel=batch_of_posterior_kernels,
    index_points=predictive_index_points_,
    observation_index_points=observation_index_points_,
    observations=observations_,
    observation_noise_variance=observation_noise_variance_samples,
    predictive_noise_variance=0.)

# To construct the marginal predictive distribution, we average with uniform
# weight over the posterior samples.
predictive_gprm = tfd.MixtureSameFamily(
    mixture_distribution=tfd.Categorical(logits=tf.zeros([num_results])),
    components_distribution=batch_gprm)

num_samples = 50
samples = predictive_gprm.sample(num_samples)
```

```
# Plot the true function, observations, and posterior samples.
plt.figure(figsize=(12, 4))
plt.plot(predictive_index_points_, sinusoid(predictive_index_points_),
         label='True fn')
plt.scatter(observation_index_points_[:, 0], observations_,
            label='Observations')
for i in range(num_samples):
  plt.plot(predictive_index_points_, samples[i, :], c='r', alpha=.1,
           label='Posterior Sample' if i == 0 else None)
leg = plt.legend(loc='upper right')
for lh in leg.legendHandles:
    lh.set_alpha(1)
plt.xlabel(r"Index points ($\mathbb{R}^1$)")
plt.ylabel("Observation space")
plt.show()
```

Although the differences are subtle in this case, in general, we would expect the posterior predictive distribution to generalize better (give higher likelihood to held-out data) than just using the most likely parameters as we did above.


```python
# -*- coding: utf-8 -*-
"""
Created on Sun May 27 17:17:43 2018

Blinker analysis

@author: Viva Horowitz
"""

import numpy as np
import matplotlib as mpl
mpl.rcParams.update(mpl.rcParamsDefault) # default params
import matplotlib.pyplot as plt
import pandas as pd
import os
import datetime
import seaborn as sns
import numpy as np
sns.set(style='ticks')
sns.set_context('poster')
from __future__ import division # division should return float

%pylab inline

print(np.__version__)
print(mpl.__version__)
print(pd.__version__)
print(sns.__version__)

# ** set for each dataset
datadate = '2018-05-25'
datafolder = r'O:\User\Viva\data\2018-05-25_blinking-statistics'
filename = 'GoodBlinker_5-25-2018.txt'
analysisfolder = r'O:\User\Viva\analysis\blinker_analysis'

os.chdir(datafolder)

tname = 'Time (s)'
PLname = 'PL Intensity (kCts/s)'
data = pd.read_csv(filename,
                   skiprows=1,
                   header=None,
                   sep='\t',
                   names=['Time (s)','PL Intensity (kCts/s)'],
                   usecols=[0,1])
data = data.set_index(tname)
print(data.head())
rebincount = 0

data.plot(linewidth=0.3)
plt.ylabel('photoluminescence (kCts/s)')
plt.show()

filename = filename.replace('\\','-')
origfilename = filename
os.chdir(analysisfolder)
today = datetime.date.isoformat(datetime.date.today())
if not os.path.exists('taken-' + datadate):
    os.makedirs('taken-' + datadate)
os.chdir('taken-' + datadate)

print('today: '+today)
os.getcwd()
```

```python
fig1,axPL = plt.subplots()
#axPL.plot(data, 'k')
#axPL.legend.remove()
data.plot(marker='.',linestyle='None',legend=False, ax=axPL, color='k',  alpha=.1)
data.plot(legend=False, linewidth=0.5, alpha=0.2, ax=axPL, color='k')
plt.ylabel('photoluminescence (kCts/s)')
plt.xlabel(tname)

plt.title(filename)

sns.despine()
plt.savefig(today + ',' + filename+',blinking.pdf', bbox_inches='tight')
plt.savefig(today + ',' + filename+',blinking.png', bbox_inches='tight', dpi=200)

plt.show()

def arangebylength(start, step, length):
    return np.arange(start = start, stop=start+step*(length-.5), step = step )

# rebin: combine two rows into one.
def rebindata(data, colname=PLname, indexname = tname):
    first = 1 # this is a value that indicates if we're on the first (=1) or the second (-1) of a pair of two bins
    rebinvals = []

    for value in data[colname]:
        if first == 1:
            prevval = value
        else:
            # calculate average of the two bins
            combined = (prevval + value) /2.0
            rebinvals.append(combined)
            del prevval
        first = -1 * first

    timesetting = data.index[1]-data.index[0]
    rebintimes = arangebylength(start =0, step = timesetting*2, length = len(rebinvals) )
    rebinneddata = pd.DataFrame(data = rebinvals, index = rebintimes, columns =[colname])
    rebinneddata.index.name=indexname
    return rebinneddata

# The data was collected at a particular rate
timesetting = data.index[1]-data.index[0]
fps = 1/timesetting
print('Collection rate: ' + str(fps) + ' Hz')
print('Time bin size: ' + str(timesetting) + ' sec')
```

```python
# *** run this an appropriate number of times, can skip if not rebinning

n = 0 # number of times to rebin.

for i in range(n):
    data = rebindata(data)
    rebincount = rebincount + 1
    print(rebincount)

timesetting = data.index[1]-data.index[0]
fps = 1/timesetting
print('Effective collection rate: ' + str(fps) + ' Hz')
print('Time bin size: ' + str(timesetting) + ' sec')
filename = origfilename + ', ' + str(fps) + ' Hz'

fig1,axPL = plt.subplots()
#axPL.plot(data, 'k')
#axPL.legend.remove()
data.plot(marker='.',linestyle='None',legend=False, ax=axPL, color='k',  alpha=.1)
data.plot(legend=False, linewidth=0.5, alpha=0.2, ax=axPL, color='k')
plt.ylabel('photoluminescence (kCts/s)')
plt.xlabel(tname)

plt.title(filename)

sns.despine()
#plt.savefig(today + ',' + filename+',blinking.pdf', bbox_inches='tight')
#plt.savefig(today + ',' + filename+',blinking.png', bbox_inches='tight', dpi=200)

plt.show()
```

```python
binsequence = np.arange(data.min(), data.max(), fps/200)# ** adjust bins as needed. Higher denominator: more bins
print (str(len(binsequence)) + ' bins')

#data.hist(bins= binsequence)
sns.distplot(data, bins=binsequence, kde=False, rug=False, kde_kws={'bw':.5, 'label':'KDE'}, norm_hist=False); # must norm if KDE
plt.xlabel('photoluminescence (kCts/s)')
#plt.locator_params(axis='x', nbins=10)
ax = plt.gca()

# ** Need to pick an appropriate threshold between on and off for each dataset
uthreshold = 15.2
lthreshold = uthreshold-2.5

ax.axvline(x=uthreshold, color='k')
ax.axvline(x=lthreshold, color = 'k')
#ax.tick_params(axis='x',which='minor',bottom='on')
#ax.xaxis.set_minor_locator(minorLocator)

plt.title(filename)
plt.ylabel('Occurrences')
sns.despine()

#plt.savefig(today + ',' + filename+',PLhistogram.pdf', bbox_inches='tight')
#plt.savefig(today + ',' + filename+',PLhistogram.png', bbox_inches='tight', dpi=200)

plt.show()
```

```python
# With an uthreshold and an lthreshold
# This code has an in between status where it won't be counted as remaining in the state
# but it won't be counted in the other state, either.
# This seems like the most straightforward way to deal with (rare) cases in between.
# Other options are shown.

threshdata = data[PLname].clip(upper=uthreshold, lower=lthreshold)

fig2,[axPL,axhist] = plt.subplots(nrows=1, ncols=2, figsize=(10,5), sharey=True, gridspec_kw = {'width_ratios':[4,2]})
data.plot(marker='.',linestyle='None',legend=False, alpha=0.1, ax=axPL, color='k')
data.plot(legend=False, linewidth=0.5, alpha=0.2, ax=axPL, color='k')
threshdata.plot(legend=False, ax=axPL)
axPL.set_ylabel('photoluminescence (kCts/s)')
axPL.set_xlabel(tname)
plt.suptitle(filename)
sns.despine()
plt.tight_layout()

sns.distplot(data, bins=binsequence, kde=True, rug=False, ax=axhist, vertical=True, color='k',
             kde_kws={'bw':.5});
#             kde_kws={'bw':.5, 'label':'Kernal density\nestimate (KDE)'});
#axhist.set_xlabel('photoluminescence (kCts/s)')
axhist.set_xlabel('Occurrences (normed)')
axhist.axhline(y=uthreshold)
axhist.axhline(y=lthreshold)

plt.savefig(today + ',' + filename+',blinkingandthreshold.pdf', bbox_inches='tight')
plt.savefig(today + ',' + filename+',blinkingandthreshold.png', bbox_inches='tight', dpi=200)

plt.show();
```

```python
# Main method: in between status where it won't be counted as remaining in the state
# but it won't be counted in the other state, either.
downtimes2= threshdata[threshdata != uthreshold].index
uptimes2=threshdata[threshdata != lthreshold].index

#print(uptimes2)
#print(downtimes2)

# Alternate option (not used)
uptimes= threshdata[threshdata == uthreshold].index
downtimes=threshdata[threshdata == lthreshold].index

#print(uptimes)
#print(downtimes)

# We can compare the two ways of interpreting blinks.
# There is a discrepancy at about 22 seconds between the two interpretations.
# It's a matter of whether we consider it stayed off or
# if we consider that it came up enough to have ended the off blink and began another.
# I tend to think it ended one and began another in this case, so I'm going to use uptimesep2 as my off duration.

# subtracting each time from the last time we had the same state gives the amount of time it was not in that state.
# Therefore the separation between up times is in fact the duration of time off.
uptimesep = uptimes[1:] - uptimes[:-1]
uptimesep2 = uptimes2[1:] - uptimes2[:-1]

"""
plt.plot(uptimes[:-1],uptimesep, '.-')
plt.plot(uptimes2[:-1],uptimesep2, '-')
plt.ylim(ymin=3*timesetting)
#plt.axis('equal')
plt.xlabel(tname)
plt.ylabel('duration of blink off (s)')
plt.figure()
data.plot()
plt.show()
"""

# We can compare the two ways of interpreting blinks. It looks like they mostly agree.

downtimesep = downtimes[1:] - downtimes[:-1]
downtimesep2 = downtimes2[1:] - downtimes2[:-1]

"""
plt.plot(downtimes[:-1],downtimesep,'.-')
plt.plot(downtimes2[:-1],downtimesep2,'-')
plt.ylim(ymin=3*timesetting)
#plt.axis('equal')
plt.xlabel(tname)
plt.ylabel('duration of blink on (s)')
#plt.figure()
data.plot()
plt.show()
"""

# I chose to use downtimesep2 and uptimesep2, as detailed above.

ontimes = filter(lambda a: a > 1.5*timesetting, downtimesep2) # requires python 2
offtimes = filter(lambda a: a > 1.5*timesetting, uptimesep2)

# save analyzed data: bright state duration
ontimesdf = pd.DataFrame(ontimes, columns=["Bright state duration (s)"])
ontimesdf.to_csv(filename+' uptimes.csv', index=False, header=True)
print('Uptimes saved to: ')
print(os.path.join(os.getcwd(), filename+ ' uptimes.csv'))
# save analyzed data: dim state duration
offtimesdf = pd.DataFrame(offtimes, columns=["Dim state duration (s)"])
offtimesdf.to_csv(filename+' downtimes.csv', index=False, header=True)
print('Downtimes saved to: ')
print(os.path.join(os.getcwd(), filename+ ' downtimes.csv'))
## to open:
# ontimesdf = pd.read_csv(filename+' ontimes.csv'
# offtimesdf = pd.read_csv(filename+' downtimes.csv')
#  # convert to list or else the rest of the code won't work.
# ontimes = list(ontimesdf['Bright state duration (s)'])
# offtimes = list(offtimesdf["Dim state duration (s)"])

## https://stackoverflow.com/questions/32765333/how-do-i-replicate-this-matlab-function-in-numpy/32765547#32765547
def histc(X, bins):
    map_to_bins = np.digitize(X,bins)
    r = np.zeros(bins.shape)
    for i in map_to_bins:
        r[i-1] += 1
    return [r, map_to_bins]

def CDF(S):
    M = len(S); # size of array = number of elements
    sorteddata = sort(S); # ascending sort is CDF, descending sort is 1-CDF
    cum_prob = [elem/M for elem in range(M)] # probability integrated from 0 to sorteddata.
    cum_prob2 = [1-p for p in cum_prob] # probability integrated from sorteddata to inf
    return sorteddata,cum_prob2

def nonlinearhistc(X,bins, thresh=3, verbose = True):
    map_to_bins = np.digitize(X,bins)
    r = np.zeros(bins.shape)
    for i in map_to_bins:
        r[i-1] += 1
    if verbose:
        print r
        #print bins
    ## normalize by bin width
    probability = np.zeros(bins.shape)
    area = 0;
    thinbincount = 0
    for i in range(len(bins)-1):
        if r[i]<=1:
            thinbincount += 1;
        thisbinwidth = bins[i+1] - bins[i]
        probability[i] = r[i]/thisbinwidth
        area += probability[i]*thisbinwidth;

    if thinbincount > thresh:
        print("Warning: too many bins for data, thinbincount=" + str(thinbincount))
    elif verbose:
        print("thinbincount=" + str(thinbincount))
    ## normalize area.
    normedprobability = [eachprobability / area for eachprobability in probability]
    return normedprobability, map_to_bins

def correlationpair(sequence):
    pairs = [(sequence[i], sequence[i+1]) for i in range(len(sequence)-1)]
    x = [sequence[i] for i in range(len(sequence)-1)]
    y = [sequence[i+1] for i in range(len(sequence)-1)]
    return x,y

```

    Uptimes saved to:
    O:\User\Viva\analysis\blinker_analysis\taken-2018-05-25\GoodBlinker_5-25-2018.txt, 100.0 Hz uptimes.csv
    Downtimes saved to:
    O:\User\Viva\analysis\blinker_analysis\taken-2018-05-25\GoodBlinker_5-25-2018.txt, 100.0 Hz downtimes.csv

```python
ontimesprev, ontimescurrent = correlationpair(ontimes)
offtimesprev, offtimescurrent = correlationpair(offtimes)
figure(figsize(3,3))
plt.scatter(ontimesprev, ontimescurrent, alpha=.5, label="bright", s=20)
plt.scatter(offtimesprev, offtimescurrent, alpha=.5, label="dim", s=20)
pylab.axis('equal')
ylim(ymin=0, ymax = 2)
xlim(xmin=0, xmax = 2)
ax = plt.gca()
#ax.set_xscale('log')
#ax.set_yscale('log')
ax.legend(loc='center left', bbox_to_anchor=(1, 0.5))

xlabel('Previous state duration (s)')
ylabel('Current state duration (s)');
```

```python
# ** Choose a nice number of bins to see how the data look.
# It's better to have as many bins as possible before the first empty bin.

numonbins = 60
numoffbins = 60

#plt.hist(ontimes, bins=numonbins, histtype='step', linewidth=1)
sns.distplot(ontimes, bins=numonbins, kde=False, rug=True,
             hist_kws={"histtype": "step", "linewidth": 1, "alpha": 1, "color": "k"});
plt.xlabel('Bright state duration (s)')
plt.ylabel('Occurrences')
ax = plt.gca()
ax.set_yscale('symlog', linthreshy=1)
ax.set_xscale('log')
sns.despine()
plt.show()
#plt.hist(offtimes, bins=numoffbins, histtype='step', linewidth=1)
sns.distplot(offtimes, bins=numoffbins, kde=False, rug=True,
             hist_kws={"histtype": "step", "linewidth": 1, "alpha": 1, "color": "k"});
plt.xlabel('Dim state duration (s)')
plt.ylabel('Occurrences')
ax = plt.gca()
ax.set_yscale('symlog', linthreshy=1)
ax.set_xscale('log')
sns.despine()
plt.show()
```

```python
# logarithmic bins.

## ** choose number of bins
numonbins_log = 10
numoffbins_log = 10

small = 1e-4
onbins=np.logspace(np.log10(min(ontimes)-small),np.log10(max(ontimes)+small), numonbins_log)
offbins = np.logspace(np.log10(min(offtimes)-small),np.log10(max(offtimes)+small), numoffbins_log)

onfitx = np.logspace(np.log10(min(ontimes)),np.log10(max(ontimes)), numonbins_log*10)
offfitx = np.logspace(np.log10(min(offtimes)),np.log10(max(offtimes)), numoffbins_log*10)

#plt.hist(ontimes, bins=numonbins, histtype='step', linewidth=1)
sns.distplot(ontimes, bins=onbins, kde=False, rug=True,
             hist_kws={"histtype": "step", "linewidth": 1, "alpha": 1, "color": "k"});
plt.xlabel('Bright state duration (s)')
plt.ylabel('Occurrences')
ax = plt.gca()
xlim(xmin=1e-2)
#ax.set_yscale('symlog', linthreshy=1)
ax.set_xscale('log')
sns.despine()
title('Log bins, unnormalized' )

onprobs, _ = nonlinearhistc(ontimes, onbins)
offprobs, _ = nonlinearhistc(offtimes, offbins)

plt.figure()
plot(onbins,onprobs, '.-', label="Bright state");
plot(offbins,offprobs, '.-', label="Dim state");

gca().set_yscale('log')
gca().set_xscale('log')
title('normalized log bins')
plt.xlabel('State duration (s)')
plt.ylabel('Probability');
legend();

figure()
sortedontimes,CDFontimes=CDF(ontimes)
sortedofftimes,CDFofftimes=CDF(offtimes)
loglog(sortedontimes,CDFontimes, '.-', label='Bright state');
loglog(sortedofftimes,CDFofftimes, '.-', label='Dim state');
xlabel('State duration (s)');
ylabel('Cumulative distribution function');
legend();
```

```python
from scipy.optimize import curve_fit
import scipy
import sympy
print(scipy.__version__)
print(sympy.__version__)
# need at least version 14.0 of scipy.

def expfit(t, A, tau):
    return (A * np.exp(t/(-tau)))

# integral from t to infinity of expfit
def CDFexp(t, A, tau):
    return A*tau*np.exp(-t/tau)

def powlaw(t, C, m):
    return C * t**(-m)

# integral from t to infinity of powlaw
def CDFpowlaw(t,C,m):
    return (C/(m-1)) * t**(-(m-1)) # https://arxiv.org/pdf/cond-mat/0412004.pdf

# truncated power law from  https://www.nature.com/articles/srep08898
def truncpow(t,C,m,tau):
    return(C * np.exp(t/(-tau)) * t**(-m))

# integral from t to infinity of truncpow, assuming m<1
def CDFtruncpow(t,C,m,tau):
    """I (Viva) calculated this in Mathematica"""
    return (C*(t**(-m))*(t * sympy.N(sympy.expint(m, t/tau))
                         + (t**m (1/tau)**m - (t/tau)**m) * tau * scipy.special.gamma(1-m)))

# Trying to find good guess
#onparamexp[1]=-.8
#onfitexp = 30 * np.exp(centeredonbins/(onparamexp[1]))
```

    0.17.1
    1.1.1

```python
showexpfit = False
showpowfit = True

### ==== FITS ====

# on time exponential fit, not truncated
onparamexp, onpcovexp = curve_fit(expfit, onbins, onprobs,p0 = (30, 2))
onfitexp = expfit(onfitx,onparamexp[0],onparamexp[1])

# on time power law fit, not truncated
onparampow, onpcovpow = curve_fit(powlaw, onbins, onprobs,p0 = (2, 1))
onfitpow = powlaw(onfitx,onparampow[0],onparampow[1])

# on time power law fit to CDF
onparampow_fromCDF, onpcovpow_fromCDF = curve_fit(CDFpowlaw, sortedontimes,CDFontimes,p0 = onparampow)
onfitpow_fromCDF = powlaw(onfitx,onparampow_fromCDF[0],onparampow_fromCDF[1])
onfitpowCDF = CDFpowlaw(onfitx,onparampow_fromCDF[0],onparampow_fromCDF[1])

# off time exponential fit, not truncated
offparamexp, offpcovexp = curve_fit(expfit, offbins, offprobs,p0 = (30, 2))
offfitexp = expfit(offfitx,offparamexp[0],offparamexp[1])

# off time power law fit, not truncated
offparampow, offpcovpow = curve_fit(powlaw, offbins, offprobs,p0 = (2, 1))
offfitpow = powlaw(offfitx,offparampow[0],offparampow[1])

# off time power law fit to CDF
offparampow_fromCDF, offpcovpow_fromCDF = curve_fit(CDFpowlaw, sortedofftimes,CDFofftimes,p0 = offparampow)
offfitpow_fromCDF = powlaw(offfitx,offparampow_fromCDF[0],offparampow_fromCDF[1])
offfitpowCDF = CDFpowlaw(offfitx,offparampow_fromCDF[0],offparampow_fromCDF[1])

### ===  display fits ===

fig,ax1 = plt.subplots()
ax1.plot(offbins,offprobs,'.-', label='data')
plt.xlabel('Dim state duration (s)')
plt.ylabel('Probability')
#ax1.set_yscale('symlog', linthreshy=1)
if showexpfit:
    ax1.plot(offfitx,offfitexp, label='exponential fit')
if showpowfit:
    ax1.plot(offfitx, offfitpow, label= 'power law fit')
    ax1.plot(offfitx, offfitpow_fromCDF, label= 'power law fit (from CDF)')
plt.title(filename)
plt.legend()
plt.show()

fig,ax1 = plt.subplots()
ax1.plot(offbins,offprobs,'.-', label='data')
plt.xlabel('Dim state duration (s)')
plt.ylabel('Probability')
#ax1.set_yscale('symlog', linthreshy=1)
if showexpfit:
    ax1.plot(offfitx,offfitexp, label='exponential fit')
if showpowfit:
    ax1.plot(offfitx, offfitpow, label= 'power law fit')
    ax1.plot(offfitx, offfitpow_fromCDF, label= 'power law fit (from CDF)')
plt.title(filename)
plt.legend()
ax1.set_yscale('log', linthreshy=1)
plt.show()

fig,ax1 = plt.subplots()
ax1.plot(offbins,offprobs,'.-', label='data')
plt.xlabel('Dim state duration (s)')
plt.ylabel('Probability')
#ax1.set_yscale('log')
if showexpfit:
    ax1.plot(offfitx,offfitexp, label='exponential fit')
if showpowfit:
    ax1.plot(offfitx, offfitpow, label= 'power law fit')
    ax1.plot(offfitx, offfitpow_fromCDF, label= 'power law fit (from CDF)')
plt.title(filename)
plt.legend()
ax1.set_yscale('log')
ax1.set_xscale('log')
plt.show()

figure() # display CDF
sortedontimes,CDFontimes=CDF(ontimes)
sortedofftimes,CDFofftimes=CDF(offtimes)
loglog(sortedontimes,CDFontimes, '.-', label='Bright state');
if showpowfit:
    plot(onfitx,onfitpowCDF, label='Bright state power law fit')
loglog(sortedofftimes,CDFofftimes, '.-', label='Dim state');
if showpowfit:
    plot(offfitx, offfitpowCDF, label='Dim state power law fit')
xlabel('State duration (s)');
ylabel('1-CDF')
title(filename + '\nCumulative distribution function');
legend();

figure() # display CDF
loglog(sortedontimes,[1-c for c in CDFontimes], '.-', label='Bright state');
if showpowfit:
    plot(onfitx,[1-p for p in onfitpowCDF], label='Bright state power law fit')
loglog(sortedofftimes,[1-c for c in CDFofftimes], '.-', label='Dim state');
if showpowfit:
    plot(offfitx, [1-p for p in offfitpowCDF], label='Dim state power law fit')
xlabel('State duration (s)');
ylabel('CDF')
title(filename + '\nCumulative distribution function');
legend();

# === BRIGHT STATE fit params ===
figure(figsize=(1,6))
plt.imshow(abs(onpcovexp), cmap="gray", interpolation="nearest", vmin=0)
plt.colorbar()
plt.title('Covariance matrix, exp fit, bright state, absolute values')
plt.show()
print("It's ok to use the uncertainties below as long as there aren't strong on-diagonal values.")

figure(figsize=(1,6))
plt.imshow(abs(onpcovexp), cmap="gray", interpolation="nearest", vmin=0)
plt.colorbar()
plt.title('Covariance matrix, pow fit, bright state, absolute values')
plt.show()
print("It's ok to use the uncertainties below as long as there aren't strong on-diagonal values.")

print 'Exponential fit:'
print  'on Coefficient A_bright = ' + str(onparamexp[0]) + ' ± ' + str(np.sqrt(onpcovexp[0,0]))
print 'time constant tau_bright = ' + str(onparamexp[1]) + ' ± ' + str(np.sqrt(onpcovexp[1,1])) + ' sec'

print 'Power law:'
print  'on Coefficient C_bright = ' + str(onparampow[0]) + ' ± ' + str(np.sqrt(onpcovpow[0,0]))
print 'logarithmic slope m_bright = ' + str(onparampow[1]) + ' ± ' + str(np.sqrt(onpcovpow[1,1]))

print 'Power law (from CDF):'
print  'on Coefficient C_bright = ' + str(onparampow_fromCDF[0]) + ' ± ' + str(np.sqrt(onpcovpow_fromCDF[0,0]))
print 'logarithmic slope m_bright = ' + str(onparampow_fromCDF[1]) + ' ± ' + str(np.sqrt(onpcovpow_fromCDF[1,1]))

# === DIM STATE fit params ===
figure(figsize=(1,6))
plt.imshow(abs(offpcovexp), cmap="gray", interpolation="nearest", vmin=0)
plt.colorbar()
plt.title('Covariance matrix, exp fit, dim state, absolute values')
plt.show()
print("It's ok to use the uncertainties below as long as there aren't strong off-diagonal values.")

figure(figsize=(1,6))
plt.imshow(abs(offpcovexp), cmap="gray", interpolation="nearest", vmin=0)
plt.colorbar()
plt.title('Covariance matrix, pow fit, dim state, absolute values')
plt.show()
print("It's ok to use the uncertainties below as long as there aren't strong off-diagonal values.")

print 'Exponential fit:'
print  'off Coefficient A_dim = ' + str(offparamexp[0]) + ' ± ' + str(np.sqrt(offpcovexp[0,0]))
print 'time constant tau_dim = ' + str(offparamexp[1]) + ' ± ' + str(np.sqrt(offpcovexp[1,1])) + ' sec'

print 'Power law:'
print  'off Coefficient C_dim = ' + str(offparampow[0]) + ' ± ' + str(np.sqrt(offpcovpow[0,0]))
print 'logarithmic slope m_dim = ' + str(offparampow[1]) + ' ± ' + str(np.sqrt(offpcovpow[1,1]))

print 'Power law (from CDF):'
print  'off Coefficient C_dim = ' + str(offparampow_fromCDF[0]) + ' ± ' + str(np.sqrt(offpcovpow_fromCDF[0,0]))
print 'logarithmic slope m_dim = ' + str(offparampow_fromCDF[1]) + ' ± ' + str(np.sqrt(offpcovpow_fromCDF[1,1]))

```

```python
# Save figures

# mpl.rcParams['figure.dpi'] = 200

font = {'size'   : 22}
plt.rc('font', **font)

# display off blinks
fig,ax1 = plt.subplots()
ax1.loglog(offbins,offprobs,'o-', label="Dim state")
plt.xlabel('Dim state duration (s)')
plt.ylabel('Probability')
ax1.loglog(offfitx,offfitpow_fromCDF, label="dim state power law")

#ploteveryfig(fig,ax1,today + ',' + filename+',offblinks' + infostr,offfitx,offfitpow_fromCDF, save=True)

# On blinks
#fig,ax1 = plt.subplots()
ax1.loglog(onbins,onprobs,'o-', label="Bright state")
plt.xlabel('State duration (s)')
plt.ylabel('Probability')
plt.title(filename)
sns.despine()
ax1.loglog(onfitx,onfitpow_fromCDF, label="bright state power law")
legend()
#ploteveryfig(fig,ax1,today + ',' + filename+',onblinks' + infostr,onfitx,onfitpow_fromCDF, save=True)

print os.getcwd()
savename = today + ',' + filename + ',powfitfromCDF,loglog'
print savename
savefig(savename + '.pdf', bbox_inches='tight', dpi=200)
savefig(savename + '.png', bbox_inches='tight', dpi=200)
plt.show()

print 'Bright state: Power law (from CDF):'
print  'Coefficient C_bright = ' + str(onparampow_fromCDF[0]) + ' ± ' + str(np.sqrt(onpcovpow_fromCDF[0,0]))
print 'logarithmic slope m_bright = ' + str(onparampow_fromCDF[1]) + ' ± ' + str(np.sqrt(onpcovpow_fromCDF[1,1]))

print 'Dim state: Power law (from CDF):'
print  'Coefficient C_dim = ' + str(offparampow_fromCDF[0]) + ' ± ' + str(np.sqrt(offpcovpow_fromCDF[0,0]))
print 'logarithmic slope m_dim = ' + str(offparampow_fromCDF[1]) + ' ± ' + str(np.sqrt(offpcovpow_fromCDF[1,1]))
```

```python
from collections import OrderedDict
from IPython.display import display

onsavename = today + ',' + filename + ',powfitfromCDF,on.csv'
offsavename = today + ',' + filename + ',powfitfromCDF,off.csv'

onparamsdf = pd.DataFrame(OrderedDict([('file', filename),
                ('C',onparampow_fromCDF[0]),
                ('C_unc', np.sqrt(onpcovpow_fromCDF[0,0])),
                ('m', onparampow_fromCDF[1]),
                ('m_unc',np.sqrt(onpcovpow_fromCDF[1,1]))]), index = [0])
display(onparamsdf)
onparamsdf.to_csv(onsavename)
print onsavename

offparamsdf = pd.DataFrame(OrderedDict([('file', filename),
                ('C',offparampow_fromCDF[0]),
                ('C_unc', np.sqrt(offpcovpow_fromCDF[0,0])),
                ('m', offparampow_fromCDF[1]),
                ('m_unc',np.sqrt(offpcovpow_fromCDF[1,1]))]), index = [0])
display(offparamsdf)
offparamsdf.to_csv(offsavename)
print offsavename

os.getcwd()
```

<div>
<style>
    .dataframe thead tr:only-child th {
        text-align: right;
    }

    .dataframe thead th {
        text-align: left;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>file</th>
      <th>C</th>
      <th>C_unc</th>
      <th>m</th>
      <th>m_unc</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>GoodBlinker_5-25-2018.txt, 100.0 Hz</td>
      <td>0.098106</td>
      <td>0.002397</td>
      <td>1.281884</td>
      <td>0.014401</td>
    </tr>
  </tbody>
</table>
</div>

    2018-07-02,GoodBlinker_5-25-2018.txt, 100.0 Hz,powfitfromCDF,on.csv

<div>
<style>
    .dataframe thead tr:only-child th {
        text-align: right;
    }

    .dataframe thead th {
        text-align: left;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>file</th>
      <th>C</th>
      <th>C_unc</th>
      <th>m</th>
      <th>m_unc</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>GoodBlinker_5-25-2018.txt, 100.0 Hz</td>
      <td>0.087727</td>
      <td>0.002008</td>
      <td>1.323489</td>
      <td>0.018702</td>
    </tr>
  </tbody>
</table>
</div>

    2018-07-02,GoodBlinker_5-25-2018.txt, 100.0 Hz,powfitfromCDF,off.csv

    'O:\\User\\Viva\\analysis\\blinker_analysis\\taken-2018-05-25'

```python
print 'Exp [A, tau]'
print offparamexp
print onparamexp

print 'Pow [C, m]'
print offparampow
print onparampow

print 'Pow,CDF [C, m]'
print offparampow_fromCDF
print onparampow_fromCDF
```

    Exp [A, tau]
    [  1.28149292e+02   9.39423437e-03]
    [  4.32624449e+01   1.31303199e-02]
    Pow [C, m]
    [  4.69026177e-04   2.65442277e+00]
    [  1.29556031e-03   2.27207987e+00]
    Pow,CDF [C, m]
    [ 0.0877272   1.32348887]
    [ 0.09810592  1.28188407]

```python

```


# Muscle modeling

> Marcos Duarte
> Laboratory of Biomechanics and Motor Control ([http://demotu.org/](http://demotu.org/))
> Federal University of ABC, Brazil

There are two major classes of muscle models that have been used in biomechanics and motor control: the Hill-type and Huxley-type models. They differ mainly on how the contractile element is modeled. In Hill-type models, the modeling of the contractile element is phenomenological; arbitrary mathematical functions are used to reproduce experimental observations relating muscle characteristics (such as excitation/activation, muscle length and velocity) with the muscle force. In Huxley-type models, the modeling of the contractile element is mechanistic; the mathematical functions used represent the hypothesized mechanisms for the cross-bridge dynamics (Tsianos and Loeb, 2013). Huxley-type models tend to produce more realistic results than Hill-type models for certain conditions but they have a higher computational demand. For this reason, Hill-type models are more often employed in musculoskeletal modeling and simulation.

Hill-type muscle models are presented in several texts (e.g., Erdermir et al. 2007; He et al., 1991; McMahon, 1984; Nigg and Herzog, 2007; Robertson et al., 2013, Thelen, 2003; Tsianos and Loeb, 2013, Winters, 1990; Zajac, 1989; Zatsiorsky and Prilutsky, 2012) and implemented in many software for modeling and simulation of the musculoskeletal dynamics of human movement (e.g., the free and open source software [OpenSim](https://simtk.org/home/opensim)).

Next, let's see a brief overview of a Hill-type muscle model and a basic implementation in Python.

## Hill-type muscle model

Hill-type models are developed to reproduce the dependence of force with the length and velocity of the muscle-tendon unit and parameters are lumped and made dimensionless in order to represent different muscles with few changes in these parameters. A Hill-type model is complemented with the modeling of the activation dynamics (i.e., the temporal pattern of muscle activation and deactivation as a function of the neural excitation) to produce more realistic results. As a result, the force generated will be a function of three factors: the length and velocity of the muscle-tendon unit and its activation level $a$.

A Hill-type muscle model has three components (see figure below): two for the muscle, an active contractile element (CE) and a passive elastic element (PE) in parallel with the CE, and one component for the tendon, an elastic element (SE) in series with the muscle. In some variations, a damping component is added parallel to the CE as a fourth element. A [pennation angle](http://en.wikipedia.org/wiki/Muscle_architecture) (angle of the pennate fibers with respect to the force-generating axis) is also included in the model. In a simpler approach, the muscle and tendon are assumed massless.

Figure. A Hill-type muscle model with three components: two for the muscle, an active contractile element, $\mathsf{CE}$, and a passive elastic element in parallel, $\mathsf{PE}$, with the $\mathsf{CE}$, and one component for the tendon, an elastic element in series, $\mathsf{SE}$, with the muscle. $\mathsf{L_{MT}}$: muscle–tendon length, $\mathsf{L_T}$: tendon length, $\mathsf{L_M}$: muscle fiber length, $\mathsf{F_T}$: tendon force, $\mathsf{F_M}$: muscle force, and $α$: pennation angle.

Let's now revise the models of a Hill-type muscle with three components and activation dynamics by two references:
 1. [Thelen (2003)](http://simtk-confluence.stanford.edu:8080/display/OpenSim/Thelen+2003+Muscle+Model) with some of the adjustments described in Millard et al. (2013). Hereafter, Thelen2003Muscle or T03.
 2. [McLean, Su, van den Bogert (2003)](http://www.ncbi.nlm.nih.gov/pubmed/14986412). Hereafter, McLean2003Muscle or M03.

First, let's import the necessary Python libraries and customize the environment:

```python
import numpy as np
from scipy.integrate import ode, odeint
%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib
matplotlib.rcParams['lines.linewidth'] = 3
matplotlib.rcParams['font.size'] = 13
matplotlib.rcParams['lines.markersize'] = 5
matplotlib.rc('axes', grid=True, labelsize=14, titlesize=16, ymargin=0.05)
matplotlib.rc('legend', numpoints=1, fontsize=11)
```

### Force-length relationship

In a Hill-type model, the force a muscle can generate depends on its length due to two factors:

1. The active force of the contractile element (CE), which in turn depends on the spatial superposition of the actin and myosin molecules to form cross-bridges at the sarcomere. A maximum number of cross-bridges will be formed at an optimal fiber length, generating a maximum force. When a fiber is too stretched or too shortened, fewer cross-bridges will be formed, decreasing the force generated.
2. The passive and parallel elastic element (PE), which behaves as a nonlinear spring where no force is generated below a certain length (the slack length) and force increases with the muscle elongation.

#### Force-length relationship of the contractile element

Thelen2003Muscle represented the normalized force-length relationship of the contractile element by a Gaussian function:

\begin{equation}
\bar{f}_{l,CE} = exp\left[-(\bar{L}_M-1)^2/\gamma\right]
\label{}
\end{equation}

where $\gamma$ is a shape factor and $\bar{L}_M$ is the muscle fiber length normalized by the optimal muscle fiber length at which maximal force can be produced, $L_{Mopt}$:

\begin{equation}
\bar{L}_M=\dfrac{L_M}{L_{Mopt}}
\label{}
\end{equation}

Thelen2003Muscle adopted $\gamma=0.45$. The actual force produced is obtained multiplying $\bar{f}_{l,CE}$ by the maximum isometric muscle force, $F_{M0}$. Thelen2003Muscle assumed that the maximum isometric muscle forces for old adults were 30% lower than those used for young adults.

McLean2003Muscle represented the force-length relationship of the contractile element (not normalized) as a function of muscle length (not normalized) by a quadratic function:

\begin{equation}
f_{l,CE} = max \left\{
  \begin{array}{l l}
    F_{Mmin} \\
    F_{M0}\left[1 - \left(\dfrac{L_M-L_{Mopt}}{WL_{Mopt}}\right)^2\right]
\end{array} \right.
\label{}
\end{equation}

where $W$ is a dimensionless parameter describing the width of the force-length relationship. A minimum force level $F_{Mmin}$ is employed for numerical stability.
McLean2003Muscle adopted $W=1$ and $F_{Mmin}=10 N$.

The corresponding Python functions are:

```python
def flce_T03(lm=1, gammal=0.45):
    """Thelen (2003) force of the contractile element as function of muscle length.

    Parameters
    ----------
    lm  : float, optional (default=1)
        normalized muscle fiber length
    gammal  : float, optional (default=0.45)
        shape factor

    Returns
    -------
    fl  : float
        normalized force of the muscle contractile element
    """

    fl = np.exp(-(lm-1)**2/gammal)

    return fl
```

```python
def flce_M03(lm=1, lmopt=1, fm0=1, fmmin=0.001, wl=1):
    """McLean (2003) force of the contractile element as function of muscle length.

    Parameters
    ----------
    lm  : float, optional (default=1)
        muscle (fiber) length
    lmopt  : float, optional (default=1)
        optimal muscle fiber length
    fm0  : float, optional (default=1)
        maximum isometric muscle force
    fmmin  : float, optional (default=0.001)
        minimum muscle force
    wl  : float, optional (default=1)
        shape factor of the contractile element force-length curve

    Returns
    -------
    fl  : float
        force of the muscle contractile element
    """

    fl = np.max([fmmin, fm0*(1 - ((lm - lmopt)/(wl*lmopt))**2)])

    return fl
```

And plots of these functions:

```python
lm = np.arange(0, 2.02, .02)
fce_T03 = np.zeros(lm.size)
fce_M03 = np.zeros(lm.size)
for i in range(len(lm)):
    fce_T03[i] = flce_T03(lm[i])
    fce_M03[i] = flce_M03(lm[i])
```

```python
plt.figure(figsize=(7, 4))
plt.plot(lm, fce_T03, 'b', label='T03')
plt.plot(lm, fce_M03, 'g', label='M03')
plt.xlabel('Normalized length')
plt.ylabel('Normalized force')
plt.legend(loc='best')
plt.suptitle('Force-length relationship of the contractile element', y=1, fontsize=16)
plt.show()
```

Similar results when the same parameters are used.

#### Force-length relationship of the parallel element

Thelen2003Muscle represents the normalized force of the parallel (passive) element of the muscle as a function of muscle length (normalized by the optimal muscle fiber length) by an exponential function:

\begin{equation}
\bar{F}_{PE}(\bar{L}_M) = \dfrac{exp\left[k_{PE}(\bar{L}_M-1)/\epsilon_{M0}\right]-1}{exp(k_{PE})-1}
\label{}
\end{equation}

where $k_{PE}$ is an exponential shape factor and $\epsilon_{M0}$ is the passive muscle strain due to maximum isometric force:

\begin{equation}
\epsilon_{M0}=\dfrac{L_M(F_{M0})-L_{Mslack}}{L_{Mslack}}
\label{}
\end{equation}

where $L_{Mslack}$ is the muscle slack length. Thelen2003Muscle adopted $L_{Mslack} = L_{Mopt}$.
Thelen2003Muscle adopted $k_{PE}=5$ and $\epsilon_{M0}=0.6$ for young adults ($\epsilon_{M0}=0.5$ for old adults). The actual force produced is obtained multiplying $\bar{F}_{PE}$ by the maximum isometric muscle force, $F_{M0}$.

McLean2003Muscle represents the force of the parallel (passive) element of the muscle (not normalized) as a function of muscle length (not normalized) by a quadratic function:

\begin{equation}
F_{PE}(L_M) = \left\{
  \begin{array}{l l}
    0 \quad & \text{if} \quad L_M \leq L_{Mslack} \\
    k_{PE}(L_M - L_{Mslack})^2 \quad & \text{if} \quad L_M > L_{Mslack}
\end{array} \right.
\label{}
\end{equation}

where $k_{PE}$ is a stiffness parameter of the parallel element such that the passive muscle force is equal to the normalized maximum isometric force of the muscle when the CE is stretched to its maximal length for active force production:

\begin{equation}
k_{PE} = \dfrac{F_{M0}}{(WL_{Mopt})^2}
\label{}
\end{equation}

McLean2003Muscle adopted $L_{Mslack} = L_{Mopt}$.

The corresponding Python functions are:

```python
def fpelm_T03(lm=1, kpe=5, epsm0=0.6):
    """Thelen (2003) force of the muscle parallel element as function of muscle length.

    Parameters
    ----------
    lm  : float, optional (default=1)
        normalized muscle fiber length
    kpe  : float, optional (default=5)
        exponential shape factor
    epsm0  : float, optional (default=0.6)
        passive muscle strain due to maximum isometric force

    Returns
    -------
    fpe  : float
        normalized force of the muscle parallel (passive) element
    """

    if lm < 1:
        fpe = 0
    else:
        fpe = (np.exp(kpe*(lm-1)/epsm0)-1)/(np.exp(kpe)-1)

    return fpe
```

```python
def fpelm_M03(lm=1, lmopt=1, fm0=1, lmslack=1, wp=1):
    """McLean (2003) force of the muscle parallel element as function of muscle length.

    Parameters
    ----------
    lm  : float, optional (default=1)
        muscle fiber length
    lmopt  : float, optional (default=1)
        optimal muscle (fiber) length
    fm0  : float, optional (default=1)
        maximum isometric muscle force
    lmslack  : float, optional (default=1)
        muscle slack length
    wp  : float, optional (default=1)
        shape factor of the parallel element force-length curve

    Returns
    -------
    fpe  : float
        force of the muscle parallel (passive) element
    """

    kpe = fm0/(wp*lmopt)**2
    if lm <= lmslack:
        fpe = 0
    else:
        fpe = kpe*(lm-lmslack)**2

    return fpe
```

And plots of these functions:

```python
lm = np.arange(0, 2.02, .02)
fpe_T03 = np.zeros(lm.size)
fpe_M03 = np.zeros(lm.size)
for i in range(len(lm)):
    fpe_T03[i] = fpelm_T03(lm[i])
    fpe_M03[i] = fpelm_M03(lm[i])
```

```python
fig, (ax1, ax2) = plt.subplots(nrows=1, ncols=2, sharex=True, sharey=True, figsize=(10, 4))
ax1.plot(lm[:86], fce_T03[:86], 'b', label='Active')
ax1.plot(lm[:86], fpe_T03[:86], 'r', label='Passive')
ax1.plot(lm[:86], fce_T03[:86] + fpe_T03[:86], 'g', label='Total')
ax1.text(0.1, 2.6, 'T03')
ax1.set_xlim([0, 1.7])
ax1.set_xlabel('Normalized length')
ax1.set_ylabel('Normalized force')
#ax1.legend(loc='best')
ax2.plot(lm[:86], fce_M03[:86], 'b', label='Active')
ax2.plot(lm[:86], fpe_M03[:86], 'r', label='Passive')
ax2.plot(lm[:86], fce_M03[:86] + fpe_M03[:86], 'g', label='Total')
ax2.text(0.1, 2.6, 'M03')
ax2.set_xlim([0, 1.7])
ax2.set_xlabel('Normalized length')
ax2.legend(loc='best')
plt.suptitle('Muscle force-length relationship', y=1, fontsize=16)
plt.tight_layout()
plt.show()
```

The results are different at the maximum stretching because Thelen2003Muscle and McLean2003Muscle model differently the passive component.
These results were simulated for a maximum muscle activation (an activation level, $a$, of 1, where 0 is no activation). The effect of different activation levels on the total muscle force (but only the active force is affected) is shown in the next figure:

```python
lm = np.arange(0, 2.02, .02)
fce_T03_als = np.zeros((lm.size, 5))
als = [0, 0.25, 0.50, 0.75, 1.0]
for j, al in enumerate(als):
    for i in range(len(lm)):
        fce_T03_als[i, j] = flce_T03(lm[i])*al
```

```python
fig, ax = plt.subplots(nrows=1, ncols=1, sharex=True, sharey=True, figsize=(6, 5))
for j, al in enumerate(als):
    ax.plot(lm[:86], fce_T03_als[:86, j] + fpe_T03[:86], label='%.2f'%al)
ax.text(0.1, 2.6, 'T03')
ax.set_xlim([0, 1.7])
ax.set_xlabel('Normalized length')
ax.set_ylabel('Normalized force')
ax.legend(loc='best', title='Activation level')
ax.set_title('Muscle force-length relationship', y=1, fontsize=16)
plt.tight_layout()
plt.show()
```

#### Force-length relationship of the series element (tendon)

Thelen2003Muscle represented the tendon force of the series element as a function of the normalized tendon length (in fact, tendon strain) by an exponential function during an initial nonlinear toe region and by a linear function thereafter:

\begin{equation}
\bar{F}_{SE}(\bar{L}_T) = \left\{
  \begin{array}{l l}
    \dfrac{\bar{F}_{Ttoe}}{exp(k_{Ttoe})-1}\left[exp(k_{Ttoe}\epsilon_T/\epsilon_{Ttoe})-1\right] \quad & \text{if} \quad \epsilon_T \leq \epsilon_{Ttoe} \\
    k_{Tlin}(\epsilon_T - \epsilon_{Ttoe}) + \bar{F}_{Ttoe} \quad & \text{if} \quad \epsilon_T > \epsilon_{Ttoe}
\end{array} \right.
\label{}
\end{equation}

where $\epsilon_{T}$ is the tendon strain:

\begin{equation}
\epsilon_{T} = \dfrac{L_T-L_{Tslack}}{L_{Tslack}}
\label{}
\end{equation}

$L_{Tslack}$ is the tendon slack length, $\epsilon_{Ttoe}$ is the tendon strain above which the tendon exhibits linear behavior, $k_{Ttoe}$ is an exponential shape factor, and $k_{Tlin}$ is a linear scale factor. The parameters are chosen such that the tendon elongation at the normalized maximal isometric force of the muscle is 4% of the tendon length ($\epsilon_{T0}=0.04$).
Thelen2003Muscle adopted $k_{Ttoe}=3$ and the transition from nonlinear to linear behavior occurs for normalized tendon forces greater than $\bar{F}_{Ttoe}=0.33$. For continuity of slopes at the transition, $\epsilon_{Ttoe}=0.609\epsilon_{T0}$ and $k_{Tlin}=1.712/\epsilon_{T0}$. The actual force produced is obtained multiplying $\bar{F}_{SE}$ by the maximum isometric muscle force, $F_{M0}$.

McLean2003Muscle represented the tendon force (not normalized) of the series element as a function of the tendon length (not normalized) by the same quadratic function used for the force of the muscle passive element:

\begin{equation}
F_{SE}(L_T) = \left\{
  \begin{array}{l l}
    0 \quad & \text{if} \quad L_T \leq L_{Tslack} \\
    k_T(L_T - L_{Tslack})^2 \quad & \text{if} \quad L_T > L_{Tslack}
\end{array} \right.
\label{}
\end{equation}

where $k_T$ is the tendon stiffness. The stiffness parameter $k_T$ is chosen such that the tendon elongation is 4% at the maximum isometric force, $k_T=(1/\epsilon_{T0})^2=625$ for $F_{M0}=1$.

The corresponding Python functions are:

```python
def fselt_T03(lt=1, ltslack=1, epst0=0.04, kttoe=3):
    """Thelen (2003) force-length relationship of tendon as function of tendon length.

    Parameters
    ----------
    lt  : float, optional (default=1)
        normalized tendon length
    ltslack  : float, optional (default=1)
        normalized tendon slack length
    epst0  : float, optional (default=0.04)
        tendon strain at the maximal isometric muscle force
    kttoe  : float, optional (default=3)
        linear scale factor

    Returns
    -------
    fse  : float
        normalized force of the tendon series element
    """

    epst = (lt-ltslack)/ltslack
    fttoe = 0.33
    # values from OpenSim Thelen2003Muscle
    epsttoe =  .99*epst0*np.e**3/(1.66*np.e**3 - .67)
    ktlin =  .67/(epst0 - epsttoe)
    #
    if epst <= 0:
        fse = 0
    elif epst <= epsttoe:
        fse = fttoe/(np.exp(kttoe)-1)*(np.exp(kttoe*epst/epsttoe)-1)
    else:
        fse = ktlin*(epst-epsttoe) + fttoe

    return fse
```

```python
def fselt_M03(lt, ltslack=1, fm0=1, epst0=0.04):
    """McLean (2003) force-length relationship of tendon as function of tendon length.

    Parameters
    ----------
    lt  : float, optional (default=1)
        tendon length
    ltslack  : float, optional (default=1)
        tendon slack length
    fm0  : float, optional (default=1)
        maximum isometric muscle force
    epst0  : float, optional (default=0.04)
        tendon strain at the maximal isometric muscle force

    Returns
    -------
    fse  : float
        force of the tendon series element
    """

    kt = fm0/epst0**2
    if lt <= ltslack:
        fse = 0
    else:
        fse = kt*(lt-ltslack)**2

    return fse
```

And plots of these functions:

```python
lt = np.arange(1, 1.051, .001)
fse_T03 = np.zeros(lt.size)
fse_M03 = np.zeros(lt.size)
for i in range(len(lt)):
    fse_T03[i] = fselt_T03(lt[i])
    fse_M03[i] = fselt_M03(lt[i])
```

```python
plt.figure(figsize=(7, 4))
plt.plot(lt-1, fse_T03, 'b', label='T03')
plt.plot(lt-1, fse_M03, 'g', label='M03')
plt.plot(0.04, 1, 'ro', markersize=8)
plt.text(0.04, 0.7, '$\epsilon_{T0}$', fontsize=22)
plt.xlabel('Tendon strain')
plt.ylabel('Normalized force')
plt.legend(loc='upper left')
plt.suptitle('Tendon force-length relationship (series element)', y=1, fontsize=16)
plt.show()
```

Similar results when the same parameters are used.

### Force-velocity relationship of the contractile element

The force-velocity relation of the contractile element for shortening (concentric activation) is based on the well known Hill's equation of a hyperbola describing that the product between force $F$ and velocity $V$ of the contractile element is constant (Winters, 1990; Winters, 1995):

\begin{equation}
(F+a')(V+b') = (F_{0}+a')b'
\label{}
\end{equation}

where $a'$, $b'$, and $F_{0}$ are constants.

We can rewrite the equation above with constants more meaningful to our modeling:

\begin{equation}
(F_{M}+A_f F_{Mlen})(V_M+A_f V_{Mmax}) = A_f F_{Mlen}V_{Mmax}(1+A_f)
\label{}
\end{equation}

where $F_{M}$ and $V_M$ are the contractile element force and velocity, respectively, and the three constants are: $V_{Mmax}$, the maximum unloaded velocity (when $F_{M}=0$), $F_{Mlen}$, the maximum isometric force (when $V_M=0$), and $A_f$, a shape factor which specifies the concavity of the hyperbola.

Based on the equation above for the shortening phase and in Winters (1990, 1995) for the lengthening phase, Thelen2003Muscle employed the following force-velocity equation:

\begin{equation}
V_M = (0.25+0.75a)\,V_{Mmax}\dfrac{\bar{F}_M-a\bar{f}_{l,CE}}{b}
\label{}
\end{equation}

where

\begin{equation}
b = \left\{
  \begin{array}{l l l}
    a\bar{f}_{l,CE} + \bar{F}_M/A_f \quad & \text{if} \quad \bar{F}_M \leq a\bar{f}_{l,CE} & \text{(shortening)} \\
    \\
    \dfrac{(2+2/A_f)(a\bar{f}_{l,CE}\bar{f}_{Mlen} - \bar{F}_M)}{\bar{f}_{Mlen}-1} \quad & \text{if} \quad \bar{F}_M > a\bar{f}_{l,CE} & \text{(lengthening)}
\end{array} \right.
\label{}
\end{equation}

where $a$ is the activation level and $\bar{f}_{Mlen}$ is a constant for the maximum force generated at the lengthening phase (normalized by the maximum isometric force).
Thelen2003Muscle adopted $A_f=0.25$, $V_{Mmax}=10L_{Mopt}/s$, $\bar{f}_{Mlen}=1.4$ for young adults ($V_{Mmax}=8L_{Mopt}/s$ and $\bar{f}_{Mlen}=1.8$ for old adults). Note that the dependences of the force with the activation level and with the muscle length are already incorporated in the expression above.

McLean2013Muscle employed:

\begin{equation}
\bar{f}_{v,CE} = \left\{
  \begin{array}{l l l}
    \dfrac{\lambda(a)V_{Mmax} + V_M}{\lambda(a)V_{Mmax} - V_M/A_f} \quad & \text{if} \quad V_M \leq 0 & \text{(shortening)} \\
    \\
    \dfrac{\bar{f}_{Mlen}V_M + d_1}{V_M + d_1} \quad & \text{if} \quad 0 < V_M \leq \gamma d_1 & \text{(slow lengthening)} \\
    \\
    d_3 + d_2V_M \quad & \text{if} \quad V_M > \gamma d_1 & \text{(fast lengthening)}
\end{array} \right.
\label{}
\end{equation}

where

\begin{equation}
\begin{array}{l l}
    \lambda(a) = 1-e^{-3.82a} + a\:e^{-3.82} \\
    \\
    d_1 = \dfrac{V_{Mmax}A_f(\bar{f}_{Mlen}-1)}{S(A_f+1)} \\
    \\
    d_2 = \dfrac{S(A_f+1)}{V_{Mmax}A_f(\gamma+1)^2} \\
    \\
    d_3 = \dfrac{(\bar{f}_{Mlen}-1)\gamma^2}{(\gamma+1)^2} + 1
\end{array}
\label{}
\end{equation}

where $\lambda(a)$ is a scaling factor to account for the influence of the activation level $a$ on the force-velocity relationship, $\bar{f}_{Mlen}$ is the asymptotic (maximum) value of $\bar{F}_M$, $S$ is a parameter to double the slope of the force-velocity curve at zero velocity, and $\gamma$ is a dimensionless parameter to ensure the transition between the hyperbolic and linear parts of the lengthening phase.
McLean2013Muscle adopted $A_f=0.25$, $V_{Mmax}=10L_{Mopt}/s$, $\bar{f}_{Mlen}=1.5$, $S=2.0$, and $\gamma=5.67$.

Let's write these expressions as Python code and visualize them:

```python
def vmfce_T03(fm, flce=1, lmopt=1, a=1, vmmax=1, fmlen=1.4, af=0.25):
    """Thelen (2003) velocity of the force-velocity relationship as function of CE force.

    Parameters
    ----------
    fm  : float
        normalized muscle force
    flce  : float, optional (default=1)
        normalized muscle force due to the force-length relationship
    lmopt  : float, optional (default=1)
        optimal muscle fiber length
    a  : float, optional (default=1)
        muscle activation level
    vmmax  : float, optional (default=1)
        maximum muscle velocity for concentric activation
    fmlen  : float, optional (default=1.4)
        normalized maximum force generated at the lengthening phase
    af  : float, optional (default=0.25)
        shape factor

    Returns
    -------
    vm  : float
        velocity of the muscle
    """

    vmmax = vmmax*lmopt
    if fm <= a*flce:  # isometric and concentric activation
        b = a*flce + fm/af
    else:             # eccentric activation
        b = (2 + 2/af)*(a*flce*fmlen - fm)/(fmlen - 1)
    vm = (0.25  + 0.75*a)*vmmax*(fm - a*flce)/b

    return vm
```

Let's find an expression for contractile element force as function of muscle velocity given the equation above, i.e. we want to invert the equation. For that, let's use [Sympy](http://www.sympy.org/):

```python
def fvce_T03_symb():
    # Thelen (2003) velocity of the force-velocity relationship as function of CE force

    from sympy import symbols, solve, collect, Eq
    a, flce, fm, af, fmlen, vmmax = symbols('a, flce, fm, af, fmlen, vmmax', positive=True)
    vm = symbols('vm', real=True)

    b = a*flce + fm/af
    vm_eq = Eq(vm - (0.25 + 0.75*a)*vmmax*(fm - a*flce)/b)
    sol = solve(vm_eq, fm)
    print('fm <= a*flce:\n', collect(sol[0], vmmax),'\n')

    b = (2 + 2/af)*(a*flce*fmlen - fm)/(fmlen - 1)
    vm_eq = Eq(vm - (0.25 + 0.75*a)*vmmax*(fm - a*flce)/b)
    sol = solve(vm_eq, fm)
    print('fm > a*flce:\n', collect(sol[0], (vmmax*af, fmlen, vm)))

fvce_T03_symb()
```

    fm <= a*flce:
     a*af*flce*(4.0*vm + vmmax*(3.0*a + 1))/(-4.0*vm + vmmax*(3.0*a*af + af))

    fm > a*flce:
     a*flce*(af*vmmax*(3.0*a*fmlen - 3.0*a + fmlen - 1) + fmlen*(8.0*af*vm + 8.0*vm))/(af*vmmax*(3.0*a*fmlen - 3.0*a + fmlen - 1) + vm*(8.0*af + 8.0))

And here is the function we need to compute contractile element force as function of muscle velocity:

```python
def fvce_T03(vm=0, flce=1, lmopt=1, a=1, vmmax=1, fmlen=1.4, af=0.25):
    """Thelen (2003) force of the contractile element as function of muscle velocity.

    Parameters
    ----------
    vm  : float, optional (default=0)
        muscle velocity
    flce  : float, optional (default=1)
        normalized muscle force due to the force-length relationship
    lmopt  : float, optional (default=1)
        optimal muscle fiber length
    a  : float, optional (default=1)
        muscle activation level
    vmmax  : float, optional (default=1)
        maximum muscle velocity for concentric activation
    fmlen  : float, optional (default=1.4)
        normalized maximum force generated at the lengthening phase
    af  : float, optional (default=0.25)
        shape factor

    Returns
    -------
    fvce  : float
        normalized force of the muscle contractile element
    """

    vmmax = vmmax*lmopt
    if vm <= 0:  # isometric and concentric activation
        fvce = af*a*flce*(4*vm + vmmax*(3*a + 1))/(-4*vm + vmmax*af*(3*a + 1))
    else:        # eccentric activation
        fvce = a*flce*(af*vmmax*(3*a*fmlen - 3*a + fmlen - 1) + 8*vm*fmlen*(af + 1))/\
               (af*vmmax*(3*a*fmlen - 3*a + fmlen - 1) + 8*vm*(af + 1))

    return fvce
```

Here is the Python function for the McLean (2003) model:

```python
def fvce_M03(vm=0, lmopt=1, a=1, vmmax=1, fmlen=1.5, af=0.25, s=2, gammav=5.67):
    """McLean (2003) contractile element force as function of muscle velocity.

    Parameters
    ----------
    vm  : float, optional (default=0)
        muscle velocity
    lmopt  : float, optional (default=1)
        optimal muscle fiber length
    a  : float, optional (default=1)
        muscle activation level
    vmmax  : float, optional (default=1)
        maximum muscle velocity for concentric activation
    fmlen  : float, optional (default=1.5)
        normalized maximum force generated at the lengthening phase
    af  : float, optional (default=0.25)
        shape factor
    s  : float, optional (default=2)
        to double the slope of the force-velocity curve at zero velocity
    gammav  : float, optional (default=5.67)
        to ensure the smooth transition of the lengthening phase

    Returns
    -------
    fvce  : float
        normalized force of the muscle contractile element
    """

    vmmax = vmmax*lmopt
    d1 = vmmax*af*(fmlen - 1)/(s*(af + 1))
    d2 = s*(af + 1)/(vmmax*af*(gammav + 1)**2)
    d3 = (fmlen - 1)*gammav**2/(gammav + 1)**2 + 1
    lbd = 1 - np.exp(-3.82*a) + a*np.exp(-3.82)
    if vm <= 0:                # isometric and concentric activation
        fvce = (lbd*vmmax + vm)/(lbd*vmmax - vm/af)
    elif 0 < vm <= gammav*d1:  # slow lengthening
        fvce = (fmlen*vm + d1)/(vm + d1)
    elif vm > gammav*d1:       # fast lengthening
        fvce = d3 + d2*vm

    return fvce
```

We can invert this equation to get an expression for muscle velocity as function of the contractile element force:

```python
def vmfce_M03(fvce=1, lmopt=1, a=1, vmmax=1, fmlen=1.5, af=0.25, s=2, gammav=5.67):
    """McLean (2003) contractile element velocity as function of CE force.

    Parameters
    ----------
    fvce  : float, optional (default=1)
        normalized muscle force
    lmopt  : float, optional (default=1)
        optimal muscle fiber length
    a  : float, optional (default=1)
        muscle activation level
    vmmax  : float, optional (default=1)
        maximum muscle velocity for concentric activation
    fmlen  : float, optional (default=1.5)
        normalized maximum force generated at the lengthening phase
    af  : float, optional (default=0.25)
        shape factor
    s  : float, optional (default=2)
        to double the slope of the force-velocity curve at zero velocity
    gammav  : float, optional (default=5.67)
        to ensure the smooth transition of the lengthening phase

    Returns
    -------
    fvce  : float
        muscle velocity
    """

    vmmax = vmmax*lmopt
    d1 = vmmax*af*(fmlen - 1)/(s*(af + 1))
    d2 = s*(af + 1)/(vmmax*af*(gammav + 1)**2)
    d3 = (fmlen - 1)*gammav**2/(gammav + 1)**2 + 1
    lbd = 1 - np.exp(-3.82*a) + a*np.exp(-3.82)
    if 0 <= fvce <= 1:                   # isometric and concentric activation
        vm = (lbd*vmmax*(1 - fvce))/(1 + fvce/af)
    elif 1 < fvce <= gammav*d1*d2 + d3:  # slow lengthening
        vm = d1*(fvce - 1)/(fmlen - fvce)
    elif fvce > gammav*d1*d2 + d3:       # fast lengthening
        vm = (fvce - d3)/d2

    return vm
```

Let's use these functions to compute muscle force as a function of the muscle velocity considering two levels of activation:

```python
vm1_T03 = np.linspace(-1, 1, 201)
fce1_T03 = np.zeros(vm1_T03.size)
vm2_T03 = np.linspace(-.63, .63, 201)
fce2_T03 = np.zeros(vm2_T03.size)
for i in range(len(vm1_T03)):
    fce1_T03[i] = fvce_T03(vm=vm1_T03[i])
    fce2_T03[i] = fvce_T03(vm=vm2_T03[i], a=0.5)
```

```python
vm1_M03 = np.linspace(-1, 1, 201)
fce1_M03 = np.zeros(vm1_M03.size)
vm2_M03 = np.linspace(-.63, .63, 201)
fce2_M03 = np.zeros(vm2_M03.size)
for i in range(len(vm1_M03)):
    fce1_M03[i] = fvce_M03(vm=vm1_M03[i])
    fce2_M03[i] = fvce_M03(vm=vm2_M03[i], a=0.5)
fce2_M03 = fce2_M03*0.5
```

```python
fig, (ax1, ax2) = plt.subplots(nrows=1, ncols=2, sharex=True, sharey=True, figsize=(10, 4))
ax1.plot(vm1_T03, fce1_T03, 'b', label='T03)')
ax1.plot(vm1_M03, fce1_M03, 'g', label='M03)')
ax1.set_ylabel('Normalized force')
ax1.set_xlabel('Normalized velocity')
ax1.text(-.9, 1.5, 'Activation = 1.0')
ax2.plot(vm2_T03, fce2_T03, 'b', label='T03')
ax2.plot(vm2_M03, fce2_M03, 'g', label='M03')
ax2.text(-.9, 1.5, 'Activation = 0.5')
ax2.set_xlabel('Normalized velocity')
ax2.legend(loc='best')
plt.suptitle('Force-velocity relationship of the contractile element', y=1.05, fontsize=16)
plt.tight_layout()
plt.show()
```

Identical results for the shortening phase when $a=1$ and similar results for the lengthening phase when the same parameters are used.

#### Muscle power

The muscle power is the product between force and velocity:

```python
P_T03 = np.abs(fce1_T03*vm1_T03)
```

Let's visualize the muscle power only for the concentric phase (muscle shortening):

```python
plt.figure(figsize=(7, 4))
plt.plot(vm1_T03[:101], fce1_T03[:101], 'b', label='Force')
plt.xlabel('Normalized velocity')
plt.ylabel('Normalized force', color='b')
#plt.legend(loc='upper left')
plt.gca().invert_xaxis()
plt.gca().twinx()
plt.plot(vm1_T03[:101], P_T03[:101], 'g', label='Power')
plt.ylabel('Normalized power', color='g')
#plt.legend(loc='upper right')
plt.suptitle('Muscle power', y=1, fontsize=16)
plt.show()
```

#### Force-length-velocity relationship

Let's visualize the effects of the length and velocity on the total (active plus passive) muscle force:

```python
lms = np.linspace(0, 1.65, 101)
vms = np.linspace(-1, .76, 101)
fce_T03 = np.zeros(lms.size)
fpe_T03 = np.zeros(lms.size)
fm_T03 = np.zeros((lms.size, vms.size))
for i in range(len(lms)):
    fce_T03[i] = flce_T03(lm=lms[i])
    fpe_T03[i] = fpelm_T03(lm=lms[i])
    for j in range(len(vms)):
        fm_T03[j, i] = fvce_T03(vm=vms[j], flce=fce_T03[i]) + fpe_T03[i]
```

```python
lms = np.linspace(0, 1.65, 101)
vms = np.linspace(-1, .76, 101)
fce_M03 = np.zeros(lms.size)
fpe_M03 = np.zeros(lms.size)
fm_M03 = np.zeros((lms.size, vms.size))
for i in range(len(lms)):
    fce_M03[i] = flce_M03(lm=lms[i])
    fpe_M03[i] = fpelm_M03(lm=lms[i])
    for j in range(len(vms)):
        fm_M03[j, i] = fvce_M03(vm=vms[j])*fce_M03[i] + fpe_M03[i]
```

```python
from mpl_toolkits.mplot3d import Axes3D

def flv3dplot(ax, lm, vm, fm, model):
    # 3d plot
    lm2, vm2 = np.meshgrid(lm, vm)
    ax.plot_surface(lm2, vm2, fm, rstride=2, cstride=2, cmap=plt.cm.coolwarm,
                    linewidth=.5, antialiased=True)
    ax.plot(np.ones(vms.size), vms, fm[:, np.argmax(lm>=1)], 'w', linewidth=4)
    ax.plot(lm, np.zeros(lm.size), fm[np.argmax(vm>=0),:], 'w', linewidth=4)
    ax.set_xlim3d(lm[0], lm[-1])
    ax.set_ylim3d(vm[0], vm[-1])
    #ax.set_zlim3d(np.min(fm), np.max(fm))
    ax.set_zlim3d(0, 2)
    ax.set_xlabel('Normalized length')
    ax.set_ylabel('Normalized velocity')
    ax.set_zlabel('Normalized force')
    ax.view_init(20, 225)
    ax.locator_params(nbins=6)
    ax.text(-0.4, 0.7, 2.5, model, fontsize=14)

fig = plt.figure(figsize=(12, 6))
ax1 = fig.add_subplot(1, 2, 1, projection='3d')
flv3dplot(ax1, lms, vms, fm_T03, 'T03')
ax2 = fig.add_subplot(1, 2, 2, projection='3d')
flv3dplot(ax2, lms, vms, fm_M03, 'M03')
plt.suptitle('Force-length-velocity relationship', y=1, fontsize=16)
plt.tight_layout()
plt.show()
```

### Activation dynamics

Activation dynamics represents the fact that a muscle cannot instantly activate or deactivate because of the electrical and chemical processes involved and it is usually integrated with a Hill-type model. In its simplest form, the activation dynamics is generally represented as a first-order ODE.

Thelen2003Muscle employed the following first-order [ordinary differential equation (ODE)](http://en.wikipedia.org/wiki/Ordinary_differential_equation):

\begin{equation}
\frac{\mathrm{d}a}{\mathrm{d}t} = \dfrac{u-a}{\tau(a, u)}
\label{}
\end{equation}

with a lower activation bound to both activation and excitation.

where $u$ and $a$ are the muscle excitation and activation, respectively (both are function of time), and $\tau$ is a variable time constant to represent the activation and deactivation times, given by:

\begin{equation}
\tau(a, u) = \left\{
  \begin{array}{l l}
    t_{act}(0.5+1.5a) \quad & \text{if} \quad u > a\\
    \dfrac{t_{deact}}{(0.5+1.5a)} \quad & \text{if} \quad u \leq a
\end{array} \right.
\label{}
\end{equation}

Thelen2003Muscle adopted activation, $t_{act}$, and deactivation, $t_{deact}$, time constants for young adults equal to 15 and 50 ms, respectively (for old adults, Thelen2003Muscle adopted 15 and 60 ms, respectively).

McLean2003Muscle expressed the activation dynamics as the following first-order ODE:

\begin{equation}
\dfrac{\mathrm{d}a}{\mathrm{d}t} = (u - a)(c_1u + c_2)
\label{}
\end{equation}

with a lower activation bound to both activation and excitation.

where $c_1 + c_2$ is the activation rate constant (when $u = 1$), the inverse of $t_{act}$, and $c_2$ is the deactivation rate constant (when $u = 0$), the inverse of $t_{deact}$.
McLean2003Muscle adopted $c_1=3.3 s^{-1}$ and $c_2=16.7 s^{-1}$, resulting in time constants of 50 ms and 60 ms for activation and deactivation, respectively.

In Python, the numeric first-order ODE for the activation dynamics presented in Thelen2003Muscle can be expressed as:

```python
def actdyn_T03(t, a, t_act, t_deact, u_max, u_min, t0=0, t1=1):
    """Thelen (2003) activation dynamics, the derivative of `a` at `t`.

    Parameters
    ----------
    t  : float
        time instant [s]
    a  : float (0 <= a <= 1)
        muscle activation
    t_act  : float
        activation time constant [s]
    t_deact  : float
        deactivation time constant [s]
    u_max  : float (0 < u_max <= 1), optional (default=1)
        maximum value for muscle excitation
    u_min  : float (0 < u_min < 1), optional (default=0.01)
        minimum value for muscle excitation
    t0  : float [s], optional (default=0)
        initial time instant for muscle excitation equals to u_max
    t1  : float [s], optional (default=1)
        final time instant for muscle excitation equals to u_max

    Returns
    -------
    adot  : float
        derivative of `a` at `t`
    """

    u = excitation(t, u_max, u_min)
    if u > a:
        adot = (u - a)/(t_act*(0.5 + 1.5*a))
    else:
        adot = (u - a)/(t_deact/(0.5 + 1.5*a))

    return adot
```

In Python, the numeric first-order ODE for the activation dynamics presented in McLean2003Muscle can be expressed as:

```python
def actdyn_M03(t, a, t_act, t_deact, u_max=1, u_min=0.01, t0=0, t1=1):
    """McLean (2003) activation dynamics, the derivative of `a` at `t`.

    Parameters
    ----------
    t  : float
        time instant [s]
    a  : float (0 <= a <= 1)
        muscle activation
    t_act  : float
        activation time constant [s]
    t_deact  : float
        deactivation time constant [s]
    u_max  : float (0 < u_max <= 1), optional (default=1)
        maximum value for muscle excitation
    u_min  : float (0 < u_min < 1), optional (default=0.01)
        minimum value for muscle excitation
    t0  : float [s], optional (default=0)
        initial time instant for muscle excitation equals to u_max
    t1  : float [s], optional (default=1)
        final time instant for muscle excitation equals to u_max

    Returns
    -------
    adot  : float
        derivative of `a` at `t`
    """

    c2 = 1/t_deact
    c1 = 1/t_act - c2
    u = excitation(t, u_max, u_min)
    adot = (u - a)*(c1*u + c2)

    return adot
```

Let's simulate the activation signal for a rectangular function as excitation signal:

```python
def excitation(t, u_max=1, u_min=0.01, t0=0.1, t1=0.4):
    """Excitation signal, a square wave.

    Parameters
    ----------
    t  : float
        time instant [s]
    u_max  : float (0 < u_max <= 1), optional (default=1)
        maximum value for muscle excitation
    u_min  : float (0 < u_min < 1), optional (default=0.01)
        minimum value for muscle excitation
    t0  : float [s], optional (default=0.1)
        initial time instant for muscle excitation equals to u_max
    t1  : float [s], optional (default=0.4)
        final time instant for muscle excitation equals to u_max

    Returns
    -------
    u  : float (0 < u <= 1)
        excitation signal
    """

    u = u_min
    if t >= t0 and t <= t1:
        u = u_max

    return u
```

We will solve the equation for $a$ by numerical integration using the [`scipy.integrate.ode`](http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.integrate.ode.html) class of numeric integrators, particularly the `dopri5`, an explicit runge-kutta method of order (4)5 due to Dormand and Prince (a.k.a. ode45 in Matlab):

```python
import warnings
def actdyn_ode45(fun, t0=0, t1=1, a0=0, t_act=0.015, t_deact=0.050, u_max=1, u_min=0.01):
    # Runge-Kutta (4)5 due to Dormand & Prince with variable stepsize ODE solver

    f = ode(fun).set_integrator('dopri5', nsteps=1, max_step=0.01, atol=1e-8)
    f.set_initial_value(a0, t0).set_f_params(t_act, t_deact, u_max, u_min)
    # suppress Fortran warning
    warnings.filterwarnings("ignore", category=UserWarning)
    data = []
    while f.t < t1:
        f.integrate(t1, step=True)
        data.append([f.t, excitation(f.t, u_max, u_min), np.max([f.y, u_min])])
    warnings.resetwarnings()
    data = np.array(data)

    return data
```

Solving the problem for two different maximum excitation levels:

```python
# using the values for t_act and t_deact from Thelen2003Muscle for both models
act1_T03 = actdyn_ode45(fun=actdyn_T03, u_max=1.0)
act2_T03 = actdyn_ode45(fun=actdyn_T03, u_max=0.5)
act1_M03 = actdyn_ode45(fun=actdyn_M03, u_max=1.0)
act2_M03 = actdyn_ode45(fun=actdyn_M03, u_max=0.5)
# using the values for t_act and t_deact from McLean2003Muscle
act3_M03 = actdyn_ode45(fun=actdyn_M03, u_max=1.0, t_act=0.050, t_deact=0.060)
act4_M03 = actdyn_ode45(fun=actdyn_M03, u_max=0.5, t_act=0.050, t_deact=0.060)
```

And the results:

```python
fig, axs = plt.subplots(nrows=2, ncols=2, sharex=True, sharey=True, figsize=(10, 6))
axs[0, 0].plot(act1_T03[:, 0], act1_T03[:, 1], 'r:', label='Excitation')
axs[0, 0].plot(act1_T03[:, 0], act1_T03[:, 2], 'b', label='T03 [15, 50] ms')
axs[0, 0].plot(act1_M03[:, 0], act1_M03[:, 2], 'g', label='M03 [15, 50] ms')
axs[0, 0].set_ylabel('Level')
axs[0, 1].plot(act2_T03[:, 0], act2_T03[:, 1], 'r:', label='Excitation')
axs[0, 1].plot(act2_T03[:, 0], act2_T03[:, 2], 'b', label='T03 [15, 50] ms')
axs[0, 1].plot(act2_M03[:, 0], act2_M03[:, 2], 'g', label='M03 [15, 50] ms')
axs[1, 1].set_xlabel('Time (s)')
axs[0, 1].legend()
axs[1, 0].plot(act1_T03[:, 0], act1_T03[:, 1], 'r:', label='Excitation')
axs[1, 0].plot(act1_T03[:, 0], act1_T03[:, 2], 'b', label='T03 [15, 50] ms')
axs[1, 0].plot(act3_M03[:, 0], act3_M03[:, 2], 'g', label='M03 [50, 60] ms')
axs[1, 0].set_xlabel('Time (s)')
axs[1, 0].set_ylabel('Level')
axs[1, 1].plot(act2_T03[:, 0], act2_T03[:, 1], 'r:', label='Excitation')
axs[1, 1].plot(act2_T03[:, 0], act2_T03[:, 2], 'b', label='T03 [15, 50] ms')
axs[1, 1].plot(act4_M03[:, 0], act4_M03[:, 2], 'g', label='M03 [50, 60] ms')
axs[1, 1].set_xlabel('Time (s)')
axs[1, 1].legend()
plt.suptitle('Activation dynamics', y=1, fontsize=16)
plt.tight_layout()
plt.show()
```

Similar results when the same parameters are used (first row), but different bahavior when the typical values of each study are compared (second row).

### Muscle modeling parameters

We have seen two types of parameters in the muscle modeling: parameters related to the mathematical functions used to model the muscle and tendon behavior and parameters related to the properties of specific muscles and tendons (e.g., maximum isometric force, optimal fiber length, pennation angle, and tendon slack). In general the first type of parameters are independent of the muscle-tendon unit being modeled (but dependent of the model!) while the second type of parameters is changed for each muscle-tendon unit (for instance, see http://isbweb.org/data/delp/ for some of these parameters).

### Limitations of Hill-type muscle models

As with any modeling, Hill-type muscle models are a simplification of the reality. For instance, a typical Hill-type muscle model (as implemented here) does not capture time-dependent muscle behavior, such as force depression after quick muscle shortening, force enhancement after quick muscle lengthening, viscoelastic  properties (creep and relaxation), and muscle fatigue (Zatsiorsky and Prilutsky, 2012). There are enhanced models that capture these properties but it seems their complexity are not worthy for the most common applications of human movement simulation.

## Exercises

1. The results presented in this text depend on the parameters used in the model. These parameters may vary because of different properties of the muscle and tendon but also because different mathematical functions may be used.
 a. Change some of the parameters and reproduce the plots shown here and discuss these results (e.g., use the parameters for different muscles from OpenSim or the data from [http://isbweb.org/data/delp/](http://isbweb.org/data/delp/)).
 b. Select another reference (e.g., Anderson, 2007) about muscle modeling that uses different mathematical functions and repeat the previous item.

## References

- Anderson C (2007) [Equations for Modeling the Forces Generated by Muscles and Tendons](https://docs.google.com/viewer?url=https%3A%2F%2Fsimtk.org%2Fdocman%2Fview.php%2F124%2F604%2FMuscleAndTendonForcesClayAnderson20070521.doc) ([PDF](https://drive.google.com/open?id=0BxbW72zV7WmUVUh0MldGOGZ6aHc&authuser=0)). BioE215 Physics-based Simulation of Biological Structures.
- Erdemir A, McLean S, Herzog W, van den Bogert AJ (2007) [Model-based estimation of muscle forces exerted during movements](http://www.ncbi.nlm.nih.gov/pubmed/17070969). Clinical Biomechanics, 22, 131–154.
- He J, Levine WS, Loeb GE (1991) [Feedback gains for correcting small perturbations to standing posture](https://drive.google.com/open?id=0BxbW72zV7WmUekRXY09GSEhUVlE&authuser=0). IEEE Transactions on Automatic Control, 36, 322–332.
- McLean SG, Su A, van den Bogert AJ (2003) [Development and validation of a 3-D model to predict knee joint loading during dynamic movement](http://www.ncbi.nlm.nih.gov/pubmed/14986412). Journal of Biomechanical Engineering, 125, 864-74.
- McMahon TA (1984) [Muscles, Reflexes, and Locomotion](https://archive.org/details/McMahonTAMusclesReflexesAndLocomotionPrincetonUniversityPress1984). Princeton University Press, Princeton, New Jersey.
- Millard M, Uchida T, Seth A, Delp SL (2013) [Flexing computational muscle: modeling and simulation of musculotendon dynamics](http://www.ncbi.nlm.nih.gov/pubmed/23445050). Journal of Biomechanical Engineering, 135, 021005.
- Nigg BM and Herzog W (2006) [Biomechanics of the Musculo-skeletal System](https://books.google.com.br/books?id=hOIeAQAAIAAJ&dq=editions:ISBN0470017678). 3rd Edition. Wiley.
- Robertson G, Caldwell G, Hamill J, Kamen G (2013) [Research Methods in Biomechanics](http://books.google.com.br/books?id=gRn8AAAAQBAJ). 2nd Edition. Human Kinetics.
- Thelen DG (2003) [Adjustment of muscle mechanics model parameters to simulate dynamic contractions in older adults](http://homepages.cae.wisc.edu/~thelen/pubs/jbme03.pdf). Journal of Biomechanical Engineering, 125(1):70–77.
- Tsianos GA and Loeb GE (2013) [Muscle Physiology and Modeling](http://www.scholarpedia.org/article/Muscle_Physiology_and_Modeling). Scholarpedia, 8(10):12388.
- Winters JM (1990) [Hill-based muscle models: a systems engineering perspective](http://link.springer.com/chapter/10.1007%2F978-1-4613-9030-5_5). In [Multiple Muscle Systems: Biomechanics and Movement Organization](http://link.springer.com/book/10.1007/978-1-4613-9030-5), edited by JM Winters and SL Woo, Springer-Verlag, New York.
- Winters JM (1995) [An Improved Muscle-Reflex Actuator for Use in Large-Scale Neuromusculoskeletal Models](http://www.ncbi.nlm.nih.gov/pubmed/7486344). Annals of Biomedical Engineering, 23, 359–374.
- Zajac FE (1989) [Muscle and tendon: properties, models, scaling and application to biomechanics and motor control](http://www.ncbi.nlm.nih.gov/pubmed/2676342). Critical Reviews in Biomedical Engineering 17:359-411.
- Zatsiorsky V and Prilutsky B (2012) [Biomechanics of Skeletal Muscles](http://books.google.com.br/books?id=THXfHT8L5MEC). Human Kinetics.