original_docstring: 
      Initialization from Sitzmann et al.,
      "Implicit Neural Representations with Periodic Activation Functions"
      (2020)
      for MLPs with sine activations.
  
      When used on the first layer, `factor` can be thought of as a sort of (inverse)
      length scale of the MLP. Sitzmann recommends factor=30 for first layer, but this can
      cause severe overfitting when you have sparse data and regularization via a smaller
      factor can encourage better regularization/generalization.
      
docstring: Initialization from Sitzmann et al.,
  "Implicit Neural Representations with Periodic Activation Functions"
  (2020)
  for MLPs with sine activations.
  
  When used on the first layer, `factor` can be thought of as a sort of (inverse)
  length scale of the MLP. Sitzmann recommends factor=30 for first layer, but this can
  cause severe overfitting when you have sparse data and regularization via a smaller
  factor can encourage better regularization/generalization.
docstring_tokens: Initialization from Sitzmann et al . " Implicit Neural Representations with Periodic Activation Functions " ( 2020 ) for MLPs with sine activations . When used on the first layer ` factor ` can be thought of as a sort of ( inverse ) length scale of the MLP . Sitzmann recommends factor = 30 for first layer but this can cause severe overfitting when you have sparse data and regularization via a smaller factor can encourage better regularization / generalization .
code: def siren_w_init(factor=None):
      """
      Initialization from Sitzmann et al.,
      "Implicit Neural Representations with Periodic Activation Functions"
      (2020)
      for MLPs with sine activations.
  
      When used on the first layer, `factor` can be thought of as a sort of (inverse)
      length scale of the MLP. Sitzmann recommends factor=30 for first layer, but this can
      cause severe overfitting when you have sparse data and regularization via a smaller
      factor can encourage better regularization/generalization.
      """
      factor = 1.0 if factor is None else factor
      # scale=2.0 causes weights to init from U[-sqrt(6/n), sqrt(6/n)]
      # 1800 = multiply by 30^2 as I understand the paper suggests
      # (square since scale is inside the sqrt)
      return partial(
          jax.nn.initializers.variance_scaling, 2.0 * factor ** 2, "fan_in", "uniform",
      )
code_tokens: def siren_w_init ( factor = None ) : factor = 1.0 if factor is None else factor return partial ( jax . nn . initializers . variance_scaling , 2.0 * factor ** 2 , "fan_in" , "uniform" , )
short_docstring: Initialization from Sitzmann et al.,
  "Implicit Neural Representations with Periodic Activation Functions"
  (2020)
  for MLPs with sine activations.
short_docstring_tokens: Initialization from Sitzmann et al . " Implicit Neural Representations with Periodic Activation Functions " ( 2020 ) for MLPs with sine activations .
comment: """
      Initialization from Sitzmann et al.,
      "Implicit Neural Representations with Periodic Activation Functions"
      (2020)
      for MLPs with sine activations.
  
      When used on the first layer, `factor` can be thought of as a sort of (inverse)
      length scale of the MLP. Sitzmann recommends factor=30 for first layer, but this can
      cause severe overfitting when you have sparse data and regularization via a smaller
      factor can encourage better regularization/generalization.
      """$SEP$# scale=2.0 causes weights to init from U[-sqrt(6/n), sqrt(6/n)]$SEP$# 1800 = multiply by 30^2 as I understand the paper suggests$SEP$# (square since scale is inside the sqrt)