title: How to implement current pytorch activation functions with parameters?
body: <p>I am looking for a simple way to use an activation function which exist in the pytorch library, but using some sort of parameter. for example:</p>
  
  <p>Tanh(x/10)</p>
  
  <p>The only way I came up with looking for solution was implementing the custom function completely from scratch. Is there any better/more elegant way to do this?</p>
  
  <p>edit:</p>
  
  <p>I am looking for some way to append to my model the function Tanh(x/10) rather than plain Tanh(x). Here is the relevant code block:</p>
  
  <pre><code>    self.model = nn.Sequential()
      for i in range(len(self.layers)-1):
          self.model.add_module("linear_layer_" + str(i), nn.Linear(self.layers[i], self.layers[i + 1]))
          if activations == None:
              self.model.add_module("activation_" + str(i), nn.Tanh())
          else:
              if activations[i] == "T":
                  self.model.add_module("activation_" + str(i), nn.Tanh())
              elif activations[i] == "R":
                  self.model.add_module("activation_" + str(i), nn.ReLU())
              else:
                  #no activation
                  pass
  </code></pre>