title: Fade texture inner borders to transparent in LibGDX using openGL shaders (glsl)
body: <p>I'm currently working on a tile game in LibGDX and I'm trying to get a &quot;fog of war&quot; effect by obscuring unexplored tiles. The result I get from this is a dynamically generated black texture of the size of the screen that only covers unexplored tiles leaving the rest of the background visible. This is an example of the fog texture rendered on top of a white background:</p>
  <p><a href="https://i.stack.imgur.com/ObnxF.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ObnxF.png" alt="Fog texture rendered on top of a white background" /></a></p>
  <p>What I'm now trying to achieve is to dynamically fade the inner borders of this texture to make it look more like a fog that slowly thickens instead of just a bunch of black boxes put together on top of the background.</p>
  <p>Googling the problem I found out I could use shaders to do this, so I tried to learn some glsl (I'm at the very start with shaders) and I came up with this shader:</p>
  <p>VertexShader:</p>
  <pre><code>//attributes passed from openGL
  attribute vec3 a_position;
  attribute vec2 a_texCoord0;
  
  //variables visible from java
  uniform mat4 u_projTrans;
  
  //variables shared between fragment and vertex shader
  varying vec2 v_texCoord0;
  
  void main() {
  
      v_texCoord0 = a_texCoord0;
      gl_Position = u_projTrans * vec4(a_position, 1f);
  }
  </code></pre>
  <p>FragmentShader:</p>
  <pre><code>//variables shared between fragment and vertex shader
  varying vec2 v_texCoord0;
  
  //variables visible from java
  uniform sampler2D u_texture;
  uniform vec2 u_textureSize;
  uniform int u_length;
  
  void main() {
  
      vec4 texColor = texture2D(u_texture, v_texCoord0);
      vec2 step = 1.0 / u_textureSize;
  
      if(texColor.a &gt; 0) {
  
          int maxNearPixels = (u_length * 2 + 1) * (u_length * 2 + 1) - 1;
          for(int i = 0; i &lt;= u_length; i++) {
  
              for(float j = 0; j &lt;= u_length; j++) {
  
                  if(i != 0 || j != 0) {
  
                      texColor.a -= (1 - texture2D(u_texture, v_texCoord0 + vec2(step.x * float(i), step.y * float(j))).a) / float(maxNearPixels);
                      texColor.a -= (1 - texture2D(u_texture, v_texCoord0 + vec2(-step.x * float(i), step.y * float(j))).a) / float(maxNearPixels);
                      texColor.a -= (1 - texture2D(u_texture, v_texCoord0 + vec2(step.x * float(i), -step.y * float(j))).a) / float(maxNearPixels);
                      texColor.a -= (1 - texture2D(u_texture, v_texCoord0 + vec2(-step.x * float(i), -step.y * float(j))).a) / float(maxNearPixels);
                  }
              }
          }
      }
  
      gl_FragColor = texColor;
  }
  </code></pre>
  <p>This is the result I got setting a length of 20:</p>
  <p><a href="https://i.stack.imgur.com/T3fXo.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/T3fXo.png" alt="Faded fog" /></a></p>
  <p>So the shader I wrote kinda works, but has terrible performance cause it's O(n^2) where n is the length of the fade in pixels (so it can be very high, like 60 or even 80). It also has some problems, like that the edges are still a bit too sharp (I'd like a smother transition) and some of the angles of the border are less faded than others (I'd like to have a fade uniform everywhere).</p>
  <p>I'm a little bit lost at this point: is there anything I can do to make it better and faster? Like I said I'm new to shaders, so: is it even the right way to use shaders?</p>