<p>Complex AWS EKS / ENI / Route53 issue has us stumped. Need an expert.</p>
<p>Context:</p>
<p>We are working on dynamic game servers for a social platform (<a href="https://myxr.social" rel="nofollow noreferrer">https://myxr.social</a>) that transport game and video data using WebRTC / UDP SCTP/SRTP via <a href="https://MediaSoup.org" rel="nofollow noreferrer">https://MediaSoup.org</a></p>
<p>Each game server will have about 50 clients
Each client requires 2-4 UDP ports</p>
<p>Our working devops strategy
<a href="https://github.com/xr3ngine/xr3ngine/tree/dev/packages/ops" rel="nofollow noreferrer">https://github.com/xr3ngine/xr3ngine/tree/dev/packages/ops</a></p>
<p>We are provisioning these game servers using Kubernetes and <a href="https://agones.dev" rel="nofollow noreferrer">https://agones.dev</a></p>
<p>Mediasoup requires each server connection to a client be assigned individual ports. Each client will need two ports, one for sending data and one for receiving data; with a target maximum of about 50 users per server, this requires 100 ports per server be publicly accessible.</p>
<p>We need some way to route this UDP traffic to the corresponding gameserver. Ingresses appear to primarily handle HTTP(S) traffic, and configuring our NGINX ingress controller to handle UDP traffic assumes that we know our gameserver Services ahead of time, which we do not since the gameservers are spun up and down as they are needed.</p>
<p>Questions:</p>
<p>We see two possible ways to solve this problem.</p>
<p>Path 1</p>
<p>Assign each game server in the node group public IPs and then allocate ports for each client. Either IP v4 or v6. This would require SSL termination for IP ports in AWS. Can we use ENI and EKS to dynamically create and provision IP ports for each gameserver w/ SSL? Essentially expose these pods to the internet via a public subnet with them each having their own IP address or subdomain. <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html" rel="nofollow noreferrer">https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html</a> We have been referencing this documentation trying to figure out if this is possible.</p>
<p>Path 2</p>
<p>Create a subdomain (eg gameserver01.gs.xrengine.io, etc) dynamically for each gameserver w/ dynamic port allocation for each client (eg client 1 [30000-30004], etc). This seems to be limited by the ports accessible in the EKS fleet.</p>
<p>Are either of these approaches possible? Is one better? Can you give us some detail about how we should go about implementation?</p>

<p>The native way for receiving <a href="https://aws.amazon.com/about-aws/whats-new/2020/08/amazon-eks-now-supports-udp-load-balancing-with-network-load-balancer/" rel="nofollow noreferrer">UDP traffic on Amazon EKS</a> is by using a Kubernetes <a href="https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer" rel="nofollow noreferrer">Service of type Loadbalancer</a> with an extra annotation to get the <em>NLB</em>.</p>
<p>Example</p>
<pre><code>apiVersion: v1
kind: Service
metadata:
  name: my-game-app-service
  annotation:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
spec:
  selector:
    app: my-game-app
  ports:
    - name: outgoing-port   # choose your name
      protocol: UDP
      port: 9000            # choose your port
    - name: incoming-port   # choose your name
      protocol: UDP
      port: 9001            # choose your port
  type: LoadBalancer
</code></pre>
