<p>I have been trying to run an external-dns pod using the guide provided by k8s-sig group. I have followed every step of the guide, and getting the below error.</p>
<pre><code>time=&quot;2021-02-27T13:27:20Z&quot; level=error msg=&quot;records retrieval failed: failed to list hosted zones: WebIdentityErr: failed to retrieve credentials\ncaused by: AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity\n\tstatus code: 403, request id: 87a3ca86-ceb0-47be-8f90-25d0c2de9f48&quot;
</code></pre>
<p>I had created AWS IAM policy using Terraform, and it was successfully created. Except IAM Role for service account for which I had used <code>eksctl</code>, everything else has been spun via Terraform.</p>
<p>But then I got hold of this <a href="https://www.google.com/amp/s/www.freecodecamp.org/news/how-to-setup-dns-for-a-website-using-kubernetes-eks-and-nginx/amp/" rel="noreferrer">article</a> which says creating AWS IAM policy using awscli would eliminate this error. So I deleted the policy created using Terraform, and recreated it with awscli. Yet, it is throwing the same error error.</p>
<p><strong>Below is my external dns yaml file.</strong></p>
<pre><code>apiVersion: v1
kind: ServiceAccount
metadata:
  name: external-dns
  # If you're using Amazon EKS with IAM Roles for Service Accounts, specify the following annotation.
  # Otherwise, you may safely omit it.
  annotations:
    # Substitute your account ID and IAM service role name below.
    eks.amazonaws.com/role-arn: arn:aws:iam::268xxxxxxx:role/eksctl-ats-Eks1-addon-iamserviceaccoun-Role1-WMLL93xxxx
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: external-dns
rules:
- apiGroups: [&quot;&quot;]
  resources: [&quot;services&quot;,&quot;endpoints&quot;,&quot;pods&quot;]
  verbs: [&quot;get&quot;,&quot;watch&quot;,&quot;list&quot;]
- apiGroups: [&quot;extensions&quot;,&quot;networking.k8s.io&quot;]
  resources: [&quot;ingresses&quot;]
  verbs: [&quot;get&quot;,&quot;watch&quot;,&quot;list&quot;]
- apiGroups: [&quot;&quot;]
  resources: [&quot;nodes&quot;]
  verbs: [&quot;list&quot;,&quot;watch&quot;]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: external-dns-viewer
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: external-dns
subjects:
- kind: ServiceAccount
  name: external-dns
  namespace: default
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: external-dns
spec:
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: external-dns
  template:
    metadata:
      labels:
        app: external-dns
    spec:
      serviceAccountName: external-dns
      containers:
      - name: external-dns
        image: k8s.gcr.io/external-dns/external-dns:v0.7.6
        args:
        - --source=service
        - --source=ingress
        - --domain-filter=xyz.com # will make ExternalDNS see only the hosted zones matching provided domain, omit to process all available hosted zones
        - --provider=aws
        - --policy=upsert-only # would prevent ExternalDNS from deleting any records, omit to enable full synchronization
        - --aws-zone-type=public # only look at public hosted zones (valid values are public, private or no value for both)
        - --registry=txt
        - --txt-owner-id=Z0471542U7WSPZxxxx
      securityContext:
        fsGroup: 65534 # For ExternalDNS to be able to read Kubernetes and AWS token files
</code></pre>
<p>I am scratching my head as there is no proper solution to this error anywhere in the net. Hoping to find a solution to this issue in this forum.</p>
<p>End result must show something like below and fill up records in hosted zone.</p>
<pre><code>time=&quot;2020-05-05T02:57:31Z&quot; level=info msg=&quot;All records are already up to date&quot;
</code></pre>

<p>I also struggled with this error.</p>
<p>The problem was in the definition of the trust relationship.</p>
<p>You can see in some offical aws tutorials (like <a href="https://docs.aws.amazon.com/eks/latest/userguide/create-service-account-iam-policy-and-role.html" rel="noreferrer">this</a>) the following setup:</p>
<pre><code>{
  &quot;Version&quot;: &quot;2012-10-17&quot;,
  &quot;Statement&quot;: [
    {
      &quot;Effect&quot;: &quot;Allow&quot;,
      &quot;Principal&quot;: {
        &quot;Federated&quot;: &quot;arn:aws:iam::${AWS_ACCOUNT_ID}:oidc-provider/${OIDC_PROVIDER}&quot;
      },
      &quot;Action&quot;: &quot;sts:AssumeRoleWithWebIdentity&quot;,
      &quot;Condition&quot;: {
        &quot;StringEquals&quot;: {
          &quot;${OIDC_PROVIDER}:sub&quot;: &quot;system:serviceaccount:&lt;my-namespace&gt;:&lt;my-service-account&gt;&quot;
        }
      }
    }
  ]
}
</code></pre>
<p><strong>Option 1 for failure</strong></p>
<p>My problem was that I passed the a wrong value for <code>my-service-account</code> at the end of <code>${OIDC_PROVIDER}:sub</code> in the <code>Condition</code> part.</p>
<p><strong>Option 2 for failure</strong></p>
<p>After the previous fix - I still faced the same error - it was solved by following <a href="https://aws.amazon.com/blogs/opensource/introducing-fine-grained-iam-roles-service-accounts/" rel="noreferrer">this aws tutorial</a> which shows the output of using the eksctl with the command below:</p>
<pre><code>eksctl create iamserviceaccount \
                --name my-serviceaccount \
                --namespace &lt;your-ns&gt; \
                --cluster &lt;your-cluster-name&gt; \
                --attach-policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess \
                --approve
</code></pre>
<p>When you look at the output in the <em>trust relationship</em> tab in the AWS web console - you can see that an additional condition was added with the postfix of <code>:aud</code> and the value of <code>sts.amazonaws.com</code>:</p>
<p><a href="https://i.stack.imgur.com/ezhkK.png" rel="noreferrer"><img src="https://i.stack.imgur.com/ezhkK.png" alt="enter image description here" /></a></p>
<p>So this need to be added after the <code>&quot;${OIDC_PROVIDER}:sub&quot;</code>  condition.</p>
