{"action":"opened","issue":{"url":"https://api.github.com/repos/boto/boto3/issues/880","repository_url":"https://api.github.com/repos/boto/boto3","labels_url":"https://api.github.com/repos/boto/boto3/issues/880/labels{/name}","comments_url":"https://api.github.com/repos/boto/boto3/issues/880/comments","events_url":"https://api.github.com/repos/boto/boto3/issues/880/events","html_url":"https://github.com/boto/boto3/issues/880","id":187595945,"number":880,"title":"Boto3 1.4.1 client `describe_instances` API returns null result in case there are more than 10 tags specified in the `Filters`","user":{"login":"romanrev","id":2794589,"avatar_url":"https://avatars.githubusercontent.com/u/2794589?v=3","gravatar_id":"","url":"https://api.github.com/users/romanrev","html_url":"https://github.com/romanrev","followers_url":"https://api.github.com/users/romanrev/followers","following_url":"https://api.github.com/users/romanrev/following{/other_user}","gists_url":"https://api.github.com/users/romanrev/gists{/gist_id}","starred_url":"https://api.github.com/users/romanrev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/romanrev/subscriptions","organizations_url":"https://api.github.com/users/romanrev/orgs","repos_url":"https://api.github.com/users/romanrev/repos","events_url":"https://api.github.com/users/romanrev/events{/privacy}","received_events_url":"https://api.github.com/users/romanrev/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-11-06T22:49:47Z","updated_at":"2016-11-06T22:49:47Z","closed_at":null,"body":"It seems that the [issue with > 10 tags in the filters](https://github.com/boto/boto/issues/3637) that we previously observed working with Boto 2 isn't inherent to Boto 2, as we have since replicated the issue with Boto 3 as well:\r\n\r\nIn case we use more than 10 tag/value pairs in the call to `describe_instances` from [`client.describe_instances`](http://boto3.readthedocs.io/en/latest/reference/services/ec2.html#EC2.Client.describe_instances) we get an empty set instead of the list of instances. Below is an example:\r\n\r\n\t>>> from aws_common3 import get_ec2_instances\r\n\t>>> args = dotdict()\r\n\t>>> args.tags = \"Availability:test-schedule Backup: CostCentre: CreatedBy: Division: Environment:Prod ManagedBy: Name:test-snapshots-2vols Owner: Tag10:\"\r\n\t>>> tags = [ { t.split(\":\")[0] : t.split(\":\")[1] } for t in args.tags.split(\" \") ]\r\n\t>>> tags\r\n\t[{'Availability': 'test-schedule'}, {'Backup': ''}, {'CostCentre': ''}, {'CreatedBy': ''}, {'Division': ''}, {'Environment': 'Prod'}, {'ManagedBy': ''}, {'Name': 'test-snapshots-2vols'}, {'Owner': ''}, {'Tag10': ''}]\r\n\t>>> filters = [ { \"Name\": \"tag:\"+t.keys()[0], \"Values\": [ t.values()[0] ] } for t in tags ]\r\n\t>>> filters\r\n\t[{'Values': ['test-schedule'], 'Name': 'tag:Availability'}, {'Values': [''], 'Name': 'tag:Backup'}, {'Values': [''], 'Name': 'tag:CostCentre'}, {'Values': [''], 'Name': 'tag:CreatedBy'}, {'Values': [''], 'Name': 'tag:Division'}, {'Values': ['Prod'], 'Name': 'tag:Environment'}, {'Values': [''], 'Name': 'tag:ManagedBy'}, {'Values': ['test-snapshots-2vols'], 'Name': 'tag:Name'}, {'Values': [''], 'Name': 'tag:Owner'}, {'Values': [''], 'Name': 'tag:Tag10'}]\r\n\t>>> instances = get_ec2_instances(filters=filters, region='ap-southeast-2', profile='myawsaccount')\r\n\t>>> len(instances)\r\n\t1\r\n\t>>> instances[0]['InstanceId']\r\n\t'i-01a9d25e73544287b'\r\n\r\nSo for 10 tags we get the instance in the response list. Now adding one more existing tag (`Tag11`) to the filter:\r\n\r\n\t>>> args.tags = \"Availability:test-schedule Backup: CostCentre: CreatedBy: Division: Environment:Prod ManagedBy: Name:test-snapshots-2vols Owner: Tag10: Tag11:\"\r\n\t>>> tags = [ { t.split(\":\")[0] : t.split(\":\")[1] } for t in args.tags.split(\" \") ]\r\n\t>>> filters = [ { \"Name\": \"tag:\"+t.keys()[0], \"Values\": [ t.values()[0] ] } for t in tags ]\r\n\t>>> instances = get_ec2_instances(filters=filters, region='ap-southeast-2', profile='myawsaccount')\r\n\t>>> len(instances)\r\n\t0\r\n\r\nAs you can see, in case there are 11 tags we get an empty list of instances. Now if remove a different tag (`Tag10`) from the filter, keeping the `Tag11` at the same time to get back to 10 tags in total:\r\n\r\n\t>>> args.tags = \"Availability:test-schedule Backup: CostCentre: CreatedBy: Division: Environment:Prod ManagedBy: Name:test-snapshots-2vols Owner: Tag11:\"\r\n\t>>> tags = [ { t.split(\":\")[0] : t.split(\":\")[1] } for t in args.tags.split(\" \") ]\r\n\t>>> filters = [ { \"Name\": \"tag:\"+t.keys()[0], \"Values\": [ t.values()[0] ] } for t in tags ]\r\n\t>>> instances = get_ec2_instances(filters=filters, region='ap-southeast-2', profile='myawsaccount')\r\n\t>>> len(instances)\r\n\t1\r\n\t>>> instances[0]['InstanceId']\r\n\t'i-01a9d25e73544287b'\r\n\r\nBelow is the reference to the `get_ec2_instances` and the dependent functions that are used in the example:\r\n\r\n\t@aws_list_result\r\n\t@call_aws_api\r\n\tdef get_ec2_instances(filters=None, region=None, role_arn=None, profile=None):\r\n\t    \"\"\"\r\n\t    Wrapper function to get a list of instances, optionally filtering them\r\n\t    \"\"\"\r\n\t    ec2_conn = get_aws_client('ec2', region, role_arn, profile)\r\n\t    res = ec2_conn.describe_instances(Filters=filters)\r\n\t    instances = [i for r in res['Reservations'] for i in r['Instances'] ]\r\n\t    return instances\r\n\r\n\tdef get_aws_client(name=None, region=None, role_arn=None, profile=None):\r\n\t    \"\"\" Obtain an AWS client connection: direct or by assuming a role\r\n\t    \"\"\"\r\n\t    session=get_aws_access(region, role_arn, profile)\r\n\t    return session.client(name)\r\n\r\n\tdef get_aws_access(region=None, role_arn=None, profile=None):\r\n\t    \"\"\" Obtain an Autoscale connection: direct or by assuming a role\r\n\t    \"\"\"\r\n\t    aws_access = {}\r\n\t    if region is None:\r\n\t        # FIXME: still need to use boto.utils as this functionality\r\n\t        # is lacking in Boto 3, see https://github.com/boto/boto3/issues/313\r\n\t        instance_metadata = boto.utils.get_instance_identity()\r\n\t        region = instance_metadata['document']['region']\r\n\t    if role_arn is None:\r\n\t        aws_access['key']=None\r\n\t        aws_access['secret']=None\r\n\t        aws_access['token']=None\r\n\t    else:\r\n\t        sts_client = boto3.client('sts')\r\n\t        role_obj = sts_client.assume_role(\r\n\t            RoleArn=role_arn,\r\n\t            RoleSessionName=\"AssumeRoleSession\"\r\n\t            )\r\n\t        aws_access['key']=role_obj.credentials['AccessKeyId']\r\n\t        aws_access['secret']=role_obj.credentials['SecretAccessKey']\r\n\t        aws_access['token']=role_obj.credentials['SessionToken']\r\n\t    aws_access['profile'] = profile\r\n\t    aws_access['region']=region\r\n\t    session = boto3.session.Session(region_name=aws_access['region'],\r\n\t            aws_access_key_id=aws_access['key'],\r\n\t            aws_secret_access_key=aws_access['secret'],\r\n\t            aws_session_token=aws_access['token'],\r\n\t            profile_name=aws_access['profile'])\r\n\t    return session\r\n\r\n\tclass dotdict(dict):\r\n\t    \"\"\"dot.notation access to dictionary attributes\r\n\t    >>> dotdict({ \"key1\": \"value1\", \"key2\": \"value2\" }).key1\r\n\t    'value1'\r\n\t    >>> dotdict({ \"key1\": { \"key2\": \"value2\"} }).key1\r\n\t    {'key2': 'value2'}\r\n\t    >>> dotdict({ \"key1\": { \"key2\": \"value2\"} }).key1.key2\r\n\t    Traceback (most recent call last):\r\n\t      File \"<stdin>\", line 1, in <module>\r\n\t    AttributeError: 'dict' object has no attribute 'key2'\r\n\t    \"\"\"\r\n\t    def __getattr__(self, attr):\r\n\t        return self.get(attr)\r\n\t    __setattr__= dict.__setitem__\r\n\t    __delattr__= dict.__delitem__"}}
{"actor":{"display_login":"romanrev"}}