{"action":"created","issue":{"url":"https://api.github.com/repos/prestodb/presto/issues/10298","repository_url":"https://api.github.com/repos/prestodb/presto","labels_url":"https://api.github.com/repos/prestodb/presto/issues/10298/labels{/name}","comments_url":"https://api.github.com/repos/prestodb/presto/issues/10298/comments","events_url":"https://api.github.com/repos/prestodb/presto/issues/10298/events","html_url":"https://github.com/prestodb/presto/issues/10298","id":310045281,"number":10298,"title":"Query failed: Unable to load credentials from service endpoint","user":{"login":"sbmaggarwal","id":6919017,"avatar_url":"https://avatars3.githubusercontent.com/u/6919017?v=4","gravatar_id":"","url":"https://api.github.com/users/sbmaggarwal","html_url":"https://github.com/sbmaggarwal","followers_url":"https://api.github.com/users/sbmaggarwal/followers","following_url":"https://api.github.com/users/sbmaggarwal/following{/other_user}","gists_url":"https://api.github.com/users/sbmaggarwal/gists{/gist_id}","starred_url":"https://api.github.com/users/sbmaggarwal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sbmaggarwal/subscriptions","organizations_url":"https://api.github.com/users/sbmaggarwal/orgs","repos_url":"https://api.github.com/users/sbmaggarwal/repos","events_url":"https://api.github.com/users/sbmaggarwal/events{/privacy}","received_events_url":"https://api.github.com/users/sbmaggarwal/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-03-30T11:36:09Z","updated_at":"2018-03-31T01:30:45Z","closed_at":null,"author_association":"NONE","body":"I am trying to connect to S3 Bucket from the hive catalog in PrestoDB from HDP VM by Teradata.\r\n\r\nFor this, I configured the `hive-site.xml` file and added my AWS Access Key and Secret Key in the `/etc/hive/conf/hive-site.xml` file like:\r\n\r\n    <property>\r\n      <name>hive.s3.aws-access-key</name>\r\n      <value>something</value>\r\n    </property>\r\n    <property>\r\n      <name>hive.s3.aws-secret-key</name>\r\n      <value>some-other-thing</value>\r\n    </property>\r\n\r\n\r\nNow, my S3 Bucket URL path where `Parquet` files reside looks like:\r\n\r\n    https://s3.console.aws.amazon.com/s3/buckets/sb.mycompany.com/someFolder/anotherFolder/?region=us-east-2&tab=overview\r\n\r\nWhile Create an external table, I gave the location for S3 in the query as:\r\n\r\n    CREATE TABLE hive.project.data (... schema ...)\r\n    WITH ( format = 'PARQUET', \r\n      external_location = 's3://sb.mycompany.com/someFolder/anotherFolder/?region=us-east-2&tab=overview')\r\n\r\nPrestoDB **isn't able to connect** to S3 Buckets and giving this error:\r\n\r\n    Query 20180316_063813_11114_z22jg failed: Unable to load credentials from service endpoint\r\n\r\nI even restarted my PrestDB server after I added the Keys. Next, I tried adding my properties to `/home/presto/.prestoadmin/catalog/hive.properties`:\r\n\r\n    connector.name=hive-hadoop2\r\n    hive.metastore.uri=thrift://localhost:9083\r\n    hive.allow-drop-table=true\r\n    hive.allow-rename-table=true\r\n    hive.time-zone=UTC\r\n    hive.metastore-cache-ttl=0s\r\n    hive.s3.use-instance-credentials=false\r\n    hive.s3.aws-access-key=something\r\n    hive.s3.aws-secret-key=some-other-thing\r\n\r\nAgain restarted the PrestoDB server but still the same issue. \r\n\r\nI then modified the S3 Bucket location in query with bucket name only:\r\n\r\n    external_location = 's3://sb.mycompany.com'\r\n\r\nAnd with `s3a` scheme as well:\r\n\r\n    external_location = 's3a://sb.mycompany.com'\r\n\r\nBut the same issue is still present. What am I doing wrong?\r\n\r\n  [1]: https://prestodb.io/"},"comment":{"url":"https://api.github.com/repos/prestodb/presto/issues/comments/377656339","html_url":"https://github.com/prestodb/presto/issues/10298#issuecomment-377656339","issue_url":"https://api.github.com/repos/prestodb/presto/issues/10298","id":377656339,"user":{"login":"sbmaggarwal","id":6919017,"avatar_url":"https://avatars3.githubusercontent.com/u/6919017?v=4","gravatar_id":"","url":"https://api.github.com/users/sbmaggarwal","html_url":"https://github.com/sbmaggarwal","followers_url":"https://api.github.com/users/sbmaggarwal/followers","following_url":"https://api.github.com/users/sbmaggarwal/following{/other_user}","gists_url":"https://api.github.com/users/sbmaggarwal/gists{/gist_id}","starred_url":"https://api.github.com/users/sbmaggarwal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sbmaggarwal/subscriptions","organizations_url":"https://api.github.com/users/sbmaggarwal/orgs","repos_url":"https://api.github.com/users/sbmaggarwal/repos","events_url":"https://api.github.com/users/sbmaggarwal/events{/privacy}","received_events_url":"https://api.github.com/users/sbmaggarwal/received_events","type":"User","site_admin":false},"created_at":"2018-03-31T01:30:45Z","updated_at":"2018-03-31T01:30:45Z","author_association":"NONE","body":"@nezihyigitbasi Yes, I get this:\r\n\r\n```\r\nQuery 20180316_112407_00005_aj9x6 failed: Unable to load credentials from service endpoint\r\n\r\n=========   TECHNICAL DETAILS   =========\r\n[ Error message ]\r\nUnable to load credentials from service endpoint\r\n\r\n[ Session information ]\r\nClientSession{server=http://localhost:8080, user=presto, clientInfo=null, catalog=null, schema=null, timeZone=Zulu, locale=en_US, properties={}, transactionId=null, debug=true, quiet=false}\r\n\r\n[ Stack trace ]\r\ncom.amazonaws.AmazonClientException: Unable to load credentials from service endpoint\r\n\tat com.amazonaws.auth.EC2CredentialsFetcher.handleError(EC2CredentialsFetcher.java:180)\r\n\tat com.amazonaws.auth.EC2CredentialsFetcher.fetchCredentials(EC2CredentialsFetcher.java:159)\r\n\tat com.amazonaws.auth.EC2CredentialsFetcher.getCredentials(EC2CredentialsFetcher.java:82)\r\n\tat com.amazonaws.auth.InstanceProfileCredentialsProvider.getCredentials(InstanceProfileCredentialsProvider.java:104)\r\n\tat com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4016)\r\n\tat com.amazonaws.services.s3.AmazonS3Client.getBucketRegionViaHeadRequest(AmazonS3Client.java:4478)\r\n\tat com.amazonaws.services.s3.AmazonS3Client.fetchRegionFromCache(AmazonS3Client.java:4452)\r\n\tat com.amazonaws.services.s3.AmazonS3Client.resolveServiceEndpoint(AmazonS3Client.java:4426)\r\n\tat com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1167)\r\n\tat com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1152)\r\n\tat com.facebook.presto.hive.PrestoS3FileSystem.lambda$getS3ObjectMetadata$2(PrestoS3FileSystem.java:552)\r\n\tat com.facebook.presto.hive.RetryDriver.run(RetryDriver.java:138)\r\n\tat com.facebook.presto.hive.PrestoS3FileSystem.getS3ObjectMetadata(PrestoS3FileSystem.java:549)\r\n\tat com.facebook.presto.hive.PrestoS3FileSystem.getFileStatus(PrestoS3FileSystem.java:305)\r\n\tat org.apache.hadoop.fs.FileSystem.isDirectory(FileSystem.java:1439)\r\n\tat com.facebook.presto.hive.HiveMetadata.getExternalPath(HiveMetadata.java:719)\r\n\tat com.facebook.presto.hive.HiveMetadata.createTable(HiveMetadata.java:690)\r\n\tat com.facebook.presto.spi.connector.classloader.ClassLoaderSafeConnectorMetadata.createTable(ClassLoaderSafeConnectorMetadata.java:218)\r\n\tat com.facebook.presto.metadata.MetadataManager.createTable(MetadataManager.java:505)\r\n\tat com.facebook.presto.execution.CreateTableTask.execute(CreateTableTask.java:148)\r\n\tat com.facebook.presto.execution.CreateTableTask.execute(CreateTableTask.java:57)\r\n\tat com.facebook.presto.execution.DataDefinitionExecution.start(DataDefinitionExecution.java:111)\r\n\tat com.facebook.presto.execution.QueuedExecution.lambda$start$1(QueuedExecution.java:63)\r\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\r\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\nCaused by: java.net.ConnectException: Network is unreachable\r\n\tat java.net.PlainSocketImpl.socketConnect(Native Method)\r\n\tat java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)\r\n\tat java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)\r\n\tat java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)\r\n\tat java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)\r\n\tat java.net.Socket.connect(Socket.java:589)\r\n\tat sun.net.NetworkClient.doConnect(NetworkClient.java:175)\r\n\tat sun.net.www.http.HttpClient.openServer(HttpClient.java:432)\r\n\tat sun.net.www.http.HttpClient.openServer(HttpClient.java:527)\r\n\tat sun.net.www.http.HttpClient.<init>(HttpClient.java:211)\r\n\tat sun.net.www.http.HttpClient.New(HttpClient.java:308)\r\n\tat sun.net.www.http.HttpClient.New(HttpClient.java:326)\r\n\tat sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)\r\n\tat sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)\r\n\tat sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)\r\n\tat sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)\r\n\tat com.amazonaws.internal.ConnectionUtils.connectToEndpoint(ConnectionUtils.java:47)\r\n\tat com.amazonaws.internal.EC2CredentialsUtils.readResource(EC2CredentialsUtils.java:106)\r\n\tat com.amazonaws.internal.EC2CredentialsUtils.readResource(EC2CredentialsUtils.java:77)\r\n\tat com.amazonaws.auth.InstanceProfileCredentialsProvider$InstanceMetadataCredentialsEndpointProvider.getCredentialsEndpoint(InstanceProfileCredentialsProvider.java:117)\r\n\tat com.amazonaws.auth.EC2CredentialsFetcher.fetchCredentials(EC2CredentialsFetcher.java:121)\r\n\t... 24 more\r\n========= TECHNICAL DETAILS END =========\r\n```\r\nShould I try the `s3cmd` command to download a sample file? I can download the files using the access keys on the laptop through `s3cmd` but haven't tried the same on VM."}}
{"actor":{"display_login":"sbmaggarwal"}}