{"action":"created","issue":{"url":"https://api.github.com/repos/Azure/usql/issues/25","repository_url":"https://api.github.com/repos/Azure/usql","labels_url":"https://api.github.com/repos/Azure/usql/issues/25/labels{/name}","comments_url":"https://api.github.com/repos/Azure/usql/issues/25/comments","events_url":"https://api.github.com/repos/Azure/usql/issues/25/events","html_url":"https://github.com/Azure/usql/issues/25","id":130857169,"number":25,"title":"Extracting data from multiple files ","user":{"login":"rakeshdavanum","id":14914139,"avatar_url":"https://avatars.githubusercontent.com/u/14914139?v=3","gravatar_id":"","url":"https://api.github.com/users/rakeshdavanum","html_url":"https://github.com/rakeshdavanum","followers_url":"https://api.github.com/users/rakeshdavanum/followers","following_url":"https://api.github.com/users/rakeshdavanum/following{/other_user}","gists_url":"https://api.github.com/users/rakeshdavanum/gists{/gist_id}","starred_url":"https://api.github.com/users/rakeshdavanum/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rakeshdavanum/subscriptions","organizations_url":"https://api.github.com/users/rakeshdavanum/orgs","repos_url":"https://api.github.com/users/rakeshdavanum/repos","events_url":"https://api.github.com/users/rakeshdavanum/events{/privacy}","received_events_url":"https://api.github.com/users/rakeshdavanum/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-02-02T23:44:16Z","updated_at":"2016-02-03T06:37:04Z","closed_at":null,"body":"Hi,\r\n\r\nI am trying to use the EXTRACT USQL statement when the location pointed by FROM has multiple files. It works when I have a single file and I am explicitly specify the filename in the path. But in my use case it is not guaranteed that the filename will be the same everytime or the folder can have multiple files.\r\n\r\nIn the below statement I have tried using FROM \"/dimensiondata/carrier/\", \tFROM \"/dimensiondata/carrier/*\" and \tFROM \"/dimensiondata/carrier\" but none of them worked. All 3 of them came back with error saying \"File not found:\".\r\n\r\n@carrierdata =\r\n\tEXTRACT Carrier\tstring,\r\n\t\t\tCarrierCode\tstring\r\n\tFROM \"/dimensiondata/carrier/\"\r\n\tUSING Extractors.Csv();\r\n\r\nHow would I go about reading data from multiple files using an EXTRACT statement?\r\nAppreciate your help on this.\r\n\r\nThanks & Regards,\r\nRakesh\r\n"},"comment":{"url":"https://api.github.com/repos/Azure/usql/issues/comments/179039856","html_url":"https://github.com/Azure/usql/issues/25#issuecomment-179039856","issue_url":"https://api.github.com/repos/Azure/usql/issues/25","id":179039856,"user":{"login":"michaelamadi","id":12233644,"avatar_url":"https://avatars.githubusercontent.com/u/12233644?v=3","gravatar_id":"","url":"https://api.github.com/users/michaelamadi","html_url":"https://github.com/michaelamadi","followers_url":"https://api.github.com/users/michaelamadi/followers","following_url":"https://api.github.com/users/michaelamadi/following{/other_user}","gists_url":"https://api.github.com/users/michaelamadi/gists{/gist_id}","starred_url":"https://api.github.com/users/michaelamadi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/michaelamadi/subscriptions","organizations_url":"https://api.github.com/users/michaelamadi/orgs","repos_url":"https://api.github.com/users/michaelamadi/repos","events_url":"https://api.github.com/users/michaelamadi/events{/privacy}","received_events_url":"https://api.github.com/users/michaelamadi/received_events","type":"User","site_admin":false},"created_at":"2016-02-03T06:37:04Z","updated_at":"2016-02-03T06:37:04Z","body":"Hi Rakesh,\r\n\r\nYou'll need to use a file set pattern as described here: https://msdn.microsoft.com/en-us/library/azure/mt621294.aspx\r\n\r\nIn your specific case you could use the following if you don't need access to the file name values in the row set returned:\r\n@carrierdata =\r\nEXTRACT Carrier string,\r\nCarrierCode string\r\nFROM \"/dimensiondata/carrier/{*}\"\r\nUSING Extractors.Csv();\r\n\r\nIf you do need access to the file name values, then you can map the wild card to a virtual column like this: \r\n@carrierdata =\r\nEXTRACT Carrier string,\r\nCarrierCode string,\r\nFileName string \r\nFROM \"/dimensiondata/carrier/{FileName:*}\"\r\nUSING Extractors.Csv();\r\n\r\nPlease note that the virtual column, in this case ‘FileName’, must also appear in the column definition list of the EXTRACT expression.\r\n\r\nRegards, \r\n\r\nMichael\r\n"}}