{"action":"created","issue":{"url":"https://api.github.com/repos/jhy/jsoup/issues/1089","repository_url":"https://api.github.com/repos/jhy/jsoup","labels_url":"https://api.github.com/repos/jhy/jsoup/issues/1089/labels{/name}","comments_url":"https://api.github.com/repos/jhy/jsoup/issues/1089/comments","events_url":"https://api.github.com/repos/jhy/jsoup/issues/1089/events","html_url":"https://github.com/jhy/jsoup/issues/1089","id":339121538,"node_id":"MDU6SXNzdWUzMzkxMjE1Mzg=","number":1089,"title":"Element.attr(\"href\") returning \"#\"","user":{"login":"ladiesman6969","id":25760501,"node_id":"MDQ6VXNlcjI1NzYwNTAx","avatar_url":"https://avatars1.githubusercontent.com/u/25760501?v=4","gravatar_id":"","url":"https://api.github.com/users/ladiesman6969","html_url":"https://github.com/ladiesman6969","followers_url":"https://api.github.com/users/ladiesman6969/followers","following_url":"https://api.github.com/users/ladiesman6969/following{/other_user}","gists_url":"https://api.github.com/users/ladiesman6969/gists{/gist_id}","starred_url":"https://api.github.com/users/ladiesman6969/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ladiesman6969/subscriptions","organizations_url":"https://api.github.com/users/ladiesman6969/orgs","repos_url":"https://api.github.com/users/ladiesman6969/repos","events_url":"https://api.github.com/users/ladiesman6969/events{/privacy}","received_events_url":"https://api.github.com/users/ladiesman6969/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-07-07T06:24:49Z","updated_at":"2018-09-30T14:01:51Z","closed_at":null,"author_association":"NONE","body":"I am making a web-scraping project that basically fetches me images from Google Images. In order to get the image src, i use\r\n\r\nElement.attr(\"href\")\r\n\r\nHowever, it returns\r\n\r\n\"#\"\r\n\r\nMy Code\r\n\r\nDocument shivWall = Jsoup.connect(searchURL).get(); Elements smallImgElements = shivWall.getElementsByClass(\"rg_bx rg_di rg_el ivg-i\"); smallImgElements.get(0).select(\"a.rg_l\").get(0).attr(\"href\");\r\n\r\nI have tried numerous ways, but none of them worked. I even checked again by changing the attr argument to some random value, and it returns null as expected. But, for \"href\", it just returns a \"#\". Please help."},"comment":{"url":"https://api.github.com/repos/jhy/jsoup/issues/comments/425722976","html_url":"https://github.com/jhy/jsoup/issues/1089#issuecomment-425722976","issue_url":"https://api.github.com/repos/jhy/jsoup/issues/1089","id":425722976,"node_id":"MDEyOklzc3VlQ29tbWVudDQyNTcyMjk3Ng==","user":{"login":"razafinarivohanania","id":24853543,"node_id":"MDQ6VXNlcjI0ODUzNTQz","avatar_url":"https://avatars0.githubusercontent.com/u/24853543?v=4","gravatar_id":"","url":"https://api.github.com/users/razafinarivohanania","html_url":"https://github.com/razafinarivohanania","followers_url":"https://api.github.com/users/razafinarivohanania/followers","following_url":"https://api.github.com/users/razafinarivohanania/following{/other_user}","gists_url":"https://api.github.com/users/razafinarivohanania/gists{/gist_id}","starred_url":"https://api.github.com/users/razafinarivohanania/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/razafinarivohanania/subscriptions","organizations_url":"https://api.github.com/users/razafinarivohanania/orgs","repos_url":"https://api.github.com/users/razafinarivohanania/repos","events_url":"https://api.github.com/users/razafinarivohanania/events{/privacy}","received_events_url":"https://api.github.com/users/razafinarivohanania/received_events","type":"User","site_admin":false},"created_at":"2018-09-30T14:01:50Z","updated_at":"2018-09-30T14:01:50Z","author_association":"NONE","body":"The issue is due to **javascript rendering**. Try to open this link for example on your browser.\r\n\r\nhttps://www.google.com/search?q=poisson&hl=fr&biw=1280&bih=686&tbm=isch&gbv=2&sei=8NKwW4v2NMeIgAbS66wQ\r\n\r\nSave the html code source and clean it from https://www.freeformatter.com/ to indent tags. Now, lets see an \"rg_bx rg_di rg_el ivg-i\" and then find \"rg_l\".\r\n\r\n```html\r\n<div jscontroller=\"Q7Rsec\" data-ri=\"63\" class=\"rg_bx rg_di rg_el ivg-i\" data-ved=\"0ahUKEwiJ-L_L6-LdAhUrCsAKHRwoDnQQMwh2KD8wPw\">\r\n   <a jsname=\"hSRGPd\" href=\"#\" jsaction=\"fire.ivg_o;mouseover:str.hmov;mouseout:str.hmou\" class=\"rg_l\" rel=\"noopener\" style=\"background:rgb(248,133,50)\">\r\n      <div class=\"THL2l\"></div>\r\n      <img class=\"rg_ic rg_i\" data-src=\"https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcREkh5gjNIZrrPwvU-bBzJC-TioLCGhZnj6ayuzjnA9n7nXpAxE\" jsaction=\"load:str.tbn\" alt='Résultat de recherche d&#39;images pour \"poisson\"' onload=\"typeof google==='object'&&google.aft&&google.aft(this)\">\r\n      <div class=\"rg_ilmbg\"> 568&nbsp;&#215;&nbsp;568 </div>\r\n   </a>\r\n   <a class=\"iKjWAf a-no-hover-decoration irc-nic\" href=\"#\" jsaction=\"mouseenter:QEDpme;focus:QEDpme;\" rel=\"noopener\" target=\"_blank\">\r\n      <div class=\"mVDMnf nJGrxf\">Chaussons Poisson-Globe - Cute Land</div>\r\n      <div class=\"nJGrxf FnqxG\">cute.land</div>\r\n   </a>\r\n   <div class=\"rg_meta notranslate\">{\"cb\":9,\"cl\":3,\"cr\":6,\"ct\":6,\"id\":\"BQRe0XN-KbcAHM:\",\"isu\":\"cute.land\",\"itg\":0,\"ity\":\"jpg\",\"oh\":568,\"ou\":\"https://cute.land/1357-big_default/chaussons-poisson-globe.jpg\",\"ow\":568,\"pt\":\"Chaussons Poisson-Globe - Cute Land\",\"rh\":\"cute.land\",\"rid\":\"khat6BOchaioyM\",\"rt\":0,\"ru\":\"https://cute.land/chaussons/313-chaussons-poisson-globe\",\"s\":\"Chaussons Poisson-Globe\",\"st\":\"Cute Land\",\"th\":224,\"tu\":\"https://encrypted-tbn0.gstatic.com/images?q\\u003dtbn:ANd9GcREkh5gjNIZrrPwvU-bBzJC-TioLCGhZnj6ayuzjnA9n7nXpAxE\",\"tw\":224}</div>\r\n</div>\r\n```\r\n\r\nWhat we get? an href with \"#\" as value\r\n\r\n```html\r\n<a jsname=\"hSRGPd\" href=\"#\" jsaction=\"fire.ivg_o;mouseover:str.hmov;mouseout:str.hmou\" class=\"rg_l\" rel=\"noopener\" style=\"background:rgb(248,133,50)\">\r\n```\r\n\r\nSo where is the image? This is found on img.rg_ic by using \"data-src\" as attribute.\r\n\r\n```html\r\n<img class=\"rg_ic rg_i\" data-src=\"https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcREkh5gjNIZrrPwvU-bBzJC-TioLCGhZnj6ayuzjnA9n7nXpAxE\" jsaction=\"load:str.tbn\" alt='Résultat de recherche d&#39;images pour \"poisson\"' onload=\"typeof google==='object'&&google.aft&&google.aft(this)\">\r\n```\r\n\r\nKeep in mind that Jsoup doesn't interpret javascript. :)\r\n\r\n\r\n\r\n"}}
{"actor":{"display_login":"razafinarivohanania"}}