title: Scraping data from a website with a search box
body: <p>First of all I want to apologize if my question is too broad or generic, but it would really save me a lot of needlessly wasted time to get an answer to guide me in the right direction for the work I want to do. With that out of the way, here it goes.</p>
  <p>I am trying to retrieve some publicly available data from a website, to create a dataset to work with for a Data Science project. My big issue is that the website does not have a friendly way to download it, and, from what I gathered, it also has no API. So, getting the data requires skills that I do not possess. I would love to learn how to scrape the website (the languages I am most comfortable with are Python and R), and it would add some value to my project if I did it, but I also am somewhat pressured by time constraints, and don't know if it is even possible to scrape the website, much less to learn how to do it in a few days.</p>
  <p>The website is this one <a href="https://www.rnec.pt/pt_PT/pesquisa-de-estudos-clinicos" rel="nofollow noreferrer">https://www.rnec.pt/pt_PT/pesquisa-de-estudos-clinicos</a>. It has a search box, and the only option I configure is to click the banner that says &quot;Pesquisa Avançada&quot; and then mark the box that says &quot;Menor de 18 anos&quot;. I then click the &quot;Pesquisar&quot; button in the lower-right, and the results that show up are the ones that I want to extract (either that or, if it's simpler, all the results, without checking the &quot;Menor de 18 anos&quot; box). In my computer, 2 results show up per page, and there are 38 pages total. Each result has some of it details in the page where the results appear but, to get the full data from each entry, one has to click &quot;Detalhes&quot; in the lower right of each result, which opens a display with all the data from that result. If possible, I would love to download all the data from that &quot;Detalhes&quot; page of each result (the data there alerady contains the fields that show up in the search result page).</p>
  <p>Honestly, I am ready to waste a whole day manually transcribing all the data, but it would be much better to do it computationally, even it it takes me two or three days to learn and do it.</p>
  <p>I think that, for someone with experience in web scraping, it is probably super simple to check if it is possible to download the data I described, and what is the best way to go about it (in general terms, I would research and learn it). But I really am lost when it comes to this, and just kindly want to ask for some help in showing me the way go about it (even if the answer is &quot;it is too complicated/impossible, just do it manually&quot;). I know that there are some Python packages for web scraping, like BeautifulSoup and Selenium, but I don't really know if either of them would be appropriate.</p>
  <p>I am sorry if my request is not exactly a short and simple coding question, but I have to try to gather any help or guidance I can get. Thank you in advance to everyone who reads my question and a special thank you if you are able to give me some pointers.</p>