text: Q:
  
  extract webdata into SAS
  
  I am trying to extract data from a web page. It is displayed on the browser, but not in the source code. So When I tried to use FILENAME URL, the numbers I wanted are not in the fileref. I also tried to open a browser by DOS command and pipe it into a fileref, but it didn't work either. Does anybody have any suggestion? 
  The website I am going is 
  http://www.cargurus.com/Cars/instantMarketValueFromVIN.action?startUrl=%2F&carDescription.vin=4T1BF32K33U040430
  The data I wanted to extract are the buying and trading prices on the right hand side.
  Thanks in advance. 
  
  A:
  
  I checked the url that you had provided. The reason why you are not getting any result when you use the url is because the page uses jQuery and Ajax to fill up the webpage when the result is retrieved. So result related section/tags gets created on the fly. When SAS tries to read the url it's reads the base HTML page and not what jQuery fills it up with. To solve the issue, you can use Firebug addon on Firefox or Chrome Developer tool on Chrome browser and carefuly analyze the network section to find the data url that is used by jQuery to fill up the result section of the page. For your url, i used FireBug and was able to retrieve the url which should be used to extract the information you want.
  The url that you need to use should be : http://www.cargurus.com/Cars/priceCalculatorReportAjaxResearchPriceView.action?carDescription.autoEntityId=c3576&carDescription.postalCode=&carDescription.mileage=&carDescription.price=&carDescription.radius=75&carDescription.transmissionType=AUTOMATIC_TRANSMISSION&selectedEntity=c3576&carDescription.vin=4T1BF32K33U040430
  Below is a small reference code which might help you further with the scraping part :
  filename myurl url 'http://www.cargurus.com/Cars/priceCalculatorReportAjaxResearchPriceView.action?carDescription.autoEntityId=c3576&carDescription.postalCode=&carDescription.mileage=&carDescription.price=&carDescription.radius=75&carDescription.transmissionType=AUTOMATIC_TRANSMISSION&selectedEntity=c3576&carDescription.vin=4T1BF32K33U040430';
  data scrap_data;
  length text $500;
  retain text ;
  file 'c:\url.txt' lrecl=32767;
  infile myurl truncover;
      input;
      if find(_infile_,'<span','it') and find(_infile_,'</span','it') then do;
          text=strip(_infile_);
          put @1 text;
          call missing(text);
      end;
      else if find(_infile_,'<span','it') then do;
          do until(find(_infile_,'</span','it') ne 0);
              text=catx(' ',text,_infile_);
              input;
          end;
          put @1 text;
          call missing(text);
      end;
  run; 
  
  Hope this helps!
  
  