Take the 2-minute tour ×

Possible Duplicate:
How do I render javascript from another site, inside a PHP application?

This is the site http://www.oferta.pl/strona_v2/gazeta_v2/ . This site is built totally on JavaScript. I want to scrape using PHP and curl. Currently I use DOMXPath. In the left menu there are some category to be selected. I see no 'form' there. How can I use curl to submit that form and scrap the output page?

I have used file_get_contents() only. It doesn't get all of the page. How can I proceed?

N.B : http://www.html-form-guide.com/php-form/php-form-submit.html I have found this example which have a 'form'. But my specified site has no 'form'.

share|improve this question

marked as duplicate by Jared Farrish, Bill the Lizard Feb 18 '12 at 16:29


Scraps are found in a junkyard. You scrape your knee on the asphalt when you fall. The latter of these is a very despicable process when applied to computing. I would discourage your use of it. Besides, you don't even have a sample of the code you're using. Why not start by adding that? –  PhpMyCoder Feb 18 '12 at 16:13
offtopic: @PhpMyCoder stop using google then, as they are the biggest web-scraper in the world ;) –  c69 Feb 18 '12 at 16:18
add comment

1 Answer

up vote 0 down vote accepted

You can not scrape it. Its possible. But its way too hard.

  1. Simulate the http request by curl. Check every request it makes by ajax and try to simulate it.

  2. Simulate Javascript executions (this part is almost impossible). Some requests contains values which are generated by Javascript. You need to do it in php. If they has some complicated algorithm implemented in JS you can invoke v8 javascript engine.

share|improve this answer
Thanks for the info. –  qmaruf Feb 18 '12 at 18:26
add comment
