Take the 2-minute tour ×

I need a PHP script which takes a URL of a web page and then echoes how many times a word is mentioned.


This is a generic HTML page:

<h1> This is the title </h1>
<p> some description text here, <b>this</b> is a word. </p>

This will be the PHP script:

the script here

So the output will be a table like this:

WORDS       Mentions
This        2
is          2
the         1
title       1
some        1
description 1
text        1
a           1
word        1

This is something like the search bots do when they are surfing the web, so, any idea of how to begin, or even better, do you have a PHP script which already does this?

share|improve this question

4 Answers 4

up vote 20 down vote accepted

The one line below will do a case insensitive word count after stripping all HTML tags from your string.

Live Example

print_r(array_count_values(str_word_count(strip_tags(strtolower($str)), 1)));

To grab the source code of a page you can use cURL or file_get_contents()

$str = file_get_contents('http://www.example.com/');

From inside out:

  1. Use strtolower() to make everything lower case.
  2. Strip HTML tags using strip_tags()
  3. Create an array of words used using str_word_count(). The argument 1 returns an array containing all the words found inside the string.
  4. Use array_count_values() to capture words used more than once by counting the occurrence of each value in your array of words.
  5. Use print_r() to display the results.
share|improve this answer
Nice and simple, but doesn’t take care of HTML tags... –  Timwi Aug 15 '10 at 0:17
@Timwi - now it does –  Peter Ajtai Aug 15 '10 at 0:28
+1 I would add a strtolower() in there too. –  NullUserException Aug 15 '10 at 0:39
@NullU - Thanks, good idea. –  Peter Ajtai Aug 15 '10 at 0:46
@DomingoSL - Live example with your sample code - codepad.org/7YJGYBVt –  Peter Ajtai Aug 15 '10 at 1:21

The below script will read the contents of the remote url, remove the html tags, and count the occurrences of each unique word therein.

Caveat: In your expected output, "This" has a value of 2, but the below is case-sensitive, so both "this" and "This" are recorded as separate words. You coudl convert the whole input string to lower case before processing if the original case is not significant for your purposes.

Additionally, as only a basic strip_tags is run on the input, mal-formed tags will not be removed, so the assumption is that your source html is valid.

Edit: Charlie points out in the comments that things like the head section will still be counted. With the help of a function defined in the user notes of the strip_tags function, these are also now taken care of.


<h1> This is the title </h1>


// Fetch remote html
$contents = file_get_contents($htmlurl);

// Get rid of style, script etc
           '@<head>.*?</head>@siU',            // Lose the head section

$contents = preg_replace($search, '', $contents); 

$result = array_count_values(
                  strip_tags($contents), 1




    [This] => 1
    [is] => 2
    [the] => 1
    [title] => 1
    [some] => 1
    [description] => 1
    [text] => 1
    [here] => 1
    [this] => 1
    [a] => 1
    [word] => 1
share|improve this answer
This is a clean solution but style and script tag content still exist. Than all the head of the page should be removed. –  Charlie Aug 15 '10 at 0:31
If you use the regExpressions not valid html code could be analyzed ;) Punctuation is still a problem –  Charlie Aug 15 '10 at 0:44
Please don't parse HTML with regular expressions. –  Artefacto Aug 15 '10 at 0:57
btw, strip_tags() (which you use) already removes multi line HTML comments and CDATA - codepad.org/gpdden0T php.net/manual/en/function.strip-tags.php . –  Peter Ajtai Aug 15 '10 at 1:52

This is a complex job that you should not attempt on your own.

You have to extract text that is not part of tags/comments and is not a child for elements such as script and style. For this, you'll also need a lax HTML parser (like the one implemented in libxml2 and used in DOMDocument.

Then you have to tokenize the text, which presents its own challenges. Finally, you'd interested in some form of stemming before proceeding to counting the terms.

I recommend you use specialized tools for this. I haven't used any of these, but you can try HTMLParser for parsing and Lucene for tokenization/stemming (the purpose of Lucene is Text Retrieval, but those operations are necessary for building the index).

share|improve this answer
A complex job? The ConroyP code works well and does a big part of what you listed. HTML has a very regular syntax –  Charlie Aug 15 '10 at 0:48
@Charlie There's so many things that are missing... Dealing with encodings that are not ASCII, proper handling of HTML (I could easily build an HTML document with a bible transcription that would yield him no words whatsoever for his code), a proper tokenizer (str_word_count is very basic and only handles ASCII), a stemmer, ... –  Artefacto Aug 15 '10 at 0:53
A stemmer? Fist why add a stemmer that will not be able to find the roots of every languages? (what is the purpose? The original question asked for a simple HTML parser, not a language analyzer) –  Charlie Aug 15 '10 at 1:12
You can find stemmers for several languages. The OP dind't say he want stemming, but it's legitimate to assume he wants, esp since there's already some form of term normalization in his question ("This" and "this" are counted as the same). And I suppose you concede the other points... –  Artefacto Aug 15 '10 at 1:37
Yes, my doubts are still on the stemmer. You the Italian on the list you signaled doesn't correctly match to 30% of Italian words, and the vocabulary it contains is just the 1% of Italian words (I'm not kidding). Martin Porter has written an algorithm good for English (perhaps) but not good for other more complex languages. –  Charlie Aug 15 '10 at 10:24

The previous code is a point where start. The next step is delete html tags with the regular expressions. Look for ereg and eregi functions. Some other tricks are required for style and script tags (you have to remove the content) Points and commas have to be removed too...

share|improve this answer
ereg's been deprecated and, to begin with, regexes are not an adequate tool for parsing arbitrary HTML. –  Artefacto Aug 15 '10 at 0:58
How can regular expression be deprecated if they exist from perl O.O? –  Charlie Aug 15 '10 at 1:04
Answers are not always listed in chronological order on SO, so previous code isn't very helpful. A url link (each answer has a unique one) or author reference is better. –  Peter Ajtai Aug 15 '10 at 1:19
Regular expressions haven't been deprecated, only the ereg extension. Use PCRE instead (the preg_ function family). –  Artefacto Aug 15 '10 at 2:16
Ah ok :) I misunderstood –  Charlie Aug 15 '10 at 10:07

Your Answer


