Is there a Mathematica API for the functions.wolfram site?
If there's not, has anyone implemented a web scraper for it?

For example it would be nice to be able to access http://functions.wolfram.com/ from within Mathematica by using something like

In[1]:= FunctionsWolfram["", InputForm]
Out[1]= Hypergeometric2F1[a,b,a+b-1/2,z]
        == Hypergeometric2F1[2 a-1,2 b-1,a+b-1/2,1/2 (1-Sqrt[1-z])]/Sqrt[1-z]

In[2]:= FunctionsWolfram["", RuleForm]
Out[2]= HoldPattern[Hypergeometric2F1[a_,b_,a_+b_-1/2,z_]] 

In[3]:= FunctionsWolfram["", TraditionalForm]



Here is a shameless plug for my HTML parser posted here. The code is a bit long to reproduce here, the only change to it I'd do is to replace the function processPosList with this code:

processPosList::unmatched = "Unmatched lists `1` enountered!";
processPosList[{openlist_List, closelist_List}] := 
  Module[{opengroup, closegroup, poslist}, 
  {opengroup, closegroup} = groupPositions /@ {openlist, closelist};
   poslist = Transpose[Transpose[Sort[#]] & /@ {opengroup, closegroup}];
   If[UnsameQ @@ poslist[[1]], Return[(Message[
       processPosList::unmatched , {openlist, closelist}]; {})], 
   poslist = Transpose[{poslist[[1, 1]], Transpose /@ Transpose[poslist[[2]]]}]]];

which will issue a message when some parts can not be parsed instead of printing the details (as the original code does). I must warn that my parser for some reason can not fully parse the Wolfram Functions pages (either they are ill-formed or my parser contains bugs), but it will parse enough for our purposes. Here is a simple web-scraper based on it and on a few observations about the typical format of the page:

getForms[url_String] := 
 Quiet@ Cases[postProcess@parseText[Import[url, "Text"]],
     pContainer[attribContainer[" class='CitationInfo'"], x__String] :> 
        StringJoin@x, Infinity] //. 
       x_String :>  StringReplace[ x, {"&quot;" | "quot;" :> "\"", "&amp;" :> "", 
             "&lt;" | "&lt" :> "<", "&gt;" | "&gt" :> ">", "\n" :> " "}];

Clear[formsOk, getInputForm, getStandardForm, getRuleForm];
formsOk[forms_] := Length[forms] == 5;
getInputForm[forms_?formsOk] := ToExpression[forms[[1]], InputForm];
getStandardForm[forms_?formsOk] := ToExpression[First@ToExpression[forms[[2]]], StandardForm];
getRuleForm[forms_?formsOk] := ToExpression[First@ToExpression[forms[[4]]]];
getInputForm[__] = getStandardForm[__] = getRuleForm[__] = $Failed;

I can not say how fragile this is, probably rather fragile. Here is an example of use:

forms = getForms["http://functions.wolfram.com/"];

Out[278]= {Hypergeometric2F1[a,b,-(1/2)+a+b,z]==((Sqrt[1-z]-Sqrt[-z])^(1-2 a) 
  Hypergeometric2F1[-1+2 a,-1+a+b,-2+2 a+2 b,2 z+2 Sqrt[-z+z^2]])/Sqrt[1-z]/;Re[z]>1/2,
  HoldPattern[Hypergeometric2F1[a_,b_,a_+b_-1/2,z_]]:>((Sqrt[1-z]-Sqrt[-z])^(1-2 a) 
  Hypergeometric2F1[2 a-1,a+b-1,2 a+2 b-2,2 Sqrt[z^2-z]+2 z])/Sqrt[1-z]/;Re[z]/2}

I tested on about 10 different formulas, and this worked fine, but of course this is not an extensive test, so most likely this will not always work.

| improve this answer | |
  • $\begingroup$ Thanks Leonid! I'll have a closer look at this tomorrow. I wouldn't be too concerned about its fragility - it's probably a lot more robust than my solution... $\endgroup$ – Simon May 31 '11 at 11:57
  • $\begingroup$ @Simon For your solution, the heavy lifting is done with Import, which does a pretty good job of importing in default format (HTML here). My solution parses HTML and looks at certain pattern in the parsed document. For the case at hand, your solution seems much more robust and adequate, so I am tempted to delete mine - perhaps will give it a day or two... $\endgroup$ – Leonid Shifrin May 31 '11 at 12:08

The site is not terribly conducive to scraping as the HTML is "noisy" and looks like WRI might change the format at the drop of a hat. Throwing caution to the wind...

scrapeWolframFunction[id_] :=
  Import["http://functions.wolfram.com/" ~~ id, "XMLObject"] //
  , XMLElement["p", {___, "class" -> "CitationInfo", ___}, body_] :> body
  , Infinity
  ] & //
  ToExpression[#[[1, 1]], InputForm, HoldForm] &

The function assumes that the first CitationInfo paragraph contains the InputForm. This assumption appears to hold true for the moment.

Sample use:

In[24]:= scrapeWolframFunction[""]
Out[24]= Hypergeometric2F1[a,b,a+b-1/2,z]==Hypergeometric2F1[2 a-1,2 b-1,a+b-1/2,1/2 (1-Sqrt[1-z])]/Sqrt[1-z]

In[25]:= scrapeWolframFunction[""]
Out[25]= Sin[z]==(E^(I z)-E^(-I z))/(2 I)

In[26]:= scrapeWolframFunction[""]
Out[26]= Erf[z]==(1+I) (FresnelC[((1-I) z)/Sqrt[\[Pi]]]-I FresnelS[((1-I) z)/Sqrt[\[Pi]]])
| improve this answer | |
  • 4
    $\begingroup$ Mathematica's built-in HTML parser is TagSoup which is very good at handling HTML as it is found in the wild. It is worth the effort to persevere through XMLObject if one has any need to parse either HTML or XML. $\endgroup$ – WReach Jun 2 '11 at 3:42

Here's a quick scraper that I built (after asking the question). So far it has minimal error checking. Also note that InputForm, StandardForm and MathMLForm should all yield the same expressions.



FunctionsWolfram[id_String?FWIDQ,All] := FunctionsWolfram[id,All] = 
  Module[{imp=Import["http://functions.wolfram.com/"<>id]}, StringSplit[imp,
    "Input Form"|"Standard Form"|"MathML Form"|"Rule Form"|"Date Added"]]

FunctionsWolfram[id_String?FWIDQ,InputForm] := 

FunctionsWolfram[id_String?FWIDQ,StandardForm] := 

FunctionsWolfram[id_String?FWIDQ,MathMLForm] :=

FunctionsWolfram[id_String?FWIDQ,RuleForm] := 

FunctionsWolfram[id_String?FWIDQ, TraditionalForm] := 
  TraditionalForm[FunctionsWolfram[id, InputForm]]

It works as advertised in the question.

| improve this answer | |
  • $\begingroup$ This was just a really quick solution that just reads a functions site in as a long string and then splits it up and spits out the appropriate parts. I left the answer "deleted" until I was ready to call it a night and not check SO until morning! $\endgroup$ – Simon May 31 '11 at 11:56
  • $\begingroup$ Nice. I was already doing messy pattern matching in the html code read in using Import[...,"Text"]: StringReplace[text, ___ ~~ "<font class='CitationLabel' size='-1'>Input Form</font>" ~~ Shortest[_] ~~ "<p class='CitationInfo'>" ~~ Shortest[inputForm ~~ "</p>"] ~~ ___ :> inputForm]. This is much cleaner $\endgroup$ – Sjoerd C. de Vries May 31 '11 at 12:25
  • 1
    $\begingroup$ @Sjoerd Re your comment. Have you read this excellent post: stackoverflow.com/questions/1732348/…? $\endgroup$ – Dr. belisarius Jun 3 '11 at 3:47
  • 1
    $\begingroup$ It seems to me this great function would gain a lot if it could take WildCards, such as FunctionWolfram["HyperGeometric2F1"] so that it would returns all identities related to HyperGeometric2F1 funtions. Once the index of such entries identified one could imagine using something like Table[FunctionsWolfram[ "07.23.17." <> IntegerString[i, 10, 4] <> ".01"], {i, 4}]? The purpose of this would be to produce transformation rules that the user could map onto given mathematica results to suit his/her need. $\endgroup$ – chris Apr 28 '12 at 18:54
  • $\begingroup$ @chris: That does sound great! The functions site used to have and advanced search form that used standard Mathematica patterns... but I can't find it on the site anymore. Feel free to develop something more useful than what has been posted here and I'll accept your answer (as I couldn't choose between the three answers already posted). $\endgroup$ – Simon Apr 29 '12 at 4:12

The new MathematicalFunctionData[] function seems to know a lot of the identities that are in the Wolfram Functions site. Unfortunately, I haven't quite figured out how to have it recognize a permalink on the Wolfram Functions site, so here's an alternative way to look up the formula featured in the OP:

idList = MathematicalFunctionData[Hypergeometric2F1, "FunctionalEquations", 
                                  "IncludedSubexpressions" -> {HoldPattern[Sqrt[_]]}]

and looking at the list thus returned reveals that the first one is the desired identity:

id = First[idList]
   Function[{a, b, z},
            Inactivate[Hypergeometric2F1[a, b, a + b - 1/2, z] ==
            (1/Sqrt[1 - z]) Hypergeometric2F1[2 a - 1, 2 b - 1, a + b - 1/2,
                                              (1 - Sqrt[1 - z])/2]]]

(N.B. I had changed the formal symbols that are actually in the output to normal letters for clarity.)

One can now do something like

id[a, b, z] // Activate // TraditionalForm

to show the desired identity in the traditional notation, or

id[-1/4, 1/4, z] // Activate
   Hypergeometric2F1[-1/4, 1/4, -1/2, z] ==
   (1 + (-1 + Sqrt[1 - z])/2)^(3/2)/Sqrt[1 - z]

to display a special case.

The documentation has a description of this new curated data function's abilities. I have found it a bit slow, unfortunately, due to the extensive use of Entity[], but I guess this is the the way curated data functions are implemented these days.

| improve this answer | |

Your Answer

