{"action":"created","issue":{"url":"https://api.github.com/repos/LMFDB/lmfdb/issues/1247","repository_url":"https://api.github.com/repos/LMFDB/lmfdb","labels_url":"https://api.github.com/repos/LMFDB/lmfdb/issues/1247/labels{/name}","comments_url":"https://api.github.com/repos/LMFDB/lmfdb/issues/1247/comments","events_url":"https://api.github.com/repos/LMFDB/lmfdb/issues/1247/events","html_url":"https://github.com/LMFDB/lmfdb/issues/1247","id":153692318,"number":1247,"title":"LMFDB performance monitoring and optimization","user":{"login":"AndrewVSutherland","id":11425002,"avatar_url":"https://avatars.githubusercontent.com/u/11425002?v=3","gravatar_id":"","url":"https://api.github.com/users/AndrewVSutherland","html_url":"https://github.com/AndrewVSutherland","followers_url":"https://api.github.com/users/AndrewVSutherland/followers","following_url":"https://api.github.com/users/AndrewVSutherland/following{/other_user}","gists_url":"https://api.github.com/users/AndrewVSutherland/gists{/gist_id}","starred_url":"https://api.github.com/users/AndrewVSutherland/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AndrewVSutherland/subscriptions","organizations_url":"https://api.github.com/users/AndrewVSutherland/orgs","repos_url":"https://api.github.com/users/AndrewVSutherland/repos","events_url":"https://api.github.com/users/AndrewVSutherland/events{/privacy}","received_events_url":"https://api.github.com/users/AndrewVSutherland/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/LMFDB/lmfdb/labels/backend","name":"backend","color":"000000"},{"url":"https://api.github.com/repos/LMFDB/lmfdb/labels/ecnf","name":"ecnf","color":"bfdadc"},{"url":"https://api.github.com/repos/LMFDB/lmfdb/labels/SiegelMF","name":"SiegelMF","color":"fad8c7"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":92,"created_at":"2016-05-09T02:20:35Z","updated_at":"2016-05-26T01:43:01Z","closed_at":null,"body":"***1 Upvote*** This is a long discussion that probably should not be in the description of an \"issue\", but I want to put it where everyone can see it -- we can move it elsewhere later.  If you are not interested in the performance of the LMFDB you don't need to read it, but I would argue that anyone who cares about the success of the LMFDB should be interested in performance, as it can have a huge impact on user perception.  It contains a markup table that is best viewed on github at https://github.com/LMFDB/lmfdb/issues/1247.\r\n\r\nAs we prepare to go live with Release 1.0 we need to think about how to monitor performance and improve it where needed.  I know there are server side tools to help with this, but we need to configure and figure out how to best use them.  I also think it is important to monitor the user experience from the client side, because this may turn up issues that are not always evident on the server side.\r\n\r\nThis morning I ran a modified version a curl script created by Edgar Costa, using it to measure load times of all the top level pages of the LMFDB (anything you can get to with one click on the sidebar).  I ran it from MIT and measured the time to get the first byte of the response (TTFB).  These times are thus essentially lower bounds on response times seen by the user, since they are running over a very fast internet connection close to the backbone, and they do not include any time spent by the broswer rendering the returned page.  I measured times both to the cloud server (www.lmfdb.org) and Warwick (lmfdb.warwick.ac.uk) over 100 consecutive requests (arguably it would be better to space them out over time, but I did not take the time to do this, I was in a hurry to get the results).\r\n\r\n(discussion continued below the table)\r\n\r\n                                               www.lmfdb.org                   lmfdb.warwick.ac.uk\r\n|page|min|max|ave|dev|min|max|ave|dev|\r\n|---|---|---|---|---|---|---|---|---|---|\r\n|L/degree/1|0.066|0.188|0.072|0.013|0.188|0.826|0.230|0.139|\r\n|L/degree/2|0.066|0.074|0.069|0.002|0.188|0.206|0.192|0.003|\r\n|L/degree/3|0.066|0.086|0.069|0.003|0.188|0.199|0.191|0.003|\r\n|L/degree/4|0.067|0.131|0.070|0.007|0.188|0.199|0.192|0.002|\r\n|zeros/zeta/|0.091|0.147|0.098|0.007|0.190|0.216|0.197|0.007|\r\n|ModularForm/GL2/Q/holomorphic/|0.390|0.885|0.530|0.137|0.749|0.892|0.782|0.041|\r\n|ModularForm/GL2/Q/Maass/|0.091|0.592|0.201|0.122|0.205|0.236|0.213|0.007|\r\n|ModularForm/GL2/TotallyReal/|0.106|0.495|0.173|0.103|0.291|0.345|0.303|0.015|\r\n|L/degree3/MaassForm/|0.088|0.671|0.221|0.158|0.216|0.426|0.225|0.021|\r\n|ModularForm/GSp/Q/|0.670|1.992|1.161|0.396|0.796|2.182|1.038|0.325|\r\n|EllipticCurve/Q/ (old)|0.232|**3.173**|**1.034**|0.974|0.344|3.740|0.882|1.120|\r\n|EllipticCurve/Q/ (new)|0.114|**0.529**|**0.282**|0.164|0.294|0.811|0.397|0.193|\r\n|EllipticCurve/|0.114|1.108|0.480|0.372|0.299|1.284|0.463|0.326|\r\n|Genus2Curve/Q/|0.114|0.566|0.139|0.059|0.289|0.493|0.305|0.026|\r\n|NumberField/|0.114|0.469|0.150|0.073|0.294|0.938|0.312|0.064|\r\n|LocalNumberField/|0.106|0.172|0.118|0.010|0.291|0.924|0.318|0.106|\r\n|Character/|0.108|0.197|0.118|0.010|0.292|0.919|0.313|0.087|\r\n|Character/Dirichlet/|0.109|0.147|0.115|0.005|0.293|1.152|0.326|0.137|\r\n|ArtinRepresentation/|0.110|0.169|0.117|0.007|0.290|0.920|0.305|0.062|\r\n|GaloisGroup/|0.102|0.133|0.108|0.005|0.286|0.923|0.300|0.063|\r\n|SatoTateGroup/|0.116|0.140|0.126|0.008|0.295|0.334|0.304|0.011|\r\n|Lattice/|0.114|0.516|0.221|0.134|0.294|0.739|0.327|0.064|\r\n\r\nIdeally we would like the mean response time for all our pages below 400ms -- this is the generally accepted threshhold where users first begin to change there website usage based on perceived performance (Google and Akamai have done a lot of research on this), and anything over 2 seconds represents a place where an unmotivated user may well give up (something like 20-40 percent do).  In my initial test the time for EllipticCurve/Q jumped out as being remarkably slower than the others (which agreed with my own personal experience).  After some digging I was eventually able to track down the root cause (a missing index on an attribute that was being used to count isogeny classes of elliptic curves in ec_stats.py).  Adding this index dramatically improved performance.  I suspect there are several other \"easy wins\" to be had, and my purpose in raising this issue is to get people to think about them.\r\n\r\nWe should of course also be thinking about the performance of pages below the top level, and the web server analytics should give us coverage data that will tell us which pages we should focus on (this is the other part of this issue, configuring and using analytics).\r\n\r\nAnother thing I noticed during these experiments is that in cases where computations are cached (e.g. statistics computed in elliptic_curves, genus2_curves, and elsewhere), the response times have a bimodal distribution -- the first access by a given worker thread is slow, but later access that can reuse the cached data are much faster (as much as a 5 to 1 or 10 to 1 difference).  You might think this means that once the software is up and runnign for a while every worker thread will be using cached data and respond quickly, but this is clearly not the case.  If you ran many copies of the page fetch script in parallel for a high number of iterations, it will quickly reach a steady state where all the responses are quick.  But if you then stop and do a handful of requests 10-15 minutes later you have a good chance of seeing slow response again.  I don't know why this is happening (it may have to do with worker threads being dynamically created and killed to adjust to dynamic load), but it would be good to figure out.\r\n\r\nI realize that I have touched on a number of issues that should probably be raised individually, but I wanted to do a brain dump and put it out there so that others can think about these issues and respond.\r\n\r\nHere [http://math.mit.edu/~drew/lmfdb_timings.sh](url) is the curl script I used, in case anyone else wants to run it for their location (it separates out connect time, so you can see how much of the difference is due to the time to reach the server, versus the time it spends responding)."},"comment":{"url":"https://api.github.com/repos/LMFDB/lmfdb/issues/comments/221756249","html_url":"https://github.com/LMFDB/lmfdb/issues/1247#issuecomment-221756249","issue_url":"https://api.github.com/repos/LMFDB/lmfdb/issues/1247","id":221756249,"user":{"login":"AndrewVSutherland","id":11425002,"avatar_url":"https://avatars.githubusercontent.com/u/11425002?v=3","gravatar_id":"","url":"https://api.github.com/users/AndrewVSutherland","html_url":"https://github.com/AndrewVSutherland","followers_url":"https://api.github.com/users/AndrewVSutherland/followers","following_url":"https://api.github.com/users/AndrewVSutherland/following{/other_user}","gists_url":"https://api.github.com/users/AndrewVSutherland/gists{/gist_id}","starred_url":"https://api.github.com/users/AndrewVSutherland/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AndrewVSutherland/subscriptions","organizations_url":"https://api.github.com/users/AndrewVSutherland/orgs","repos_url":"https://api.github.com/users/AndrewVSutherland/repos","events_url":"https://api.github.com/users/AndrewVSutherland/events{/privacy}","received_events_url":"https://api.github.com/users/AndrewVSutherland/received_events","type":"User","site_admin":false},"created_at":"2016-05-26T01:43:01Z","updated_at":"2016-05-26T01:43:01Z","body":"@edgarcosta Agreed, not a priority (the savings will be measure in milliseconds, not seconds, which is what we are after at the moment)."}}
{"actor":{"display_login":"AndrewVSutherland"}}