title: Scalable way of logging page request data from a PHP application?
body: <p>A web application I am developing (in PHP) requires the ability to log each page request.</p>
  
  <p>Just like a normal access_log, it will store details like url requested, source ip address, date/time but I also need it to store the User ID of the logged in user (which is stored in a php session variable).</p>
  
  <p>This data will then be queried to create site-wide or per user analytics reports as required at a later date - things such as total number of visits/unique visits, page views in a certain time period, geo-locating the ip addresses and looking at locations, most active times of day, most active members etc.</p>
  
  <p>The obvious thing to do would be to have a mysql insert statement on each page but if the application is receiving thousands of req/sec, this is going to be a hugh bottleneck on the database so I am looking at alternative, scalable ways of doing this without big infrastructure requirements.</p>
  
  <p>A few of the ideas i've had are:</p>
  
  <p>1) Work on a way for Nginx to be able to log the user_id from the session/application in the normal web server access_log, which can be parsed and loaded into a database periodically (nightly). This feels like a bit of a hack and will need doing on each web server as the system scales out.</p>
  
  <p>2) Log each page request into Redis which has high write speeds - the problem with this is the lack of ability to query the date at a later date.</p>
  
  <p>3) Log each page request into either Memcache/Redis acting as a cache (or a message queue) and from there it would be regularly extracted, inserted into MySQL and removed.</p>
  
  <p>4) Would something like MongoDB which has more query capability be suitable?</p>
  
  <p>I'm interested in how you would approach this and if anyone has any experience of a similar application (or has come across anything online).</p>
  
  <p>I'm also interested on thoughts on how the data could be suitably structured to be stored in memcache/redis.</p>
  
  <p>Thanks</p>