Where magic lives

Friday, October 27, 2006

A Spy By Nature

A Spy by Nature is a story unwound in a very interesting way. Hints were dropped about each of the characters real intentions and affiliations. These hints were then unwound in a way that made the book hard to put down.

The flaws in the main character were at times frustrating but I supposed justified by his life trapped inbetween different pillars of secrecy.

Overall: A fun read.


Monday, October 23, 2006

File Upload Status Bar for PHP

When you are using PHP and Apache and you upload a file to a PHP script using a multipart/form-data POST your PHP script typically does not start executing until all of the POST data has reached the server. This makes it impossible to provide server-side feedback to your clients about the status of their uploads.

One way around this problem is to have another process running on the server capable of reading the clients headers, passes all data received onto the actual web server whilst keeping count of how much data it has passed on and telling clients the current status of a given request. This however sounds easier than it actually it is when it comes around to implementation. Your server program has to understand quite a bit about HTTP (not a simple protocol in its full detail) and there are logging and security implications (suddenly your webserver thinks a lot of its requests are coming from when they are not).

Anyway, I have created a "first draft" of such a program. It cheats a bit by forcing the Connection header of every HTTP request and response to Close so that it only has to deal with one request per connection (this should be fine if you only send file uploads through the proxy program). I attempt to handle the logging program by writing to standard output my own log in the same format as my Apache server does so that this output can be piped into the Apache log file (requests will then appear duplicated but at least you have a record of the actual end users who have used the server).

The solution is by no means complete, it works, and it is secure in the sense that it wont give a remote user any more that it "says on the tin" but I have not tested it under load and it is probably quite susceptible to denial of service attacks in its current state. I am releasing the Java source code hoping that people might make improvements and give feedback, use it at your own risk!

Usage instructions from the source code:

  * Command Line Usage:
  *  java UploadServer 81 that-server >> 
  *    /location/of/your/apache/access_log
  *  Where 81 is a spare TCP port to listen on, and 
  *  that-server is a server listening for web 
  *  connections on port 80.
  * Remote usage:
  *  http://this-server:81/foo
  *  Will return http://that-server:80/foo (works with GET,
  *  POST, etc.)
  *  http://this-server:81/foo___STATS___ will return a string
  *  describing how many bytes have been transmitted in the 
  *  current users (based on IP address and Cookie contents) 
  *  request to http://this-server:81/foo and (if available) 
  *  how many bytes the server is expecting.

Some JavaScript to display a status bar might work something like this (using the forms onsubmit attribute to call statusBar() and <div id="status"><h2>Upload status</h2><div id="statusinner">Starting upload...</div></div> somewhere in your HTML):

 function requestObject()
  return ((navigator.appName == "Microsoft Internet Explorer") ? 
   new ActiveXObject("Microsoft.XMLHTTP") :  new XMLHttpRequest());
 var statusconn = requestObject();
 var statusBarUri = '';
 function statusBar()
  document.getElementById('status').style.visibility = "visible";
 function startStatusBar()
  statusconn.open('get', 'http://this-server:81/foo___STATS___',
  statusconn.onreadystatechange = updateStatusBar;
 function updateStatusBar()
  if(statusconn.readyState == 4)
    document.getElementById('statusinner').innerHTML = 
     "(Status bar loading...)";
     document.getElementById('statusinner').innerHTML = "(" + 
      (Math.round(parseInt(statusconn.responseText)/1024)) + 
      "KB uploaded so far.)";
     var bits = statusconn.responseText.split("/");
      (Math.round(parseInt(bits[0])/1024)) + 
      "KB/" + (Math.round(parseInt(bits[1])/1024)) + 
      "KB uploaded so far.)";     
   setTimeout("startStatusBar()", 100);


Monday, October 16, 2006

PHP Image Uploader and Manipulator

Recently I created some code to allow users to upload a JPEG image to a website and resize and crop the image afterwards.

Seeing as this sort of feature could be useful on a lot of sites I made the code as general as possible and am releasing the source code here.

Here is a live demo of the utility.

The source code is in a zip file. You provide a new version of the cropper.php file for any place on the site that you wish to use the tool. Here is an example to show the customisation options available:


 // This script stores temporary image data in the user's session
 // The dimensions the uploaded image is resized to.
 // (The aspect ratio is maintained and no quality is lost during 
 // the final crop no matter how small you set these values)
 $default_max_width = 640;
 $default_max_height = 480;

 // The size of the image the user will create:
 $cropped_width = 440;
 $cropped_height = 280;

 // A name for the session variable the script will use (must be
 // unique for your domain)
 $unique_session_key = 'imgcropper';

 // A directory that the webserver can create files in, read from
 // and delete files from (the trailing slash is required)

 // Before the user uploads their own image they get to play with
 // this one:
 $default_image = 'default.jpg';

 // A semi transparent gif used to create the darkened effect
 // around the edge of the page.
 $border_background = 'screen.gif';
 // After the user has chosen the area to crop, the cropped 
 // image is passed to the following function that you must 
 // write your own code for:
 function storeImage($img_id)
  // $img_id is a GD image reference, if you want the actual 
  // JPEG data pass it to imgdata (as below)
  $_SESSION['imgcropper']['cropped'] = imgdata($img_id); 
 // ...then the user is redirected to this page: 
 $redirect_after_crop = 'done.php';

 // All of the code:

The reusable file cropper-include.php contains all of the PHP image uploading and resizing logic, along with the HTML, JavaScript and CSS for the page that allows users to upload their image, drag it around, resize it and save it.

My code assumes that users will not upload images that are too large and I use the memory limit setting in my php.ini file to prevent denial of service attacks. You may wish to alter the code to check the size of uploaded images.

Update (7 Feb 2007): I have previously answered peoples questions in comments on the blog post However due to new work commitments I will probably be too busy to continue doing this so I have set up a Google Group at for people to post their questions to and hopefully other users of the script will be able to provide answers.


Saturday, October 14, 2006

On the Google YouTube purchase

An article in today's Economist provides a good brief summary of Google's aquisition of YouTube. It also gives a good (but journalistic) summary of where all the big online players now stand in relation to each other.

However the article does carelessly describe Microsoft as the software giant that dominated "the first generation of the web" in a way that implies they are behind now. I hate to use the phrase "Web 2.0" but you don't have to walk far from "Web 2.0" before you see a JavaScript XMLHttpRequest object, and we all know who invented that!

Labels: ,

Wednesday, October 11, 2006

Where will your data go today?

Google have recently integrated Writely and there web-based Spreadsheets service into "Google Docs and Spreadsheets".

What these two web-based applications miss in ease of productivity (which is a lot - there is only so much you can do inside the current web browser) they make up for in ease of collaboration. However, your data remains on Google servers, this is not good and should put most potential users off. Why-oh-why can't a software company send this modern web-based software vision in a slightly different direction and make the software to be sold to businesses as server software. Leave the businesses to scale their installations appropriately, organise backups, etc. Even if the software is designed so this can be done in an easy "click and build" kind of a way, the end user needs to be in control of their own data.


Tuesday, October 03, 2006

The Computer and the Brain

The Computer and the Brain by John von Neumann, a pioneer of computing is a very interesting historical read.

The book is essentially divided into two sections. The first is an overview of what a "computer" is and how it works, the second a comparison of computing technology with the brain.

It is amazing how relevant the first section of the book is in relation to computing today. Von Neumann's account is only tainted by slight changes in technical vocabulary and major improvements in performance over the past 50 years; the fundamental theory is all there and it is fascinating to hear it in his own words.

The second section raised some very interesting comparisons in a very clear and concise manor. However I did feel that the coverage could have been a lot deeper, the reason for this is probably the authors ill health at the time of writing though. I would be interested in reading a more modern approach at the same comparison, both to see if it has changed at all (are Von Neumann's biological facts still sound and complete according to modern day Biology?) and to delve further into the subject.


Sunday, October 01, 2006

Mr David James Nicholson BA Hons (Oxon)

Yesterday was my degree ceremony at Oxford so I now officially have a degree.

A walk through Oxford in the heavy rain to the Sheldonian Theatre wearing sub fusc was followed by listening to a lot of latin, and seeing quite a lot of bowing. There was then reception and lunch, all with free wine. All this featured a lot of photography.

As for the evening entertainment afterwards, where else would we go other than the Purple Turtle?