link to Pipe Dreams

Back to Stewart Fist Article Index

CROSSROADS The Australian. August 1998

MPEG in a round hole

by Stewart Fist

No single standard in recent times has won such universal acceptance as MPEG, the Motion Picture Experts Group's global digital video compression standard. Digital television, CD-ROMS, DVD disks, videoconferencing, the Internet, and a whole range of computer-based storage and real-time communications systems use MPEG today.

This is actually a collaborative group of compression and decompression techniques grouped into a 'framework'. If one of the higher technical standards isn't available, there are cascading defaults.

The lowest member of the family doesn't even carry the MPEG name. JPEG (Joint Photographic Experts Group) is widely used on the Internet for images, illustrations and still photographs.

It depends entirely on 'spacial' or 'intra-frame' compression techniques. The idea is to identify and remove 'redundant' (non-essential) data contained in a simple two-dimensional image.

For instance, a large flat area of blue sky can be coded either pixel-by-pixel, or the same information can be compressed by an instruction: "Make the next 200 pixels the same as this one." So in this context, redundant generally means 'repeated'.

At the most fundamental level, JPEG uses very complex mathematics to identify and separate the macro-scale elements in an image (say, the general shape of a building, doors, roof, etc.) from the micro-scale elements that provide textural qualities - say, the appearance of surface roughness on a wall.

In effect, it is able to construct a hierarchy of importance which enable it to distinguish the more important elements from the less. This encoding technique is called Discrete Cosine Transform (DCT), and it's only been in chip form for a few years.

With JPEG, coloured photographs can generally be compressed to one-fifth their size and still be restored perfectly. In practice, most photo files will compress to one-tenth or more, with almost undetectable quality change, and higher ratios are possible if some fine-detail loss is permissible.

The current version of JPEG (correctly called JPEG-2) has four different modes of operation including a form of hierarchical encoding where the file has multiple resolution levels. The information is stored in a way that allows lower-quality images to be extracted quickly and easily without needing top-down decompression. This means that you can reproduce a postage-stamp size version for indexing by extracting just the macro-scale elements.

MPEG goes further than JPEG, and achieves much higher compression ratios. It is used for both motion picture and for sound, and it's now been extended from a single standard into a framework. There's a wide range of different techniques available in an MPEG decoding chip, and also a method of controlling which of these decoding standards will be selected and used.

It is important here to distinguish between two fundamental coding techniques: intra-frame coding, which applies to the two-dimensional spacial techniques used in JPEG within one frame only, and inter-frame coding which comes about when two or more similar images are compared.

Inter-frame coding is sometimes called 'temporal' (time), since it applies only to moving video images with minor changes between successive frames.

Just to confuse you, these inter-frame techniques are also called 'predictive'. What they mean is that successive movie images are pretty much the same, so the current image 'predicts' the next.

The problem of predicting video therefore involves compressing each frame as a distinct entity using inter-frame coding, then comparing the current compressed image with the next - mathematically extracting the difference - and then only transmitting the changes.

Actually in MPEG the predict forward, and backward, by holding a large number of images in memory, and only transmitting the occasional full frame.

All of these techniques require the receiver to be a small, dedicated, powerful computer, capable of storing many digital images as a Group of Pictures (GOP). For the optimal use of bandwidth or storage space, they provide full image information (using only spacial techniques) about once a second (called I-frames), and then fill in the twenty-four frames between by sending only difference information (B- and P-frames).

The term 'temporal' coding suggests that the image must be delayed a little before transmission for spacial and predictive compression, and then delayed again after reception while the new images is reconstructed.

Delays don't create a problem with one-way broadcasting, but they can with interactive forms of communications. Fortunately, with modern high-speed processing chips these delays are now only in the order of a half-second or so - but this still makes video-conferencing uncomfortable.

There's always a trade-off here between the best possible image-quality, the amount of movement taking place on the screen, interactive processing delays, and the cost of the compression and decompression equipment.

Fortunately, compression requires much more processing power than decompression - so these systems are asymmetrical in their processing requirements. With broadcasting, CD-ROM, and DVD, the best possible equipment can be used for compression and delays aren't a problem. Broadcast MPEG may reproduce very good quality video even after the images have been compressed by a ratio of 100-to-1.

However MPEG is never perfectly 'lossless' in terms of image quality. Much of these higher compression ratios are only possible because the processor knows which data to throw away with the least subjective deterioration of quality. This is especially important when objects move on the screen - and it is screen-movement which creates the differences between images.

Our eyes are very poor interpreters of fine detail with moving images - we tend to see movement more as an overall impression - a blur. So MPEG has a compensating trade-off - the more movement there is on the screen, the more information difference data there is to be transmitted. However less fine detail needs to be included in each frame since the eye won't see it anyway. This is why DCT is so important.

MPEG began life as a compression system for CD-ROM. We now call this standard MPEG-1, and for a while it was used for satellite television as an interim measure while MPEG-2 was being developed. MPEG-2 was originally designed only to replace our current analog television systems. Then MPEG-3 was suggested to provide full high-definition television quality.

Fortunately, the standards designers realised that all three approaches could be kept under the same umbrella as a 'framework' of compatible standards, with a control system which allowed the receiver to make its own choice.

So today we have only the one MPEG framework, but it is divided into four levels (different screen image qualities) and five profiles (availability of different processing power and decompression algorithms), making 20 possibilities in all.

There is another standards development outside this framework which is now called MPEG-4, and it is perhaps the most interesting of all. In fact it could be the way of the future - but it's hard to judge at this stage.

MPEG-4 was an attempt to provide even greater compression for video telephones and video conferencing, then later they upgraded it and moved into the games area. Now they are looking at extending it further into broadcasting. The final standard will be available in early 1999.

The original idea was to incorporate a different technique of base-level coding called Discrete Wavelet Transform (DWT); the existing standards have been built entirely around DCT. And they also wanted to provide for computer-generated images and sound (MIDI, speech synthesis, etc,).

DWT appears to offer some unique advantages, especially down at the low-quality, very-low-bandwidth end of the scale. However, since there was no simple way to add DWT to the existing framework, MPEG-4 became quite separate.

The key difference is that it treats elements of the image and sound as separate objects.

For instance a television news program might consist of half-a-dozen audio/visual objects (AVOs): the news footage, the image of the anchor person, the audio of his voice, the sound effects of the news footage, and perhaps closed captioning, and maybe a clock. Normally, these elements would be combined in the station before transmission.

But in MPEG-4, each AVO can be transmitted separately, and the user's receiver combines them according to a transmitted plan, or under user-control.

In addition to the conventional MPEG-2 spacial and predictive coding techniques, MPEG-4 also has sprite encoding which depends on computer-synthesised image elements. Sprites are complex mathematically constructed models, sometimes simple (such as a computer cursor) and sometimes more complex, perhaps even humanoid in appearance.

So the final image/sound may be a complex hierarchy of various objects. Some of these will be based on natural images and some on synthesised outlines, shapes, textures and/or sounds, and these can all be reconstructed and layered in two- or three-dimensions, under pre-programmed or user-control.

This isn't going to be just a change in TV quality or a wider screen, but a new form of visual and aural experience entirely.


© Stewart Fist 1998
Original Source:

Back to Stewart Fist Article Index

link to Pipe Dreams