Choosing a Compact Camera
Here's what I recommend, you can read my reasoning below!
Buy yourself an 8 to 12 megapixel camera that offers full manual control of aperture and shutter speed, plus manual white balance. Make of camera is irrelevant. Buy the housing when you buy the camera, and make certain the housing will take conversion lenses. Get yourself an external flashgun as your first accessory and a wide-angle lens or a macro lens next, depending on what you're interested in photographing. Complete your set-up with the other lens.
Then take LOTS of pictures - photographers make pictures, not cameras, and the only way to develop your ability as a photographer is to take pictures

Here's my complete (Though elderly - it's all over four years old!) compact outfit. From left - macro lens, Wide-angle lens, O-ring grease, camera battery charger, camera and housing, flashgun with arm, fibre-optic synch cable and rechargeable AA batteries
Here's my reasoning....
Brand and Price
Forget brand and price - if it's a digital compact it will, in the real world, give you exactly the same results as any other digital compact, and if you see a really cheap deal it probably means it's last year's model and no worse for that.
There are a fair number of websites that review cameras and post sample pictures, so try this test for yourself. Download a picture from the camera you would like and another from the camera you currently own, make sure both were shot at the same ISO setting and under similar conditions, then print the pictures to A4 and see if you can see any significant differences for yourself. When you're convinced, concentrate instead on the spec and feature set of the camera you should buy.
Here are two good sites from which to get hold of sample shots:
http://www.steves-digicams.com/

Taken on my first digital compact, a Ricoh Caplio RR30 - just three million pixels, no external flash and no wide-angle lens adapter.
Pixel Count
The headline feature on any digital cameras is the pixel count, and everyone knows that more is better. That's sort of true for the amount of detail the camera can record, but it isn't that easy. The sensor area of a digital compact is tiny, 7.2mm by 5.3mm is a common size, and however many pixels you have they must all be crammed into this tiny area. To get more pixels in you have to make the individual pixels smaller and squeeze them closer together. Pixels work by converting light into electrical energy, and the smaller and closer together they get the more likely it is that the electrical energy from one pixel can leak and affect neighbouring pixels. The result is less detail in the picture or more noise in the image or both. The manufacturers are well aware of this, of course, and include sophisticated image processing routines that minimise the appearance in finished images, but the first generation of a sensor will always be noisier than subsequent generations, so whatever the current headline pixel count, drop back a size or two and go for second generation models with that spec.
As of now, the headline count is almost 15 million pixels. 5 million pixels is a good minimum for a high quality A4 print, so go for a good second-generation 8 to 12 million pixel model and you get the best quality possible now.
Here's what image noise looks like:

Top picture at ISO50, bottom at ISO 320, the maximum on my Olympus. The grainy, pointillist look of the bottom shot is image noise and is particulalry obvious in the darker areas of the picture. Sensors crowded with pixels give the same effect, though not to the extent shown here. Note that this is the centre cropped from a much larger picture, you're looking at the equivaklent of a 16 x 24 inch print!
Exposure Modes
You need to be able to control the camera to deliver the results you want. The makers offer loads of scene modes to make it easier to get good pictures, but there's an easier way, turn them all off and set the shutter speed and aperture manually.
Shutter speed is the time for which the camera shutter is open and allows light to hit the sensor. Shutter speeds are marked in the sequence 1/15th sec, 1/30th sec, /160th sec, 1/125thsec, 1/250th sec, 1/500th sec and so on, with longer and shorter shutter speeds available. Each step represents a halving of the time you allow light to enter the camera. If the shutter speed gets too low, longer than about 1/60th sec, the camera can shake a little when you take the shot and you get soft pictures. If you have steady hands, particularly underwater where vibrations are damped by the water, you can get away with much longer exposures. Some people are able to use shutter speeds as slow as 1/4 sec, but a realistic minimum for hand-held shooting for most of us is 1/30th sec.
Aperture is the size of the 'hole' through which you permit light to pass on the way to the sensor. Think of it like a pair of curtains - if you want to stop some of the light getting into a room you can part-draw the curtains and make the window effectively smaller. Apertures are marked in the sequence f2, f2.8, f4, f5.6. f8, f11, f16 and so on, where each step in the sequence marks a halving of the light entering the camera. A small aperture has a big number (The smallest aperture on most digital compacts is f8) and a large aperture has a small number (Usually f2.8). The smaller the aperture (Bigger number), the less light you allow into the camera.
If you make the aperture smaller but allow the shutter to stay open longer you can keep the total amount of light hitting the sensor exactly the same, so 1/60th sec at f4 is exactly the same exposure as 1/125th at f2.8 or 1/30th at f5.6
ISO is a way to express the sensitivity to light of your sensor. The sequence runs ISO 50, ISO 100, ISO 200, ISO 400, ISO 800, ISO 1600 and so on. Some cameras offer settings of ISO 3200 or even ISO 6400, but the higher the ISO the noisier and less detailed your picture becomes, especially in compact cameras with small sensors. This is because higher sensitivities are effectively achieved by amplifying the electrical output of the pixels, and that amplifies any background noise along with the proper signal.
If you halve or double the ISO value, it halves or doubles the sensitivity of the sensor, so changing from ISO 50 to ISO 100 makes exactly the same difference to the total exposure as going from f4 to f2.8 or from 1/60th sec shutter speed to 1/30th sec.

Shot with the Olympus and external strobe at the Farne Islands
So what does this mean in real life? If you set your camera to auto it will select what it thinks is the most appropriate aperture, shutter speed and ISO setting for the picture you're trying to take. To be fair, the camera makers have determined what settings the camera will pick based on analysis of millions of pictures taken under all sorts of lighting conditions and the chances are you'll get a decent result on land. What they haven't done is to analyse many underwater pictures, and conditions underwater are very different.
So, switch your camera to manual, use the lowest ISO your camera is capable of, set 1/30th sec and f4 in UK waters (Try 1/60th and f5.6 in the tropics) and and take a picture. Look at the screen on the back of the camera and review your shot. If it looks too dark open up the aperture (From f4 to f2.8) or decrease the shutter speed (From 1/60th to 1/30th) or both if necessary. Take another shot and repeat until you're happy. It sounds tedious and laborious and very hit and miss, and compared to all-singing, all-dancing auto modes it is, but very quickly you'll realise that in certain conditions you'll need to use certain settings and before long you'll be hitting the optimum settings on your second shot, which is as good as you can expect with automation if you're at all picky.
Alternatively, you can use Aperture Priority and the exposure compensation button. You set the aperture (f4 in the UK and f5.6 in the tropics) and the camera sets the shutter speed. If the shutter speed drops too low, below 1/30th sec, you need to open the aperture (To f4 or f2.8) and take your shot. If it looks too dark or too light use the exposure comensation control.
if you can't get a fast enough shutter speed even with t he lens set wide open (Usually f2.8) increase the ISO setting, but only as much as you have to. ISO 200 is usually pretty good on new cameras, but ISO 400 is likely to be iffy, 800 horrible and 1600 (Or even 3200!) unusable for anything but thumbnail sized snaps.
Manual White Balance
The only other camera feature you really need is manual white balance. Natural daylight contains light of different colours mixed together, which is necessary for us to see colour. When you dive the water filters out the warm colours, making the water seem very blue or green. You can see this on any dive when you point a torch at a little bit of drab gray stuff and find out it is really a brilliant red splash. Go deep enough, of course, and the water absorbs all the light and it becomes totally dark, a situation familiar to UK divers!

A familar sight, the one handed compact shot! The colour in this picture comes from adjusting the colour balance in Photoshop, but it's much better to set white balance manually before taking the shot
Underwater your camera has to compensate for the missing red. Pixels on the sensor are sensitive to just one colour; red, green or blue*. If there isn't much red in the light the output from the red sensitive pixels can be boosted artificially to put it back and deliver a colourful picture. The auto white balance control on your camera tries to balance the colour of the final picture so it contains all the colours. Automatic white balance works well above the water, where it was designed and tested, but usually fails badly underwater.
*Except for Foveon sensors, but the principle remains the same!
Instead, use a white or neutral gray subject and set the white balance manually every time you change depth. I use my white fins (Well, whitish, they're pretty battered these days!), and I've seen people use a sandy bottom or grey rocks successfully. Every camera does this in a slightly different way, read the manual for yours, just make sure that setting manual white balance is possible before buy. You'll be astounded how deep you can go and still get decent colour using this technique.
Don't mix manual white balance and flash - anything lit by the flash will be brightly coloured and very, very red!
Colour filters are also available to give you even better results. These are reddish coloured filters that can help the camer's white balance setting and are extremely effective.

The left hand image was taken after manually white-balancing, the right hand image was taken with white balance set to cloudy, my usual default underwater as I like slightly warm images. Which you prefer is up to you, but the left hand image is closer to the true colous of the objects photographed whilst the right hand image is closer to what I saw on the dive. Who said the camera can't lie?
Housings
Just as important as the camera is the housing. The inexpensive housings made by the camera makers are usually rated to 40m and operate very nicely thankyou (The Olympus PT-019 seen above has been to 47 metres and works like a dream!), and there are more robust housings rated to 60m available for some cameras from specialists like Ikelite. Buy the housing when you buy the camera. New cameras are constantly being introduced and housings may not be available for older models.

40m rated Olympus PT-019 housing. Accessory lenses screw into the filter thread on the front of the red metal ring around the lens, and there are controls for all camera functions. The two inverted-T shaped blocks on top of the housing are Heinrich-Weikamp converters that permit the internal flash to trigger old non-digital Nikonos compatible strobes and the silver foil inside the housing prevents light from the internal flash escaping forward to create backscatter.
The only thing to look for is that the housing you buy will take accessory lenses. Olympus housings will have a 46mm or 67mm thread on the front of the housing, Inon make adapters for various Canon, Fuji, Sony and other housings, and Ikelite do their own adapters to fit either 67mm or Inon bayonet mount lenses to their housings.
Accessory lenses aren't essential, but they make it easier to photograph a wider range of subjects.
External Flash
Every compact on the market boasts a built-in flash. These, without fail, are so close to the lens that they're next to useless underwater for anything except macro (Close-up) photography. The reason is that the light from the flash lights up the detritus floating in the water and reflects straight back into the lens, producing an effect that's usually described as driving into a snowstorm with your headlights on full. To be fair, it isn't really that bad, but backscatter, as it's called, can still ruin your pictures.

Extremes of the strobe market: an old non-digital Ikelite 100A, which is about the same size as the current DS125/DS160 strobes form Ikelite, and the little Epoque 150, the smallest, lightest and cheapest decent underwater strobe available. Other strobes fall between these two in size and weight, though there are some even bigger flahguns out there. Surprisingly, the light output from these two units is very similar, but the Ikelite units give a much broader, softer beam.
The only way to completely avoid backscatter is not to use flash and instead to rely on manual white balance to restore some colour, with or without a filter. If you do want to use flash, and for saturated colour you'll need to, use an external flashgun, or strobe, as they're called in America, sited well away from the camera lens and pointed outwards rather than straight-forward or inwards. If you're shooting close-ups you can use the flash directed straight at the subject, there probably won't be enough rubbish in the small amount of water between camera and subject to cause backscatter.

There are a number of underwater strobes available, ranging from simple manually adjusted models to complex beasts that can set the flash output automatically to give you good exposures. These are very reliable for close-ups, but not so good for wide-angle shots.
The strobe can be triggered optically or electrically. Optical triggering requires a fibre-optic cable fitted over the internal flash of your compact to transmit the flash of the camera to a sensor on the strobe, which then fires at the same time. Some compacts have a pre-flash which is used to set exposure and which the external strobe must mimic or ignore - just make sure you tell the underwater photography dealer which camera you have and they'll sort you out an appropriate system. If you use the camera flash to trigger your external strobe cover the inside of the housing with silver foil, or something else that is opaque, to prevent the built-in flash lighting up the water and creating backscatter.
Electrical triggering is by a synchronisation cable, and only applicable to a very small number of compact cameras. The advantage is you can turn off the internal flash of the camera and save battery power, the disadvantage is that these cables are temperamental beasts and always break on day one of liveaboard trips, so make sure you take a spare!
Using an external flashgun
A good starting point is to set your aperture and shutter speed to produce a good shot, then close the aperture by one or two stops (From f4 to f5.6 or f8 respectively, for example) and use the strobe to deliver just enough light to restore brightness and colour. The flash is very bright but so short duration that shutter speed has no effect on flash exposures.
If you have an auto exposure camera, set -1 or -2 stop of under-exposure as a starting point.
Position one strobe so that it is level with either the centre of your lens as seen from from above or as seen from the side, and angle the strobe outwards slightly so that you use the edge of the beam of light. Don't aim the strobe straight at your subject, that'll deliver a lot of backscatter.
Position two strobes so that they're at the same height as the lens when seen from the front, one either side of the camera and both angled slightly out to minimse backscatter - see the pictures on the SLR page.
Finally, remember that no flashgun will illuminate objects over about five feet away, the water simply absorbs the light from the flashgun exactly as it does with the ordinary light, so get close. For best results stay within three feet (A metre) of your subject.
Conversion Lenses
The lenses of most compacts offer a zoom range that goes from moderate wide-angle to moderate telephoto. The problem underwater is that the flat port fitted to compact camera housings (See the SLR section for more on ports) effectively increases focal length so you lose the wide-angle setting. To get it back you need to fit a wide-angle conversion lens.
Wide-angle conversion lenses are available from two or three makers, and screw or bayonet-fit onto the front of your housing, provided your housing can take them or a suitable adapter is available. Having used both types, bayonet fittings are easier underwater.

The same housing as above, but with an Epoque wide-angle conversion lens attached
Most reduce focal length by a factor of 0.56x, and this combines with the effect of the water to give you a reasonably wide-angle view. If you want more, and you will, Inon do a very wide-angle bayonet-fit fisheye lens, or an accessory dome to fit their accessory wide-angle, either one of which will give you a dramatically wider view than the standard wide-angle converter.
With a wide-angle you can get sharper, more contrasty and better coloured pictures because you're closer - the thing that spoils most underwater pictures is too much water 'twixt camera and subject.
Here's what wide-angle conversion lenses do:

Note the increase in apparent sharpness and contrast with each step wider, and also the change in perspective - look at the length of the foredeck on the little cabin-cruiser. All shots full frame, UK lake, vis was about 2m
Macro, or close-up, lenses also fit to the front of the housing.

Housing plus macro lens
Most compacts have a macro setting, so why bother with an accessory lens? The close-up setting on your compact will almost certainly lock the lens at the widest angle setting, so to take good close-ups you need to get the camera very close to the subject, sometimes so close that you'll be touching it, and almost certainly have frightened anything that can run away into moving off at high speed, not to mention making it well nigh impossible to get a flashgun in to light your subject.
Here's what a macro lens can do:

Common Pyjama Slug, shot with the Olympus and a macro converter, manual white balance
Macro lenses work by limiting the maximum distance on which the camera lens can focus, regardless of focal length. Technically, the maximum distance the lens will focus on becomes 1metre divided by the power of the lens, so a +4 diopter macro lens will focus at 1/4 of a metre, which is 25cm or 10 inches. That means you can fill the frame with small critters whilst staying far enough away to avoid spooking them, and have room to position your flashgun. Most macro lenses work best at the longer end of your camera zoom, but don't be afraid to experiment.
For more information on shooting close-ups, see the Compact Macro page