Mick Jagger, David Bowie, Prince William – Princess Margaret, of course…

And now me.

Mustique is not an island you can bypass. Mustique is one of those islands that actually deserves the words “unique” and “iconic”.

Just north of the Tobago Cays in the Grenadines, it really is the world’s most exclusive (and most expensive) holiday destination, contrived for the rich, the very rich, and the uber-rich.

The only way anyone else can get there is by rocking up on a boat.

Admittedly, they charge you £60 to stop (taking a mooring buoy is mandatory), but that gives you three nights and £20 a night in the Solent would be considered good value.

Also, it’s a lot cheaper than renting a villa or taking a suite at the Cotton House hotel.

I looked up the villas. OK, from the hundred-or-so to choose from, I looked up Princess Margaret’s 10-acre estate, Les Jolies Eaux, now owned by the Mustique Company and available to rent at $42,500 a week.

For that, you get 5-6 bedrooms, a “Media Room”, both a dining pavilion and a dining room – and a pool overlooking the Atlantic in one direction and the Caribbean in the other.

Your live-in staff comprise of a butler, a chef, a gardener and two housekeepers (all expecting a tip of between five and ten per cent.)

On top of that, you have to stump up another 10% for using the island’s amenities – everything from the tennis club to the tasteful wooden boxes to hide the litter bins when your butler serves a picnic on one of the island’s nine contrasting beaches.

There is no chance that you would baulk at the cost of any of this because, even before your booking is accepted, you have to be “approved”. This might be why 85% of the bookings are repeat business – the same families coming back year after year so that they’re as much a part of the community as the Jaggers and Hilfiger’s who own their homes on the island (Jagger has two).

But I wouldn’t be playing tennis – or going to the weekly cocktail party or inviting the neighbours round for dinner (everyone wants to show off their chef, and there is a certain amount of good-natured rivalry).

Actually, I wasn’t even supposed to see as much as I did. No sooner had the harbourmaster collected my money and given me a receipt complete with a 60-word disclaimer than I leapt on my bike to pedal off and tour the island – it’s only as big as Alderney, after all.

It seems the Mustique Company had not imagined that boats stopping here would conceal rusty folding bicycles. They assumed that if anyone wanted to see the island, they would hire a mule with a driver – not the animal, but a curious six-seater conveyance that is something of a cross between a golf cart and a dune buggy.

So, when I arrived at the airport and paused to photograph one of these things outside the bijou bamboo terminal (no private jets, please), a smart young man approached, slipping a discreet blue badge from his pocket: “Security”, it said.

Respectfully, he explained that the regulations prohibited anyone who was not a “resident” from wandering around wherever they pleased – something I should have learned from the information supplied by the harbourmaster.

I assured him the only information I had was the 60-word disclaimer. We agreed that I would make my way back to the harbour where I belonged. I got a little lost on the way, but it was remarkable how many polite security men I found to help me.

Still, it did mean I got to see the statue of Colin Tennant – Lord Glenconner – the man who made Mustique what it is today. He bought the island in 1958 when it was nothing more than a few wild goats and a ramshackle village. He paid £45,000 (£442,000 at today’s prices) and did nothing with it until his father sold the family financial business, leaving him without a job.

Oh yes, he did one thing – one very astute thing: He gave Princess Margaret ten acres as a wedding present. Lady Glenconner was her lady-in-waiting, and they offered the Princess the choice of a silver cocktail shaker from Asprey’s or a whole headland with the best views on the island. There, she commissioned the stage designer Oliver Messel to build her the perfect hideaway from her imploding marriage and the tabloid press.

If you have seen The Crown, you will know all about this – particularly the scene with Roddy Llewellyn and the sun cream – and the paparazzo in the bushes.

Those were the heady days of wild parties with young men in gold loincloths – although, it must also be said that the electricity had a tendency to pack up at awkward moments and there were no shops: everybody had to go down to the harbour and buy their food directly from the boat.

Or, at least, everybody’s chefs had to go down to the harbour…

As it became more civilized in the 1990s, Tennent moved on to another development in St Lucia, and Mustique became quieter – more family-orientated. You can see this in Basil’s Bar – a collection of huts on stilts built out over the water. This is where the two Mustiques meet – where it was always said you might see a fisherman dance with a duchess.

Now merchant bankers dance with their three-year-old daughters as they might at an up-market wedding reception. You can imagine Prince William twirling Princess Charlotte.

I stayed the third night just so I could experience the Sunday night “Sunset Blues” party. The music was first class – just as everything is first class on Mustique, from the lobster to the horses in the riding stables.

But I still felt like an outsider. It took me a while to work out why this was: I was the only one handing fistfuls of Eastern Caribbean dollars across the bar – or worse still, a whole handful of change that had been rattling about the chart table.

The rich don’t carry money. They certainly wouldn’t be seen dead with a pocketful of change spoiling the cut of their white linen trousers (£178 from LottyB at The Pink House).

Whether it’s a round of Mustique Whammies at Basil’s or ice creams for a gang of Manhattan teenagers invading the Sweetie Pie Bakery … on Mustique, just charge it to the villa.