English – and possibly other languages – get quite weird when it comes to place names. Each seems to become an idiom on its own.

Some place name descriptors, or ‘geographicals’ like Mount, River, and Island, precede the name of that location:
Mount Everest, not *Everest Mount or *Everest Mountain
but there are also those which follow the name:
Magic Mountain, not *Mountain Magic.

Similarly:
Isle of Wight, not *Wight Isle or *Wight Island
Treasure Island, not *Island Treasure or *Isle Treasure.

Indeed, if you heard someone speak ‘Mountain Magic’ or ‘Island Treasure’, you would assume that they were using common nouns, not proper nouns, and gather quite a different meaning.

However most other geographical nouns follow the location name:
Atlantic Ocean
Black Sea
Danube River, but note River Thames, and a few similar
Beaver Creek
Arrowhead Lake, but note some like Lake Huron
College Pond
Red Hill.

Bays and Gulfs (?Gulves) tend to occur preceding or following, depending on which location we are referring to:
Bay of Biscay, but Mount’s Bay
Gulf of Mexico, but Persian Gulf.

Such geographicals fall between toponymy (the study of place names) and regular linguistics, and I sense have not been well studied by either side.

Does anyone know of any literature which elucidates these idiosyncratic ‘rules’, or do we just learn each one individually?

Thanks to Peter Culicover and Ray Jackendoff for drawing attention to these in their Simpler Syntax (pp 28-9, Oxford UP, 2005). I had previously incorrectly attributed these to the former alone, in his equally fascinating Syntactic Nuts (Oxford UP, 1999).