The site dates from the late 13th century; the earliest charter for the lands dating from 1256, but the remains of the building are of 15th- and 16th-century origin. One tradition is that the castle was built by the Danes and another states that the Mackinnons held the castle from Alexander III, as a reward for their valour at the Battle of Largs. The castle is the point of origin of the Kennedys of Carrick, who once ruled over much of south western Scotland and were granted the lands in 1357. Sir James Balfour described Dunure as a grate and pleasand stronge housse, the most ancient habitation of the surname of Kennedy, Lairds of Dunure, now Earles of Cassiles. This family should not be confused with the renowned American Kennedy family which came from Co. Wexford in Ireland. In August 1563, Mary, Queen of Scots, visited the castle for three days during her third progress round the west of the country. The Celtic name Dunure or Dunoure is said to derive from the "hill" or "fort of the yew tree".