See below from Wikipedia. Basically, things like earthquakes and melting glaciers redistribute the earth's mass, thus shifting the moment of inertia and making rotation vary unpredictably.

Leap seconds are irregularly spaced because the Earth's rotation speed changes irregularly. Indeed, the Earth's rotation is quite unpredictable in the long term, which explains why leap seconds are announced only six months in advance.

A mathematical model of the variations in the length of the solar day was developed by F. R. Stephenson and L. V. Morrison,[12] based on records of eclipses for the period 700 BC to 1623 AD, telescopic observations of occultations for the period 1623 until 1967 and atomic clocks thereafter. The model shows a steady increase of the mean solar day by 1.70 ms (± 0.05 ms) per century, plus a periodic shift of about 4 ms amplitude and period of about 1,500 yr.[12] Over the last few centuries, the periodic component reduced the rate of lengthening of the mean solar day to about 1.4 ms per century.[17]

The main reason for the slowing down of the Earth's rotation is tidal friction, which alone would lengthen the day by 2.3 ms/century.[12] Other contributing factors are the movement of the Earth's crust relative to its core, changes in mantle convection, and any other events or processes that cause a significant redistribution of mass. These processes change the Earth's moment of inertia, affecting the rate of rotation due to conservation of angular momentum, sometimes increasing earth's rotational speed (decreasing the solar day and opposing tidal friction). For example, glacial rebound shortens the solar day by 0.6 ms/century and the 2004 Indian Ocean earthquake is thought to have shortened it by 2.68 microseconds.[18]