A second isn’t what you think it is. Scientifically, it’s not defined as 1/60th of a minute, but as “the duration of 9,192,631,770 periods of the radiation corresponding to the transition between the two hyperfine levels of the ground state of the caesium 133 atom”.
What you think of as a day (how long it takes the Earth to rotate) isn’t 24 hours. It’s 23 hours, 56 minutes and 4.2 seconds. The reason it’s 24 hours from sunrise to sunrise is because every day the Earth moves further in its orbit round the sun – and the change in its position lengthens the day slightly.