Time zones
Time zones are determined politically. Keeping track of what time it is everywhere is subject to the whims of sovereigns and legislatures.
Let's step back a little and ask, what do we want out of a clock? For instance, what time should solar noon be? What time should dawn be? What time should sunset be? Why should that matter?
In the tropics, this is not a big deal. However as you leave the tropics, the length of the extent of useful sunlight varies until a day (or night) lasts half a year at the poles. Approaching the poles, "the time" and "the date" are concepts that make sense somewhere else, especially after sundown.
The idea of standardized times and time zones only really dates from the age of railroads. Prior to that, you couldn't get anywhere fast enough to make a difference, and time was very local, based on the loudest clock tower.
The continental US (sorry, Alaska and Hawaii and Guam) is about 3 hrs and 48 minutes wide. This is currently fit into 4 time zones. These zones have borders based on political boundaries, so parts of each zone may be astronomically separated by more than 1 hour. Furthermore, the length of the day in the southern parts of the country varies less than in the northern parts throughout the year. Summer daylight in Brunswick, ME is about 7 hours longer than winter daylight, whereas in Corpus Christi, TX the difference is only 4 hours. Therefore, it's less disruptive in the South to make no accommodations to the official local time than in the North.
Nevertheless, it'd be nice to get daylight aligned to a working day: so as to be able to be used during whatever passes for a commute on both ends. The easiest way to do that is to make the working day vary with the light available, or to eliminate the commute, period. Agriculture has always needed to align with natural cycles, why not everything else?
That said, rather than linking everything to Universal Coordinated Time (UTC), which makes a lot of sense for any inter-temporal transactions, we'd like to have local times that give you an idea of how much light and heat there will be from the number alone.
Right now, the time zones we have are not particularly accurate everywhere locally within them, but it's not that impractical. We could move to a system of finer grained zones, say, 8 of them roughly 1/2 hour wide each, and split up north to south so that southern ones don't need to compensate through the year whereas northern ones do, possibly more than twice a year. Fortunately, they'd only be a half hour early or late if they missed the time zone change. This would be much improved in terms of daylight alignment, but people would be even more angry about clock changes than they are now.
Globally, there already are a number of time zones aligned on the half hour relative to UTC. All of India, and Iran, for example. Nepal Time is 5 hours and 45 minutes ahead of UTC, showing it can be even more fine-grained.
China, which also spans about 4 hours in width, is all on one time zone, and the day length difference is about 14 hours longer in the summer than in winter in the far north, whereas the difference in Hong Kong is only 3 hours. Clearly, China is not fiddling with the number on the clock to make it correspond to any nation-wide concept of how a named hour is experienced!
To summarize, the current twice-a-year changes, spread among the states and possessions, is a little frustrating twice a year, but again, it's less frustrating than either making it more precise or making it less precise.