Belmont is also my favorite racetrack, and I grew up around Keeneland.
Toga is very cool, but a real pain in the ass for travelers.
I would rather stay in Manhattan Better hotels, + the entertainmrent shopping and food are beyond compare. Airports are much more convenient and you can't beat the LIRR deal.

Keeneland still offers the best racing and a mellow atmosphere, with great food considering.

Outside of Belmont,Keeneland,CD and Royal Ascot, I don't care if I never see another racetracck.

JMO