I'm an old Long Islander, now living in Florida. As I recall, the ferry from CT docks at Port Jefferson. Although the LIRR has service from there, there is nothing direct to Belmont Park, which is served only from NYC, Brooklyn, and Jamaica. Take it from me, you will undoubtedly be better off driving in, unless you could arrange a rental car at Port Jefferson off the ferry.

As far as getting to the city after the races, yes, it's doable. Again, you'd be better off at least starting out by car, then maybe parking near a subway stop or LIRR stop in Queens to complete your trip into the city that way. Watch what neighborhood you choose to park at, however. I wouldn't recommend driving all the way into Manhattan.