For people in their 20s and 30s in a culture where showing up late or not at all after rsvp is acceptable it is better to invite lots of people having all the foods on the stove keeping warm or laid out family style so people can grab a plate when they arrive or are hungry. Say it starts at 7pm and expect people arriving as late as 11pm.
I've found a good way to actually figure out how many people are coming is to require people to actually write a message to say they're going. If you just hit the "going" button and don't reply saying you're actually coming, I assume you have no intention of showing up.
Also, it's incredibly rude to show up to an event 4 hours late. 30 minutes, I can understand, if you got caught up, but 4 hours is taking the piss.