I had a huge 1st bday party for my firstborn. At home (catered! Seriously?) with probably 60+ people. Poor baby was so underwhelmed, he slept through most of it and was only interested at cake time (then promptly went back to sleep).
I lived, I learned. Everyone else had fun, though! Ha!
After that, didn't have big/elaborate birthday parties for the kids until around 5. That's when the fun starts!