According to my mum, I was 2 days old at my first race meeting - an evening in the commentary box at Hackney Speedway. Our kids have grown up with racing so have been going for as long as I can remember - I can certainly remember weekends with my eldest at F3 races when he was three. Their first Le Mans they were 3 and 18 months (although this was just a flying visit!). They've been most years since and my eldest is now 14 and, as Wishy will testify, a full member of our team ....
My thoughts, for what it's worth, is it's not their age (providing you take all the sensible precautions such as proper ear defenders etc.) but more what your wife's views on the matter are
This was an F3 at Thruxton, many years ago, I think he was about three (maybe a little younger)