Most good parents don't leave their very young children unattended whilst they are wining and dining a good distance away.

I agree, they are guilty of neglect, but equally most people are not involved in abductions and child trafficking either, which is most probably the crime that was committed on that day.
There is a chance that Madeleine could have been taken, even if the parents had been in the apartment.