What matters more, in this context, is if you let this personal prejudice influence how you treat your co-workers. Do you value their input and opinions less? Do you think you somehow do a better job because of your kid-raising credentials?