Chamomile/Lavender tea

Does anybody know if it is safe to drink chamomile & lavender tea when you're trying to conceive?  I see differing opinions on the internet.  I wonder if anyone has asked their NaPro doctor about it?  I sure need the stress relief benefits, but don't want to mess with my fertility if chamomile or lavender could possibly affect anything.  I'm super nervous about it because with my last miscarriage, I had been drinking peppermint tea at least 2 times a day.  It may have had nothing to do with it, but I have read that mint can somehow cause miscarriages.