CD 11 - TTC... more like WTT

So DH and I work at the same company and today I noticed that he is due to go back to work May 28.  My fertile days are usually around the first of every month (CD 17).  I did get confirmation that he will be home next week on May 3 so I will most likely miss ovulation this cycle as well as next.  TTC does not seem to be going our way, is this a sign that we should not be TTC? We will still try as much as possible when he is home, but I feel like I am Waiting To Try (conceive) more than Trying To Conceive.  Due to the time change, I have not had a chance to talk to DH about the schedule, but i just needed to vent a little.