My boy friends ex stayed at his house, can I still trust him?

My boyfriend of 6 month has 4 children with his ex wife. He has custody of two of them and she has the other 2. They have been divorced for 8 years.She brought them up for a week visit. They live 8 hours away. Once she got there she told him the kids could only stay a few days. Instead of driving home and back and home again she stayed at his house! He called and let me know what was going on and the sleeping arangements. Because of work I was not able to be there often but we spoke on the phone alot. I have no reason not to trust him but all of my friends say I am crazy to cont. with him. He said if he wanted her back he could have had her before now. He said the other kids wanted to see there mom and he didn't think it was wrong. I trust him. Am I a fool?