Jennifer Lopez is determined to stay amicable with ex Ben Affleck and see him as much as possible over the holidays and beyond – but she’s finding Jennifer Garner a close competitor and it’s becoming quite the contest behind the scenes, a source exclusively tells In Touch.

A large part of the issue comes from the fact that the Gone Girl star, 52, has three children of his own with ex wife Jen Garner, 52 – Violet, 19, Fin, 15, and Samuel, 12 – and had blended his family with that of his most recent ex, J. Lo, who has two children – twins Max and Emme, 16 – with ex husband Marc Anthony. The complicated situation has Ben “being pulled in opposite directions,” according to the insider.

“On one hand, J. Lo is telling him he should be spending more time with her, because the blended family situation is so new, and it’s vital that they all get off to a good start following the divorce, and just establish a pattern.” the source explains, “and then he has Jen pushing him to spend time with her and their kids together.”

According to a different source who previously spoke to In Touch, the children themselves are planning on sticking together through the divorce. “They have a great relationship. They’re still hanging out and talking all the time,” the second source said of the five of them. “They’re not going to let this divorce ruin what they have.”

Even if their brood is eager to make time for one another, the first source says that, even though Ben “completely acknowledges that their kids are very close – and he loves that and has every intention of encouraging that – from his perspective that doesn’t necessarily mean spending holidays together.”

In fact, he would rather be around Jen Garner to make his life easier. “Let’s face it, spending time with Jen and their kids is just much more comfortable than trying to do the blended family thing. He’s so at ease with her, there’s no tension, no pressure, he can totally be himself.”

Ben and his first ex wife Jen even spent Thanksgiving together with their three children, volunteering in Los Angeles with the Midnight Mission organization on Thursday, November 28, to hand out food to the homeless. The two remained on good terms since splitting in 2015, with the Oscar-winning actor frequently seen running to her for comfort amid the messy split from J. Lo.

The inside adds that, while “he’s still committed to consciously uncoupling with J. Lo, or at least he says he is,” it’s really a matter of trying to calm things down in his life after his relationship to the “Let’s Get Loud” singer exploded over the summer.

“When push comes to shove he’s always going to choose the less stressful route. The fact is he does feel a lot of pressure when he’s with J. Lo there’s no sense of ease for him. He would love to get to that place one day, but he is not there now, and he’s not going to fake it and that has her blaming Jen, not him.”