The primary custody loss could happen if the Changeling star doesn't attempt to improve her relationship with Pitt.
According to documents, the issue stems from the children's lack of relationship with their father due to the amount of time they are spending with Jolie.
According to The Blast, a judge in the couple's ongoing divorce case said it would be "harmful" for the children if they had no relationship with their father as it's in their best interest to spend time with each parent. The court stated that the kids "are safe with their father".
Jolie has reportedly been given a parenting blueprint by the judge, including asking her to give Pitt each child's mobile phone numbers so he can call anytime.
The arrangement kicked off on June 8, with Pitt granted access to the kids for four hours per day as he visits them in London, where Jolie is now filming Maleficent 2.
Jolie and Pitt split in 2016 amid allegations of child abuse against Pitt during an incident aboard a private plane.
May faces another day of Brexit compromise in parliament
That clause - drafted by Grieve - basically hands a lot of power to Parliament if no deal has been agreed by the end of November. Grieve told MPs: "If we don't achieve a deal at all, the fact is we are going to be facing an enormous crisis".
If she does not respect the new rules, she could lose custody of her children.
The Fury actor is set to spend time with his kids 8 June to 17 June for four hours per day while he stays in London, where Jolie now rents a house.
From June 27 to July 1 he will get the children for 10 hours a day and then in July he will see them from July 8 to 14 for four hours a day.
The court also outlined a new custody schedule for the children for the northern summer.
The couple initially separated in a nasty, high-profile breakup in September 2016.
At the end of July Pitt will be able to see them in Los Angeles for a week from July 21 to 29.
Jolie must also provide Pitt with the cellphone number of each child, and allow him to call at will, according to the reports. From early to mid-July, he'll have the kids for four consecutive days, and in late July they'll stay with him in Los Angeles for about a week.
Jolie can only call once a day when the kids are with the actor.
In some ways, the globe-trotting Jolie has come to resent Pitt for "essentially forcing her to settle down in Los Angeles with the kids", Us Weekly reported in May. The arrangement applies to all kids, except the eldest Maddox, 16, who is allowed to decide how much time he wants to spend with each of his parents, Daily Mail reports.