They can be spotted meandering the streets with babies in knapsacks, or pushing strollers, or changing diapers. They garner the soft smiles of women of all ages, melting hearts with their seemingly out-of-place ability to nurture. The fact is, times are changing when it comes to family dynamics: more and more men choose to be stay-at-home dads, or working fathers who split home duties with their wives.

More women are in the workforce than ever before, all while balancing kids, cooking, cleaning, and everything else it takes to be a functional adult with offspring. And while it was often assumed that women were the ones who struggled to put their needs first when they were raising kids, a new study shows that fathers have the same exact problem.

It’s not just women who are working the double shift — the full-time, high-powered job in addition to taking care of kids and home. It doesn’t matter whether you’re male or female anymore: if you’re in your 30s and 40s, your schedule is going to be jam-packed. If you want to stay fit, bring in the bacon, as well as spend quality time with your family, you’re going to have a difficult time, whether you’re the matriarch or the patriarch of the family.

The study, published in BioMed Central Public Health, focused on the responsibilities of fathers as gender roles continue to shift in society, as well as its link to the overall physical activity and fitness of dads. “A decline or lack of exercise among working parents has mostly been recognized as a female issue,” Emily Mailey, assistant professor of kinesiology at Kansas State University and lead researcher of the study, said in the press release. “The ethic of care theory — that females have been socialized to meet everyone else’s needs before their own — explains why women feel guilty when they take time to exercise, though the same principle hasn’t been studied for fathers.”

The research team found that fathers reported feeling more guilty about exercise than women did, seeing it as a selfish act. In addition, fathers also felt guilty about not spending enough time with their spouses. “That really didn’t come up for women,” Mailey said. “The men felt guilty about exercising after the kids go to bed because that would be time they could spend with their wives.”

While both males and females reported feeling somewhat guilty when they took time to exercise — seeing it as a “selfish” time suck that took them away from their kids or other duties — Mailey says that active parents were more likely to reconcile the idea of exercise with their other duties: “Active parents were able to see exercise as something that contributed to the good of the family and that was not at odds with being good parents,” she said. “As a result, they felt less guilty about taking time to exercise and were more apt to prioritize physical activity because they valued the benefits.”

That being said, whether you’re a stay-at-home dad or a female housewife — or if you’re both working parents with equal duties at home — don’t neglect your own needs for exercise. They might just complement your ability to take care of your family.