Shemar Moore Recalls Struggling with Being Mixed-Race as a Kid: ‘Where Do I Fit In?’ (Exclusive)

Back when Shemar Moore was growing up, he felt weighed down by a lot of existential questions.

An only child born in Oakland, Calif., to mother Marilyn Wilson-Moore, who was White, and father Sherrod Moore, who was Black, the S.W.A.T. star, 54, recalls pondering as kid, “‘Am I Brown enough? Am I Black enough? Do I walk the walk? I’ll never be White because I’m too Brown. How should I talk? How should I move? How should I dress? Where do I fit in?’ ”

For much of his early life Moore was raised overseas by his mother, an educator, after she and his dad, an Army vet, split up. “When I was six months old, we left the country. My father ran with the Black Panthers. He had a purpose, a positive purpose that he believed in, but it didn’t keep him out of trouble,” he explains.

Moore continues, “He was a very dark-skinned Black man and he caught a lot of heat for that. He got called the N-word a lot. And then he got with this White woman with this big ol’ butt. But she also had a big ol’ brain,” he says of his mom.

Back when his parents met in the ’60s, “Interracial relationships were trendy but they weren’t accepted,” he says. “There was a lot of racial tension and my mother didn’t want to raise me in that. She got a job in Denmark and we left.”

Though his father made visits the first few years, “he showed out a little too much for my mother’s liking and she said, ‘Enough is enough.’ He came back to the States, got in trouble, and spent four years in San Quentin prison.”

Moore and his mom bounced around the globe, from Denmark to Bahrain, Ghana to Greece. “Nothing about my life was ordinary,” he says. But when they returned to the U.S. when he was a little older, he felt the pressure to conform.

When he’d tell his mother about struggling with his racial identity she offered perspective. “She said, ‘Shemar, understand both families’ backgrounds, your Black side, your White side. Respect it all, but just be yourself.”

He credits another adult for helping shape his outlook. “My high school baseball coach Melvin Harrison. He was the closest thing to a father that I ever knew,” says Moore. “He mentored me, taught me baseball, pushed me to be the best I could be, and pushed me to be a proud Black man and an even prouder human being.”

Though being mixed “was an insecurity as a kid,” Moore says that’s not the case anymore. “I don’t apologize for being mixed. I’m a Black man. That’s the way society treats you. But I’m also half White.”

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories.

He continues, “I got Barack Obama tattooed on my back. It’s not about politics. It is the first Black president, but what is Barack Obama? He’s half-Black, half-White. He was raised by the White side of him. I was raised by the White side, but stayed true to the Black side, honored the Black side.”

Throughout Moore’s Los Angeles dream home, custom art depicting notable Black figures like Malcolm X, Mohammad Ali and even Steph Curry deck the walls. “The art depicts my heart,” he says, “I am very much an African-American, but I’m also White and very proud of what comes with that. I honor African-American culture, the code, the cadence, the language, but I also stay in my lane and I don’t try to be something I’m not.”