“Everybody’s coming to Southern California and we’re gonna do a lot of cooking and a lot of eating,” John Legend says of his upcoming holiday plans

John Legend was named PEOPLE’s Sexiest Man Alive last week, and he’s already feeling the weight of the crown.

“It’s been fun. It’s a lot of pressure because that title is very heavy,” he told PEOPLE before hitting the stage to help launch Sirius XM’s Dial Up the Moment campaign in Los Angeles. “You know, it’s like Sexiest Man Alive. Obviously you’re not supposed to take it literally and think you’re the absolute sexiest man alive, but it’s a lot to live up to.”

“I’m also following Idris Elba, which makes it even harder to live up to,” he adds. “But I’m having fun with it, and it’s really nice of PEOPLE to give me the recognition. It’s been a fun week.”

Elba, 47, who was PEOPLE’s Sexiest Man Alive last year, previously gave Legend, 40, some advice on how to cope with his status as the reigning man. “Just enjoy it. Try not to be sexy, just be you,” the actor said. “One thing I learned very quickly: I don’t have to try to be sexy, I am sexy.”

EMMA MCINTYRE/GETTY
John Legend Is PEOPLE’s Sexiest Man Alive 2019: ‘I’m Excited but a Little Scared at the Same Time’

The singer’s wife, Chrissy Teigen, 33, is also enjoying the moment. When the news first broke, the model hilariously shared her thoughts on Twitter by writing, “My secret is out. I have fulfilled my dream of having boned @people‘s sexiest man alive!! an honor!!!!!”

“She’s been having fun with it,” Legend said of his wife’s reaction to his latest honor. “She was genuinely happy for me. She knew it would be a mixed bag whenever you get this title because there’s always haters that’ll come up. She’s had fun with both sides, the good and the bad.”

Blake Shelton, who held the title in 2017, gave his fellow Voice coach some advice on being the Sexiest Man Alive: Stay off of social media. “Everybody that ever hated you is gonna have something to say about [the honor],” he warned.

Shelton, 43, announced Legend’s big news during an episode of The Voice last Tuesday. He even presented the honoree with a customized pair of bedazzled shorts with the acronym “VEGOTSMA” — which stands for Legend’s ultra-impressive Voice, Emmy, Grammy, Oscar, Tony wins, plus this latest honor — plastered along the backside in silver sparkles

JOHN LEGEND/INSTAGRAM

“We had so much fun with Blake,” Legend says. “He was that person in 2017, and our friend Adam Levine has also been Sexiest Man Alive. We joked about how I was the only guy on the stage last season who hadn’t been called Sexiest Man Alive and here we are. I’m in the elite fraternity now and I got some daisy-dukes from Blake.”

The short-shorts have yet to be worn by Legend (or Teigen, and definitely not daughter Luna, 3, or son Miles, 18 months). “No one’s tried them on yet,” he says. “They’re hanging in my closet. We like to keep like a memorabilia stash for future purposes. Maybe we’ll do some kind of auction in the future. Who knows how we’ll use it.”

TRAE PATTON/NBC
Watch Chrissy Teigen Hilariously Scare John Legend as He Guest Hosts The Ellen DeGeneres Show

As for the holidays, Legend is looking forward to having family over for Thanksgiving. “Everybody’s coming to Southern California and we’re gonna do a lot of cooking and a lot of eating and we’re gonna have a good ol’ time.”

John Legend and Chrissy Teigen. CARLOS ERIC LOPEZ

They’ll be celebrating Teigen’s 34th birthday during the same weekend, though he says she wants to be low-key this year. “I don’t know if she wants to do a big thing. Everybody’s already gonna be together so we’ll make sure we celebrate her well.”

John Legend. EMMA MCINTYRE/GETTY

For Christmas, the family is heading to Wyoming to enjoy some snowy weather.

“We decided to go to the snow since we live in LA and it doesn’t feel Christmassy here in December no matter what time of year it is. So we decided we’re gonna go up to Wyoming and be around the snow.”

Jason Aldean will hit the stage in Nashville for the next Sirius XM Dial Up The Moment performance this Friday.