Jimmy Fallon Admits This 1 Thing Is His ‘Biggest Fear’ Tied to His Career

Comedian Jimmy Fallon got serious about his biggest fears.

On the Thursday, Oct. 31 episode of Hot Ones, the comedian, 50, sat down with host Sean Evans ahead of the Nov. 1 release of his comedy album, Holiday Seasoning.

Though Fallon may have a successful career as a late-night talk show host and actor, he says there’s still one thing that scares him more than anything when it comes to his job. 

“As a comedian, silence,” Fallon confessed. “The deafening silence is my biggest fear, crying on camera which is happening now, my biggest fear. Probably my biggest fear is giving up in front of the American public and I’m not doing that.”

The Tonight Show Starring Jimmy Fallon star also shared his fondest moment from his 1998 to 2004 run on Saturday Night Live. 

“’Cowbell’ — it just crushed, I mean next level,” he said of the classic sketch that aired on April 8, 2000. 

The skit, titled “More Cowbell” featured Fallon, Christopher Walken, Will Ferrell, Chris Kattan, Chris Parnell and Horatio Sanz portraying rock band Blue Öyster Cult during a recording session. 

“Everyone was so funny in that sketch,” Fallon told Evans, 38. “Christopher Walken wasn’t even talking like a human being, he was doing like an impression of himself.”

Fallon also recalled how one the the scene’s iconic moments came about.

“I remember I had one line and right as I’m about to give my line, Kattan and Will improvised and pushed each other,” he said. “Kattan knocked Will’s sunglasses off and I could see the eyes of a lunatic and I started laughing, and then that’s what got my reputation as the guy who laughs during all the sketches. That was great, Debby Downer, I was a part of that sketch. I was really solid on that until the way, way, way end.” 

Throughout the years, Fallon has worked on many comedy sketches with big names. 

During his visit to Hot Ones, the Fever Pitch actor also shared the time when he came to realize just how seriously Justin Timberlake takes his craft. 

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories.

“He’s one of the most talented human beings on earth, he’s a perfectionist,” Fallon said of the “Selfish” singer. 

“I remember we were doing SNL, and we’re doing the Barry Gibb talk show, so we’re about to go, like 20 seconds to air, and he goes, ‘Hey Jimmy remember, hit that harmony,’ and I go, ‘Yeah, I will,’ and he squeezed my hand and goes, ‘I’m serious dude, remember to hit that harmony.’ And I’m like calm down dude, we’re dressed like the Bee Gees, for that second I felt what it was like to be Lance Bass,” Fallon quipped, referring to Timberlake’s *NSYNC bandmate.