Rocco played Moe Greene, the casino boss who angrily resisted a takeover attempt from rising mafia boss Michael Corleone and paid for it with a bullet through his eye.
In a career spanning half a century, Rocco won an Emmy for his role in 1990s sitcom The Famous Teddy Z.
He also voiced the studio boss behind Itchy and Scratchy in The Simpsons.
More recently, he starred as Matt LeBlanc's curmudgeonly father on BBC Two's Episodes.
Born Alexander Federico Petricone in 1936 in Cambridge, Massachusetts, Rocco was - according to a US organised crime informant - a young hanger-on of the Boston area's Winter Hill Gang.
After he was arrested, but not charged, over a gang-related killing, he fled to California to work as a bartender.
Shortly after arriving in Los Angeles, he studied acting under Star Trek actor Leonard Nimoy, a fellow Massachusetts native who advised Rocco to lose his Boston accent. "I wouldn't spend five bucks to see Leonard in a film, but he was one of the greatest coaches I ever had," he later said.
Rocco made his debut in cult director Russ Meyer's 1965 film Motor Psycho.
Six years later, he won a part in what was to become regarded as one of the best films ever made.
He described his role as casino boss Moe Greene in The Godfather as "without a doubt, my biggest ticket anywhere" in an interview with the AV Club.
His character met a sticky end after a financial dispute with Michael Corleone, played by Al Pacino, which leads Rocco to utter the immortal phrase: "Don't you know who I am?' I'm Moe Greene!"
It was a line of dialogue which never left him. He later said: "People on the golf course will say, 'Hey, Alex, would you call my dad and leave a line from The Godfather?'
"I say, 'Okay. I buy you out, you don't buy me out!... Don't you know who I am?'
"But I enjoy doing it. It's fun. I've been leaving Moe Greene messages for 40 years. People's dads, girlfriends, whoever."
Frequently cast as gangsters, his notable roles included appearances in The Friends of Eddie Coyle and Get Shorty. He also had a part in 1980s sitcom The Facts of Life.
He said in a 2011 interview: "Playing gangsters is great. They usually dress you sharp. And you have a license to pretty much bully anybody. I mean, I wouldn't dare to that at home. My wife will give me a back hander."
