The decision is a loss for President Trump's administration, which has emphasized stricter enforcement of immigration law.
After the conservative Gorsuch was appointed to the court, the justices heard the case re-argued, and Gorsuch joined the court's more liberal justices in finding the clause too vague. "The truth is, no one knows", he said.
The appeals court relied on a decision that same year by the U.S. Supreme Court, which found that a similar provision in a federal criminal sentencing law was overly broad.
James Garcia Dimaya, a native of the Philippines, was admitted to the United States in 1992 when he was 13 as a lawful permanent resident.
James Comey describes Donald Trump as a 'stain' in long-awaited TV interview
He considered that "the most important thing" in a president is that he tell the truth and that Trump "is not qualified for that". The ex-FBI chief's comments in an ABC News interview are nearly certain to escalate his war of words with the president.
Justice Elena Kagan praised the ruling in her official opinion on the case, saying a conviction of a loosely-defined "violent crime" would result in the "virtual certainty" of deportation.
Federal authorities ordered Dimaya deported after he was convicted in two California home burglaries, in 2007 and 2009, though neither crime involved violence.
A federal appeals court in San Francisco previously struck down the provision as too vague, and on Monday the Supreme Court agreed. The Supreme Court affirmed that ruling Tuesday. The government argued among other things that he could be removed from the country because his convictions qualified as crimes of violence that allowed his removal under immigration law. The ruling is limited to a category of crimes that carry a prison term of more than a year, but do not otherwise comfortably fit in a long list of "aggravated felonies" that can lead to deportation. While conservatives may be frustrated by Gorsuch's vote, the court supposedly used the late conservative Justice Antonin Scalia's reasoning in their opinion. Deadlocked 4-4, the justices scheduled a new round of arguments once Gorsuch joined the court.
Most of those focus on ousting illegal immigrants who never had permission to be in the USA, while Tuesday's case deals with legal permanent residents who committed crimes while here, and whether those are serious enough to deserve automatic deportation.