Yes, there are several movies where the character's reality turns out to be an illusion or a delusion, and they are revealed to be insane or mentally unstable in the end. Here are some notable examples:
"Fight Club" (1999) - Directed by David Fincher, this movie follows an unnamed narrator (played by Edward Norton) who suffers from insomnia and creates an alter ego named Tyler Durden (played by Brad Pitt). As the story unfolds, it becomes clear that Tyler Durden is a figment of the narrator's imagination, and he suffers from dissociative identity disorder.
"Shutter Island" (2010) - Directed by Martin Scorsese, the film revolves around a U.S. Marshal named Teddy Daniels (played by Leonardo DiCaprio) who is investigating the disappearance of a patient from a mental hospital on Shutter Island. As the investigation progresses, Teddy starts to question his own sanity, and the truth behind his presence on the island is gradually revealed.
"A Beautiful Mind" (2001) - Directed by Ron Howard, this biographical drama tells the story of mathematician John Nash (played by Russell Crowe), who struggles with schizophrenia. The movie blurs the line between reality and Nash's delusional world, making it difficult for the audience to distinguish between the two.
"The Machinist" (2004) - Directed by Brad Anderson, the film follows Trevor Reznik (played by Christian Bale), an industrial worker who hasn't slept in a year and starts experiencing delusions and hallucinations. The movie keeps the audience guessing about what is real and what is a product of Trevor's deteriorating mental state.
"Black Swan" (2010) - Directed by Darren Aronofsky, this psychological thriller revolves around a ballerina named Nina Sayers (played by Natalie Portman) who struggles with her sanity as she prepares for the lead role in "Swan Lake." The film blurs the boundaries between reality and hallucination as Nina's mental state becomes increasingly unstable.
These are just a few examples, but there are many other movies with similar themes of characters facing delusions or mental instability, leading to shocking twists and revelations in the end.