The Tulare County Sheriff's Office says it happened around 2 pm at the Friant Kern Canal. Deputies found out that the man went under after falling in.
The sheriff's office's dive team and a helicopter were also part of the rescue mission.
Deputies found the man's body in the area he was last seen going underwater.
He was pronounced dead at the scene.
The identity of the man has not been released.