The accident occurred below the Mannlkarscharte pass, about 80 kilometres (50 miles) south of the central city of Salzburg, when a rope came loose after one of the group skidded while climbing on a glacier at an altitude of 3,000 meters.
Witnesses from another group said two of the climbers "tried to prevent the fall by using ice picks, but to no avail," said local mountain rescue chief Franz Gensbichler.
Five helicopters were sent to the scene, and the surviving climber was transported to a hospital in Schwarzach where he was said to be in serious condition.
The victims have not yet been identified.
It was the deadliest climbing accident so far this season in the Austrian Alps, where several dozen mountaineering deaths are reported each summer.