Federal prosecutors said the arrest warrant for the 31-year-old suspect, identified only as Akram A., on charges of membership in a terrorist organisation and a war crime had been lifted.
He was detained in the northeastern state of Mecklenburg-Western Pomerania in February.
Prosecutors at the time alleged that he manned a checkpoint in Syria that sought to prevent citizens from escaping Isis-controlled territory.
In early 2016, he was believed to have stopped a woman who was trying to flee with her children, they said.
"Under the pretext of obtaining the necessary 'exit permit', the accused lured her to a house and raped her," prosecutors said in the statement at the time, adding that the rape could amount to a war crime.
But in a statement Monday, the federal prosecutor's office said that "further investigation" had determined that "another course of events may have been possible".
For this reason, there was no longer enough evidence to keep the suspect in custody.
German federal prosecutors have opened about a dozen investigations concerning alleged war crimes in Syria or Iraq, alongside dozens of cases of suspected membership in jihadist groups.
The investigations have gained momentum with the arrival of more than one million asylum seekers since 2015, including hundreds of thousands from Syria and Iraq.
Last July, in the first such conviction in Germany, a German jihadist was sentenced to two years in prison on war crimes charges after posing for pictures in Syria with the severed and impaled heads of two government soldiers.