In two landmark reports delivered on Wednesday, the taskforce urged the government to hold a Royal Commission into some of the most severe cases – a spate of rapes and sexual abuse episodes at the Australian Defence Force Academy.

Defence Minister David Johnston said the government would consider "very carefully" the call for a Royal Commission – even as his own job fell under a cloud following his widely condemned attack on the government's shipbuilding firm.

Prime Minister Tony Abbott stuck by Senator Johnston during question time in Parliament on Wednesday, but Coalition sources have said that the Defence Minister is unlikely to survive an expected cabinet reshuffle early next year.

The Defence Abuse Response Taskforce stated it was aware of at least 36 women who said they had been raped or sexually abused at ADFA between 1991 and 1998, many while still in their teens.

And it was aware of at least 13 alleged perpetrators of sexual abuse at ADFA in the 1990s who are still serving with the military and another three who are standby reservists.

Across the whole of the Australian Defence Force, 594 people still serving in the permanent force, or within the Defence Department, are alleged to have abused their comrades, the taskforce found.

Another 204 people in the active reserves, and 341 in the standby reserves, are also alleged abusers.

But the chairman of the taskforce, retired Western Australian Judge Len Roberts-Smith, expressed some pessimism about what action could or would ever be taken against alleged perpetrators.

Mr Roberts-Smith said Defence had so badly mismanaged the cases at the time it was very difficult now for senior officers to do anything about them now, given no records or evidence were kept.

"There needs to be evidence and in most of these cases, because they were mismanaged at the time, there is very little evidence," he said.

"The end result is it seems fairly clear there are likely to be in Defence at the moment middle-ranked to senior serving officers who were sexual or physical abusers, or are alleged to have been, and there appears to be nothing that Defence is actually able to do, or in fact anyone else."

He said the taskforce had sent many cases to civilian police, but "I wouldn't be all that optimistic that a prosecution even is likely".

The ADFA report also states the Chief of the Defence Force, Mark Binskin, wrote to the taskforce in September saying a five-year statute of limitations on the Defence Force Discipline Act meant that the ADF was unable to act on 19 cases dating from the 1990s that the taskforce had referred to Defence.

Mr Roberts-Smith stressed he wasn't blaming the recent and current Defence leadership, who he said had taken strides in preventing abuse and accepting the problems of the past.

But he said a Royal Commission was now the only way to get some closure for at least the ADFA sexual cases, which are considered among the most serious abuse cases. The taskforce is not at this stage recommending a broader Royal Commission for abuse right across the ADF.

Mr Roberts-Smith said at the heart of the issue was that abused service people felt let down by an organisation in which they were "always taught to watch each other's backs and to protect each other".

"We're talking about the Defence Force – Army, Navy and Air Force. We're talking about people ultimately who have to look after each other because they're going into battle and their lives depend upon each other.

"They see it as a betrayal by their family – their service family."

Air Chief Marshal Binskin said the ADF remained "committed to action against alleged abusers".

But he said the ADF also had a responsibility not to do further harm to complainants. Many alleged victims have made it clear to the taskforce they did not want to formally pursue any further action.

Air Chief Marshal Binskin said Defence was also "bound by the application of legal and administrative standards of proof and requirements of procedural fairness".