"I am sure the bill will take a different form when it is tabled on the floor in parliament," Mr Oryem told the BBC's Network Africa programme.
However, he also pointed out that it was a Private Member's Bill and so the government did not have the powers to alter it at this stage.
"Homosexuality is not a top priority for the people of Uganda," he deputy minister said.
"Our priority is to make sure there is food on the table of our people - that we deal with the issue of disease."
Homosexual acts are already illegal in Uganda and punishable by up to 14 years in prison.
The bill would raise that penalty to life in prison.
It also proposes the death penalty for a new offence of "aggravated homosexuality" - defined as when one of the participants is a minor, HIV-positive, disabled or a "serial offender".
Uganda's President Yoweri Museveni has already distanced himself from the bill, saying it did not represent the views of his government.
Two weeks ago its sponsor, David Bahati, told a Ugandan newspaper he was willing to "amend some clauses".
The cabinet has set up a committee to look at his proposals.
Advertisement
Controversy over homosexuality is not new in Uganda.
Bookmark with:
What are these?