The Home Office will also waive fees for those needing to prove their residency rights, she said.
International Development Secretary Penny Mordaunt told BBC Radio 4's Today programme: 'People who are in that situation, there is absolutely no question of their right to remain, and their right to gain access to services such as healthcare.
It has seen some Windrush generation residents, who might never have felt the need to apply for a United Kingdom passport before, left without the documentation now required by officials.
However, the minister could not give an estimate for the number of people affected.
Rudd said: "I do not want any of the Commonwealth citizens who are here legally to be impacted in the way they have and frankly, some of the ways they have been treated is wrong, has been appalling, and I am sorry".
There is widespread anger that long-term British residents have fallen victim to rule changes in 2012 - when May herself was interior minister - aimed at stopping overstaying.
The Home Secretary said: "I wouldn't want anyone who has made their life in the United Kingdom to feel unwelcome or be in any doubt of their right to remain here".
It comes as Commonwealth leaders gathered for a summit in London.
"It is inhumane and cruel for so many of that Windrush generation to have suffered so long in this condition and for the Secretary of State only to have made a statement today on this issue".
Jamaica remained a British colony until 1960.
When the mother country sent out a call to the Caribbean after World War II thousands of people answered and came to Britain to staff our NHS.
Named the Windrush generation after British ship the Empire Windrush, which arrived at Tilbury Docks in Essex with 492 Caribbean passengers in 1948, "many have made the United Kingdom their home for their entire lives", says the Channel 4 News website.
British media have reported cases such as a man who was denied treatment for cancer and a special needs teaching assistant who lost his job after being accused of being illegal immigrants despite living in Britain for more than 40 years.
It is estimated as many as 50,000 Windrush generation residents are facing problems, with the row flaring up as Caribbean leaders arrived in London for the start of the Commonwealth Heads of Government Meeting (CHOGM) this week.
May only became aware of a request for a meeting on Monday morning, and will discuss the issue with counterparts from Caribbean nations this week, her spokesman said.
The Home Office's renewed guidance, published last week, offers no security or certainty.
Under the 1971 Immigration Act, all Commonwealth citizens already living in the United Kingdom were given indefinite leave to remain - but the right to free movement between Commonwealth nations was ended from that date onwards.
But the Home Office did not keep a record of those granted leave to remain or issue any paperwork confirming it, meaning it is hard for the individuals to now prove they are in Britain legally.