Thank you!!! That worked. I actually had to add my number back to the account and re-activate Google voice mail on the phone first. Then I was able to choose the deactivate link, and it gave me the code to enter from my phone to deactivate.
For some reason, the first time I clicked deactivate it didn't give me any instructions, just showed the link as "activate google voice mail", which is why deleted the phone since that didn't work. But thankfully it worked this time :)