Context: girl breaks up with guy, he gets sad and says this line
私だけがあなたの特別でいられたのに!
why is the passive/potential form used here? is it possible to use only the past simple with た here? I'm not sure how to make any sense out of the sentence in the passive/potential form