I sent him the letter - 彼に手紙を送ってあげた
He was the one who sent me the letter - 手紙を送ってくれたのは、彼だった
He was the one I sent the letter to - ???
You can say 私が手紙を送ったのは彼だった.
This 私が is more or less important. If you omitted 私が, the sentence would become ambiguous:
手紙を送ったのは彼だった。
- He is the one who sent a letter (to someone).
- He is the one I sent a letter to.
The use of あげる cannot solve this type of ambiguity (手紙を送ってあげたのは彼だった is still ambiguous the same way). And even the following simple phrase is ambiguous in Japanese:
手紙を送った人
- a person who sent a letter (to someone) (=the sender)
- a person who someone sent a letter to (=the receiver)
Please read the following questions for details: