Yes, they are quite different.
First, remember that "NounがAdjective" and "NounをVerb" are two very different animals.
「が」 sentences are pretty straight forward, but 「を」 means that when translating to English, you are "verb"ing the "noun" so you need to remember to mentally flip the order.
Second, keep in mind that the "adjective + する" format can generally be interpreted in two ways: "Cause the Noun to become Adjective", or "Verb in an Adjective way."
Now for your sentences.
First Sentence
「彼女が懐かしい」 itself is already somewhat awkward, as 「懐かしい」 isn't typically used to say "I miss her". It's more of a nostalgic feeling. I suppose it could be used in a fashion such as 「今と違っていつも明るかった彼女が懐かしい」. But for the sake of this question let's let it simply mean "I miss her". By saying 「彼女を懐かしくした」, you are saying either "I caused her to become nostalgic" or "I did (something) to her in a nostalgic way". Since there's no verb in the middle, the second interpretation can be eliminated, leaving only the first.
彼女が懐かしい:I miss her/I feel nostalgic about her
彼女を懐かしくした:I caused her to become nostalgic/I made her feel nostalgic
Second Sentence
「好きだ」 simply states that you like something, so that’s pretty straightforward. 「~を好きにする」, on the other hand, means to do something as you like, or to do whatever you like to something. There is no other alternative interpretation.
彼女が好きだ: I like her
彼女を好きにした: I did whatever I wanted to her
If you must change a "が+adjective" sentence using an を particle, try using "を+Verbている".
彼女がすきだ to 彼女を愛している