What purpose does に serve in set phrases that go after verbs?

Question

～するに決まっている

～するに留まった

～するにして・にしろ・にしては,etc

other than this usage you don't see に after verbs. The entire に＿＿＿　is explained as grammar point as a whole.