I understand that に can be used to mark the agent of an intransitive verb, but I often see it being used with adjectives instead. For example, in the sentence:
As for me, I can do it.
に here marks the agent of the verb. The same thing can be said about 私にはわかる. However, there are cases where に are followed by adjectives, which, although I intuitively understand the meaning, do not understand the grammar behind them.
As for me, I need money.
It's too early to sleep.
to know a lot about trains.
It's good for your body.
Scary to children. (apparently incorrect.)
What is the function of に（は） here? In what situations can this be used?
Edit: I found a related question, and this linked one, but the first one seems to have conflicts about in which situations you can do this and the second question only uses に to mark the agent of intransitive/potential verbs. This question also asks something very similar, but the answer doesn't really address why 「子供に怖い」would be wrong.