I'm still learning Japanese but I want to start writing anyway, I got a normal notebook with horizontal lines but I would like to write vertically so I just rotated it, my question is with this setup, which side of the notebook should I start writing from?
進研ゼミ explains how Japanese high school students typically use normal (i.e., having horizontal rules) notebooks for vertical writing:
(This is a notebook for 古文, or classical Japanese classes)
So you can just rotate your normal notebook 90 degrees clockwise. With this orientation, the page order remains unchanged, and the front page of a notebook will be the same. The first line comes at the right.
Elementary school children usually use a dedicated notebook for vertical writing. They are bound the same way normal Japanese novels are bound. ジャポニカ is a famous notebook brand for elementary school children, and if you look at their website carefully, you can see a different side of a notebook is bound depending on the subject.
In case you're totally new to vertical writing, you can easily find resources for that, including this.
The vertical writing style based from traditional Japanese writing named "縦書き" (tategaki) or "縦組み" (tategumi). It starts from top-rightmost column, writing characters vertically in right-to-left column order and ended at bottom-leftmost column.
Definition from Wikipedia:
Traditionally, Japanese is written in a format called tategaki, which is inspired by the traditional Chinese system. In this format, the characters are written in columns going from top to bottom, with columns ordered from right to left. After reaching the bottom of each column, the reader continues at the top of the column to the left of the current one.