The config.yaml file contains sensitive information and is purposely not placed where it can easily be found.
I am also a long time Windows user and a big fan of Notepad++. When you add a Unix/Linux/BSD based system you need to get comfortable with the minimum of tools that are regularly used in those environments. These are SSH (PuTTY on Windows works very well) and a text file editor on the Linux side. VI is the defacto standard but that involves a significant learning curve. Nano is a lightweight editor which by default, keeps its "help" on the screen at the bottom.
A slightly more advanced tool which I use is WinSCP. This is a file manager that can speak many file-oriented protocols. You can use the username "pi" and your SSH password to bring up a window into the home directory of the "pi" user. The only tricky part is that ~/.octoprint is a hidden file and won't show until you type CTRL+ALT+H or change the setting in the panels section of Options, Preferences.
After installing WinSCP and enabling display of hidden files, you can "download" config.yaml, edit it with Notepad++, and "upload" it back to your OctoPrint system.