My path was similar. I started by powering the Pi with the official PSU and only connecting TX, RX, 22 and 23. That didn't work. So then I experimented with everything in between.
One of my experiments was to unhook the official PSU and power the Pi4 exactly like the Pi Zero using the 5v header. That worked, but still was not connecting to the serial connection.
What I now know as I have it working is:
I must have the GND connected or else I get garbage on the Prusa screen
I have to manually define the serial connection /dev/ttyAMA0
The fan that came with my Pi4 sounds like a screaming child unless I undervolt it
The case I printed for the Pi4 sucks
That said, I'm looking good right now using the beta code and the Pi4 8GB.
That sounds like headway! I have been experimenting with different fan solutions and thus far have stuck to Noctua 5v connected to the usb port on the pi. I've ordered some temperature-based pwm fan controllers made with Chinesium so the jury is out there. If you use a level converter on a pwm port of the pi, you can mitigate the noise quite a bit. I've been using James Cameron's folding "case" for lots of mine. see (https://www.youtube.com/watch?v=ImdL78Qor8Y) where he is building pi cases 26 ways.
I did the same thing in my case, but it's a little special. I have the Pi 4 mounted to one of those finned aluminum passive cooling armor cases and then my case has the noctua mounted just below the HyperPixel. I ended up splitting the 5V and ground off of one the header risers going to the HyperPixel with their quick connect.
