I was working on a HTB challenge recently, and I am positive that the first time I completed the machine a few months ago, SSH access out from the box was allowed. Seems something changed, or I dunno, I got lucky. Not sure. Either way when I was redoing the box yesterday to ensure that my writeup had all the details I may have skipped when putting together my personal notes, I found that I couldn't use Plink to SSH out to my Kali machine. This was very frustrating for a few minutes until I read around, and learned that it wasn't going to work because I couldn't connect out. I had to change the port. Actually, I didn't change the port, because I like to have SSH enabled on my default of 22. I just added a second one. Note that I have some aliases set up so that I can turn SSH on and off. I don't leave it running if I'm not actively using it at the moment.
Anyway, I just modified
Port 22 Port 2222
The above allows SSH to listen on both port 22 and port 2222. This was useful for this particular box. I leave port 22 set as well, because typically when using SSH, I want to connect to the default port. This was only useful in this case because of the aforementioned restriction.