Versions can authenticate svn+ssh:// connections both by using your password —which Versions stores in your Keychain— or by using ssh key-pair authentication. If you also use the svn command-line client, chances are you already have an ssh key-pair set up. To make Versions authenticate with your private key, simply enter your username in your repository bookmark and leave the password field blank.
If you want to create a bookmark to a repository with a custom port number, there are two ways you can do this, depending on what kind of protocol you use.
For http and https repositories, enter a colon followed by the port number at the end of the first part of the URL, like you would in a web browser. For example type http://svn.webkit.org:80/repository/webkit in the Location field for a repository bookmark to access the public Webkit repository at port 80.
If you use the svn+ssh protocol, there are two ways you can use custom port numbers.
One is by editing your Subversion configuration and creating a new tunnel with a custom port number. This is demonstrated in the SVN book, where a 'joessh' tunnel is created that connects to port 29934.
The other way is to edit the .ssh/config file and creating a host with a custom port. An example of this can be seen on the SVN users list.