Join the army, meet interesting people, and kill them.
Yes, you can change from one client to another whenever you want. The clients just control your working copy and the interaction between your working copy and the repository. The metadata inside the working copies used by the different clients is identical.
But you can only use different clients if they all use the same version of the Subversion library. The version of the Subversion library that TortoiseSVN uses is indicated in the filename of the installer, other clients have similar indications. You have to make sure that those versions match each other in the first two digits. For example, all clients using Subversion 1.6.x can be used together (the 'x' indicates that this number is not relevant for compatibility).
You must also be sure that all the clients are built for the same OS. Client compatibility is only guaranteed for a particular OS type and metadata representations may differ. You must not use a native Windows client and the Cygwin client on the same working copy. And if you share a working copy over a network you must not use a Linux and a Windows client on the same working copy.