Generally, you're placing .xtbl files in the game's base directory, where the .exe files are. If it's a .sr3mod file that you've downloaded, you need to use the mod installer tool, which at this moment is not compatible with the current version of sr3 and is being updated.