[SOLVED] Python pysftp get_r from Linux works fine on Linux but not on Windows

Issue

I would like to copy an entire directory structure with files and subfolders recursively using SFTP from a Linux server to a local machine (both Windows and Linux) using Python 2.7.

I am able to ping the server and download the files using WinSCP from the same machine.

I tried the following code, works fine on Linux but not on Windows.

I tried \, /, os.join, all gives me same error, checked permissions as well.

import os
import pysftp

cnopts = pysftp.CnOpts()
cnopts.hostkeys = None    # disable host key checking.
sftp=pysftp.Connection('xxxx.xxx.com', username='xxx', password='xxx', cnopts=cnopts)
sftp.get_r('/abc/def/ghi/klm/mno', 'C:\pqr', preserve_mtime=False)

File "<stdin>", line 1, in <module> File "C:\Python27\lib\site-packages\pysftp_init_.py", line 311, in get_r preserve_mtime=preserve_mtime)
File "C:\Python27\lib\site-packages\pysftp_init_.py", line 249, in get self._sftp.get(remotepath, localpath, callback=callback)
File "C:\Python27\lib\site-packages\paramiko\sftp_client.py", line 769, in get with open(localpath, 'wb') as fl: IOError: [Errno 2] No such file or directory: u'C:\\pqr\\./abc/def/ghi/klm/mno/.nfs0000000615c569f500000004' 

Solution

Indeed, pysftp get_r does not work on Windows. It uses os.sep and os.path functions for remote SFTP paths, what is wrong, as SFTP paths always use a forward slash.

But you can easily implement a portable replacement.

import os
from stat import S_ISDIR, S_ISREG

def get_r_portable(sftp, remotedir, localdir, preserve_mtime=False):
    for entry in sftp.listdir_attr(remotedir):
        remotepath = remotedir + "/" + entry.filename
        localpath = os.path.join(localdir, entry.filename)
        mode = entry.st_mode
        if S_ISDIR(mode):
            try:
                os.mkdir(localpath)
            except OSError:     
                pass
            get_r_portable(sftp, remotepath, localpath, preserve_mtime)
        elif S_ISREG(mode):
            sftp.get(remotepath, localpath, preserve_mtime=preserve_mtime)

Use it like:

get_r_portable(sftp, '/abc/def/ghi/klm/mno', 'C:\\pqr', preserve_mtime=False) 

---

Note that the above code can be easily modified to work with Paramiko directly, in case you do not want to use pysftp. The Paramiko SFTPClient class also has the listdir_attr and get methods. The only difference is that the Paramiko's get does not have the preserve_mtime parameter/functionality (but it can be implemented easily, if you need it).

---

Possible modifications of the code:

• Do not download empty folders while downloading from SFTP server using Python
• Download files from SFTP server that are older than 5 days using Python
• How to sync only the changed files from the remote directory using pysftp?

---

For a similar question about put_r, see:
Python pysftp put_r does not work on Windows

---

Side note: Do not "disable host key checking". You are losing a protection against MITM attacks.

For a correct solution, see Verify host key with pysftp.

Answered By - Martin Prikryl