Rsync a Shared Folder
How to keep a folder in sync between Linux and Windows when the two are already joined by an SSH tunnel, without opening a new way into the Windows machine.
The default pairing is the Windows Downloads folder against a shared folder on Linux, but any two folders work.
Which Machine Runs the Sync
Work this out first, because it decides everything else.
In the SOCKS setup, Windows is the SSH client and Linux is the SSH server. Windows dials out, and a RemoteForward puts the SOCKS proxy on the Linux side.
Windows box Linux box
SSH client SSH server
Cygwin + rsync ------- ssh -------> login account, shared folder
<-- RemoteForward ---
SOCKS, port 4020
Linux cannot start the transfer, for two reasons.
The Windows firewall drops inbound connections, so there is no route to a Windows SSH server even with both machines sitting on the same LAN.
The SOCKS proxy looks like it should cover the gap, and it does not. It carries traffic out of the Windows box into the network Windows can see. That is a path to the remote network, not a path back into the Windows disk.
Both are easy to verify from the Linux side. A healthy proxy reaches the outside world:
curl -sS --socks5-hostname 127.0.0.1:4020 -o /dev/null -w '%{http_code}\n' http://example.com/
Point that same proxy back at the Windows machine's own SSH port and it fails, because nothing is listening there:
ncat --proxy 127.0.0.1:4020 --proxy-type socks5 127.0.0.1 22 < /dev/null
So the sync runs on Windows, in the direction that already connects. Both scripts below live in scripts/.
You could instead install OpenSSH Server on Windows and add a RemoteForward 2222 127.0.0.1:22 so Linux can reach back through the tunnel. That works. It also needs an administrative install and leaves a listening service running afterwards. Driving the sync from Windows needs neither.
Linux Side: the Shared Folder
Windows logs in over SSH as its own account, so everything it copies over lands owned by that account. Put the shared folder inside that account's home and your normal user cannot read it. The fix is to keep the folder in your own home and hand the SSH account access with POSIX ACLs:
./scripts/setup-linux-share.sh --peer <account>
<account> is the local account the Windows box logs in as. If you are not sure which one that is, run who on the Linux box while the tunnel is up.
That creates ~/shared and sets two ACLs:
| Path | ACL | Effect |
|---|---|---|
$HOME | u:<peer>:rx | Reach and list $HOME, but not read anything inside it. |
$HOME/shared | u:<peer>:rwx | Full access, with a default ACL so new files inherit it. |
The r on $HOME looks like more than the job needs, and it is the part worth
understanding, because getting it wrong produces a failure that points somewhere
else entirely.
Traverse alone (u:<peer>:x) is enough to cd into the share and write a file
there. It is not enough for rsync, whose receiver also reads the destination's
parent directory. Grant only x and you get this, with the share itself plainly
writable:
rsync: [Receiver] change_dir#1 "/home/you/shared/" failed: Permission denied (13)
rsync error: errors selecting input/output files, dirs (code 3)
The direction split is the tell. Pulling keeps working while pushing fails, because only the receiving side reads that parent. It is easy to read that as a problem with the share, or with the account, and spend a while there.
The peer can now list the names of things in your home directory. It cannot read them, since your files keep their own permissions:
$ ls -ld ~/.ssh ~/.bash_history
drwx------. /home/you/.ssh
-rw-------. /home/you/.bash_history
If names alone are more than you want to share, put the folder outside your home
instead, say /srv/shared, and nothing about $HOME has to change:
./scripts/setup-linux-share.sh --peer <account> --dir /srv/shared --traverse-only
--undo withdraws the access later. Then check it:
getfacl -p ~ ~/shared
sudo -u <peer> test -w ~/shared && echo writable
Windows Side: the Sync Script
Cygwin already has what you need if you installed the package list. The two that matter are openssh and rsync.
Copy the script down from Linux, since Windows is the side that can connect:
scp <linux-host>:~/code/self/ssh-proxy-traffic/scripts/sync-folder.sh ~/
chmod +x ~/sync-folder.sh
Telling It About Your Machines
Nothing host specific is committed to this repository, so the first run needs to learn where your Linux box is. Either write the config file from the script, or copy the example next to it:
~/sync-folder.sh --init-config # ~/.config/sync-folder.conf
cp sync-folder.conf.example sync-folder.conf # or keep it beside the script
--init-config writes a blank template. It does not adopt a config you already
have, and it refuses to run at all when one exists anywhere on the search path,
because writing a blank file at a higher-precedence location would shadow the
one you filled in.
Fill in the two blanks:
SYNC_LINUX_HOST=my-linux-box
SYNC_LINUX_DIR=/home/YOUR_USER/shared
Four locations are searched, and the first file that actually sets something wins:
~/.config/sync-folder.conf
~/.sync-folder.conf
<folder holding the script>/sync-folder.conf
./sync-folder.conf
A file that sets nothing is skipped rather than allowed to win, so a leftover blank template cannot hide a real config further down the list.
If a setting does not turn up where you expect, ask the script rather than
guessing. --show-config prints every path it searched, marks the file it
actually read, and names the source of each value:
$ ~/sync-folder.sh --show-config
config search path, first readable wins:
empty /home/me/.config/sync-folder.conf (sets nothing, skipped)
absent /home/me/.sync-folder.conf
USED /home/me/sync-folder.conf
absent /home/me/work/sync-folder.conf
resolved settings:
linux host my-linux-box config file
linux folder /home/YOUR_USER/shared config file
windows folder %USERPROFILE%\Downloads not set
The file is parsed rather than sourced, so KEY = "value" with spaces, a
leading export, and the CRLF line endings a Windows editor leaves behind all
work. That last one matters: sourcing a CRLF file gives you a hostname with a
carriage return stuck on the end, and the failure it causes points nowhere near
the config file.
For SYNC_LINUX_HOST, a Host alias from your Windows ~/.ssh/config beats a
raw address. The alias already carries the user, port and key, so the script
inherits whatever you configured for the tunnel and there is no second place to
keep it in step.
You can skip the file and pass --host and --linux-dir every run, or export
SYNC_LINUX_HOST and SYNC_LINUX_DIR instead. Command line wins over
environment, environment wins over the file. Then check the plumbing before
moving any data:
~/sync-folder.sh --check
Choosing the Folders
Downloads is only the default. Either side can point anywhere:
# a shared folder on the Windows side instead of Downloads
~/sync-folder.sh -w 'C:\Users\me\Shared' -l /home/YOUR_USER/shared
# Cygwin spelling works too, if you prefer it
~/sync-folder.sh -w /cygdrive/c/Users/me/Shared
Windows paths and Cygwin paths are both accepted; the script runs cygpath on anything that looks like C:\.... The Linux path has to be absolute. A bare ~ would expand to the SSH login account's home rather than the shared folder, which is rarely what you want, so the script rejects it.
The lock file is keyed on the folder pair, so you can sync two different pairs at the same time without them tripping over each other.
Preview Before You Sync
--preview dry-runs both directions and summarises them. It writes nothing:
$ ~/sync-folder.sh --preview
PREVIEW - nothing will be written
windows : /cygdrive/c/Users/me/Downloads
linux : my-linux-box:/home/YOUR_USER/shared
down (linux -> windows): 3 new, 1 to overwrite, 1 new folder, 195.4K
new bigfile.iso 195.3K
upd conflict.txt 12B
new report.pdf 52B
new notes/meeting.md 8B
up (windows -> linux): 1 new, 0 to overwrite, 13B
new installer.exe 13B
"upd" means the copy on the destination is older and would be replaced.
Nothing is deleted on either side, in any mode.
Long lists get truncated after twelve entries per direction; raise SYNC_PREVIEW_LINES to see more.
--dry-run is the other option, and it is rsync's own itemised output rather than a digest. Use it when the summary says something surprising and you want the raw >f.st.... codes to explain why.
When it all looks right, drop the flag:
~/sync-folder.sh # both directions
~/sync-folder.sh --up # Windows to Linux only
~/sync-folder.sh --down # Linux to Windows only
What the Flags Are For
Rsync's usual -a is wrong here, and most of what replaces it exists to work around a Windows filesystem.
| Option | Reason |
|---|---|
-rt instead of -a | NTFS through Cygwin has no POSIX ownership worth preserving. -a implies -pog, which errors on every file. |
--no-perms --no-owner --no-group | Stops rsync reproducing Linux ownership on NTFS, and the reverse. |
--modify-window=1 | The two sides disagree about timestamps below one second. Without this, the same files copy again every run. |
--update | Never replace a file with an older copy. This is what makes running both directions safe. |
--protect-args | Stops the remote shell re-splitting paths that contain spaces. |
--partial --partial-dir | A dropped tunnel resumes instead of starting the transfer over. |
-l --safe-links | Windows needs a privilege to create symlinks. Set SYNC_COPY_LINKS=1 to send the target file instead. |
--chmod=D775,F664 | Forces sane modes on arrival. Without it the ACLs on the Linux share stop working; see below. |
--exclude list | Keeps desktop.ini, Thumbs.db, $RECYCLE.BIN and half-finished browser downloads out of the transfer. |
Why --chmod Is Not Optional
Files arriving from Windows are owned by the SSH account, not by you, so your claim on them comes from the ACL rather than from ownership. That claim is easy to lose.
Cygwin reports files on NTFS as mode 0700. Copy that mode to Linux and the
zero group bits force the file's POSIX ACL mask to ---. The mask caps
every named ACL entry, so the entries survive in name only:
$ getfacl boo.txt
user::rwx
user:gfe:rwx #effective:---
group::r-x #effective:---
mask::---
other::---
Everything is granted and nothing is effective. The owner of the share cannot
read what landed in it without sudo, which is a strange enough symptom to send
you looking at the ACLs, where all the right entries are sitting in plain sight.
--chmod=D775,F664 sets the mode at creation instead of inheriting a meaningless
one, the mask stays rw-, and the named entries work. setup-linux-share.sh
also adds a named entry for the share's owner, because owning the directory
grants nothing over files inside it that somebody else owns.
For files that already arrived with the broken mode, --repair resets them. It
needs sudo, since they belong to the other account:
./scripts/setup-linux-share.sh --peer <account> --repair
Deletions Do Not Sync
The script runs rsync twice, once each way, with --update and without --delete. Files added on either side show up on both. When both sides changed the same file, the newer copy wins.
Deleting a file does not delete it on the other machine. The next run copies it back.
There is no flag that fixes this. Rsync keeps no state between runs, so it cannot tell "deleted here" from "new over there", and --delete settles the ambiguity in the destructive direction. One stale or empty side would take the other with it. If deletions have to propagate, the right tools are unison or rclone bisync, both of which remember the previous run.
You can work around it by running --up only and letting the Linux folder be the side that accumulates. Clearing out Downloads then does not bring anything back.
Running It on a Schedule
Cygwin's cron package will do it. Every fifteen minutes:
crontab -e
*/15 * * * * /home/<user>/sync-folder.sh >> /home/<user>/sync-folder.log 2>&1
The lock means a slow run will not overlap the next one. Unattended runs have no terminal to unlock a key, so the SSH key needs to be passphrase-less or already loaded in a running agent.
When Plain SSH Works But the Script Does Not
If ssh <host> connects happily and the script cannot:
[FAIL] cannot ssh to 'x'
ssh said: mux_client_request_session: read from master failed: Connection reset by peer
Failed to connect to new control master
that is SSH connection multiplexing, not your key or your host. Cygwin emulates Unix-domain sockets, and ssh's control master is unreliable on top of that emulation. It fails in two directions. A socket left behind by a dead master produces a warning and a fallback to an ordinary connection, which works but is noisy:
ControlSocket /tmp/sync-folder.sh.cm.b0d8030 already exists, disabling multiplexing
A master that refuses to start produces the error above, and there is nothing to fall back to, so the run dies.
The script therefore leaves multiplexing off. All it saves is one authentication
per run, which does not pay for a failure mode that looks exactly like a broken
key. Set SYNC_MULTIPLEX=1 to turn it on if your setup handles it.
Exit Codes 23 and 24
Rsync codes 23 and 24 are treated as warnings rather than failures, because both are routine for a folder somebody is actively using.
Code 24 means a file disappeared between rsync listing it and copying it. A browser cleaning up after itself will do this.
Code 23 means some files could not be transferred at all. Usually that is a filename Linux allows and Windows does not, anything containing : * ? " < > |, or a file another program holds open.