Hard Drives Syncing Question: Same Copies Under Different Folders in Drives

[75er]

Not sure if this is the right section to post,
please feel free to move to a related area if needed

I have several hard drives full of multiple copies of videos & photos some are in different folders in the drives

how can I sync the drives please?

I have
FreeFileSync paid/donation version 11.25
AllDup 4.5.22
Allway Sync 20.1 Pro

or you think there's a better suited app, please guide me to it

w10

all the best

[jagabo]

Are you looking to keep all the duplicates or eliminate them?

[75er]

disk 1 has lot's of video clips and photos in a proper and better organized manner let's call it master disk
disk 2 has the same ones under different folders structure (minus the missing ones from the master disk )
I'd like to mirror the master disk, if I do it then folder structure is different in the disk 2 and I will end up multiple copies of the same ones under different folders in disk 1

want to Keep only 1 mirrored copy of disk 1 in the disk 2, nothing extra

[jagabo]

FreeFileSync can do that. I use it that way.

[75er]

How so?
folder structured are totally different in each drive

[jagabo]

It can copy all the files from the first disk and delete all the other files on the second disk.

[pcspeak]

I have this method.
My folders on drive C:

c:\Master\
c:\Master\Movies
c:\Master\Pics-John
c:\Master\Pics-Mary
c:\Master\Pics-Holidays
c:\Master\Pics-Misc
c:\Master\TV Misc
c:\Master\TV Shows
c:\Master\YouTube

I use a one line command to copy to backup. ROBOCOPY.EXE is part of Windows 10

ROBOCOPY "C:\Master" "D:\Master" /S /XJD /XO /R:1 /W:5 /TEE /log:c:\Master\robocopy-log.txt

Of course there are sub folders. /S copies them.
This gives me a duplicate of C:\Master

This is a once off backup to get me a new/clean backup on D: (I occasionally delete D:\Master and redo the ROBOCOPY command)
To maintain my backup I use pathsync04BETA2-install.exe
It's not been updated for a long while, but is all I need.

This is a partial hijack of your thread 75er, but I've added it to explain ROBOCOPY a little.

The following is run nightly at 10:30 pm. My main computer is on 24/7. I have C: D: & E:
There are errors because of locked system files etc, but all that is important to me is saved.
The creation of %date% may be different for you, subject to your regional settings. Me? Australia.

@echo off
:: Create the root folder for backups.
if not exist e:\Backups md e:\Backups

:: Create env.var %date% with today's date as YYYY-MM-DD.
date/t > e:\Backups\tmp
for /F "tokens=4,3,2 delims=/ " %%a in (e:\Backups\tmp) do set date=%%c-%%b-%%a
del e:\Backups\tmp
:: Create env.var %number% for the next backup for today. (Max 9)
set number=0
for /L %%x in (9,-1,1) do if not exist "e:\Backups\WIN21H2-%date%-%%x" set number=%%x
if %number%==0 goto ErrorMsg1

:: Create folder for the next backup.
set SaveFolder=e:\Backups\WIN21H2-%date%-%number%
md "%SaveFolder%"
ROBOCOPY "%homedrive%%homepath%" "%SaveFolder%\%username%" /s /xjd /r:1 /w:5 /a-:RASH /njh /fp /tee
ROBOCOPY "c:\bat" "%SaveFolder%\Bat" /s /xjd /r:1 /w:5 /a-:RASH /njh /fp /tee
ROBOCOPY "c:\Inet_dl" "%SaveFolder%\Inet_dl" /s /xjd /r:1 /w:5 /a-:RASH /njh /fp /tee
ROBOCOPY "c:\Lib" "%saveFolder%\Lib" /s /xjd /r:1 /w:5 /a-:RASH /njh /fp /tee
ROBOCOPY "c:\VidAud" "%saveFolder%\VidAud" /s /xjd /r:1 /w:5 /a-:RASH /njh /fp /tee
Attrib -S -H -R -A "%SaveFolder%\*.*" /D /S
goto :end

:ErrorMsg1
echo Error creating destination folder.
pause

:end

Whew!
And cheers.

[jagabo]

I use XCOPY in a similar way, sometimes. Though it doesn't support automatic deletion of files in the destination that don't exist in the source -- ie true mirroring.

[pcspeak]

You could add /MIR


/PURGE : Delete dest files/folders that no longer exist in source.
/MIR : MIRror a directory tree - equivalent to /PURGE plus all subfolders (/E)


ROBOCOPY "c:\bat" "%SaveFolder%\Bat" /s /MIR /xjd /r:1 /w:5 /a-:RASH /njh /fp /tee

[jagabo]

I meant XCOPY doesn't support that feature.

[75er]

It can copy all the files from the first disk and delete all the other files on the second disk.

ideal and long solution is

delete all in disk 2

and mirror the copies of disk 1 onto disk 2

Unless there's a time effective way you might suggest

about a huge content 7TB portable External Hard Drive EACH

[Alwyn]

I use Free File Sync.

I'd re-arrange the folders on disk 2 to roughly match disk 1 first. Then when you do your "mirror" sync, you won't be copying as many files over. Your "re-arrangement" doesn't need to be perfect, just get the bulk of files into the same folders. The more rearranging you do, the less copying and deleting has to be done by FFS.

I often do this if I re-arrange the folders on my master drive (I'm never happy with my drive layouts!). On my backup drives, I do the re-arrangement manually to stop Free File Sync deleting all the files on the backup drives and then copying all the master files over into the new layout. Rearranging folders/files on the same drive takes seconds.

I'm using "File Time and Size" as my Compare parameters.

And whatever you do, copy all your files for keeping onto another drive before you start this, just in case it all goes pear-shaped.

[MikeeJac]

What is the problem with FreeFileSync and AllwaySync ? bothcan do it ! and also you can check out Gs Richopy 360 and Syncback as a GUI backup tools or you can try Robocopy , the robust CLI tool .

[75er]

I use Free File Sync.

I'd re-arrange the folders on disk 2 to roughly match disk 1 first. Then when you do your "mirror" sync, you won't be copying as many files over. Your "re-arrangement" doesn't need to be perfect, just get the bulk of files into the same folders. The more rearranging you do, the less copying and deleting has to be done by FFS.

Rearranging folders/files on the same drive takes seconds.

Thank you Alwyn,
I have started re-arranging the folders on disk 2 to roughly match disk 1.
(Couldn't do the two-way as the capacity wasn't enough in each)

Then tried eliminating large duplicate ones with AllDup
Then will start mirror syncing with FFS

Thank you everyone,
I appreciate your effort time and consideration to help me sorting things out

please take care

all the best to you all

[Cornucopia]

Do batch renaming of the errant folder names, then mirror/merge.
I use 14aren, an older renamer, but it still works great. I also use FFS like above, but also use WinMerge.

Decide ahead of time what you want your naming system to be going forward, then force everything into that system and stick with it, it'll make everything go alot smoother after this short-term hurdle.


Scott