Thanks for your detailed answer, Sude.
Sude: There isn't an option that does chunk based integrity check like --repair
What I don't like with --repair is the "Redownload will delete the old file" part:
--repair
Repair downloaded files Use --repair --download to redownload
files when filesizes don't match (possibly different version).
Redownload will delete the old file
I don't like to lose old versions of files by an automatic process. Is there a way to make a backup of the old file instead? What does --repair do without the --download option?
Sude: Perhaps I should change --status to not use local XML data so it can be better used for checking integrity.
By understanding better how it works and that it only compares local with remote XML files --status looks good as it is now. I would not change it. GOG should provide XML files for every file which is download-able, but that's something different and beyond our control.
Instead of changing the behavior of --status I would prefer a separate option to check the integrity of local files. Something like --check or --check-integrity, which checks all local files against locally saved XML files (including extras when there are XML files available), giving an error when a checksum doesn't match and maybe printing out a warning when there's no local XML file available for a downloaded file.
eiii: Also probably since I use the "--xml-directory" and "--create-xml automatic" options the download takes hours even when there is only a single new game to download (and I have a fast internet connection).
The downloader causes a lot of disk I/O. Does it verify the checksums of already downloaded files when I use the --xml* options?
Sude: Downloader::downloadFile uses Downloader::getLocalFileHash to get file hash for version check.
If local XML data is available then it's a fast process as it only needs to parse the XML and get the file hash from it. However if local XML data is not available then it will call Util::getFileHash which can take a long time for big files and causes lots of disk I/O.
Local XML data makes the process much faster which is why the remote XML data is automatically saved for all installers and patches.
I'd like to remove the version check code from Downloader::downloadFile but GOG sometimes does silent updates without changing the file name. These silent updates caused lots of problems in the past which was the reason version check had to be implemented.
Looks like it's my fault then. To not lose local XML files, especially those for files which are changed or not available from GOG anymore, I start every download with a separate XML directory. Understanding how the download works this is probably not a good idea. :)
So at least I should start the download with a copy of the old XML directory or maybe put the XML files into a git repository. Time to write a wrapper script for the downloads I guess.
And yes, when GOG is unable to handle version changes properly please do not remove the version check. ;)
What do you do when the version check fails? Do you back up the changed file resulting in an *.old file? And do you also back up the local XML file?
I would prefer to have time stamps for the old files to not overwrite old *.old files accidentally (something like file-`date '+%Y%m%d%H%M%S'`.old). But when I write a wrapper script for the download I can take care of that in the script.
eiii: It looks like the --report option overwrites the report file on each run. It would be nice if I could provide a filename with this option.
Sude: I'll add an option to change the report file
Nice, thank you.
Sude: Perhaps progress information should be printed to stderr instead of stdout. I'll see what I can do.
Assuming that error messages are already written to stderr this would not be very helpful. You would lose error messages in the log when you suppress the progress information. If you separate the output this way it should be rather vice versa, progress information to stdout and the other messages to stderr.
I still would prefer to have more information in the report file so that there would be no need to capture the command line output of the downloader. An option to suppress the progress information would be another alternative, though with the downside that you also would lose the progress information on the terminal.
Sude: You can use --status to check integrity of files if you don't have XML data saved in ~/.cache/lgogdownloader/xml/gamename/
I guess that's what I already did when starting with an empty XML directory. :) So I still could use that as a hack to verify the file integrity. But if I understand you correctly this won't work for extra files, even when I have created local XML files with "--create-xml automatic", right?
