Bug 11567 - updxlrator - simultaneous downloads of the same file occur frequently for Windows
Summary: updxlrator - simultaneous downloads of the same file occur frequently for Win...
Alias: None
Product: IPFire
Classification: Unclassified
Component: --- (show other bugs)
Version: 2
Hardware: unspecified Unspecified
: - Unknown - - Unknown -
Assignee: jluth
QA Contact:
Depends on:
Reported: 2017-12-16 10:16 UTC by jluth
Modified: 2018-02-06 07:38 UTC (History)
1 user (show)

See Also:

reduceSimultaneousDownloads.patch (998 bytes, patch)
2017-12-16 10:16 UTC, jluth

Note You need to log in before you can comment on or make changes to this bug.
Description jluth 2017-12-16 10:16:15 UTC
Created attachment 547 [details]

Recently, Microsoft has started blasting multiple (threaded?) requests for different sections of the same, huge, windowsupdates file. This can easily result in the same url being downloaded by multiple wget commands. I have seen 4 or 5 identical downloads starting within 1 second. This results in a corrupt file, that is identifiable as the file grows to a larger than 100% size since multiple wgets are dumping data onto the same file.

(We have a slow internet connection, so between the time when the "lock" file is deleted and wget actually started downloading data allowed multiple threads to start. A ps ax | grep <url> shows multiple wget and download commands running.)

I have attached the modifications I have made to reduce the problem. I'll know better how effective it is after the next patch Tuesday.  Perhaps a better programmer could figure out how to effectively use flock in this situation to make it fool-proof...