Bug 11567 - updxlrator - simultaneous downloads of the same file occur frequently for Windows
Summary: updxlrator - simultaneous downloads of the same file occur frequently for Win...
Status: CLOSED FIXED
Alias: None
Product: IPFire
Classification: Unclassified
Component: --- (show other bugs)
Version: 2
Hardware: unspecified Unspecified
: - Unknown - - Unknown -
Assignee: jluth
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-12-16 10:16 UTC by jluth
Modified: 2018-02-06 07:38 UTC (History)
1 user (show)

See Also:


Attachments
reduceSimultaneousDownloads.patch (998 bytes, patch)
2017-12-16 10:16 UTC, jluth
Details

Note You need to log in before you can comment on or make changes to this bug.
Description jluth 2017-12-16 10:16:15 UTC
Created attachment 547 [details]
reduceSimultaneousDownloads.patch

Recently, Microsoft has started blasting multiple (threaded?) requests for different sections of the same, huge, windowsupdates file. This can easily result in the same url being downloaded by multiple wget commands. I have seen 4 or 5 identical downloads starting within 1 second. This results in a corrupt file, that is identifiable as the file grows to a larger than 100% size since multiple wgets are dumping data onto the same file.

(We have a slow internet connection, so between the time when the "lock" file is deleted and wget actually started downloading data allowed multiple threads to start. A ps ax | grep <url> shows multiple wget and download commands running.)

I have attached the modifications I have made to reduce the problem. I'll know better how effective it is after the next patch Tuesday.  Perhaps a better programmer could figure out how to effectively use flock in this situation to make it fool-proof...