Issue
A little background first - When I do apt-get install
downloads from my company internet it provides a high burst of speed (400-500KB/s) for the first 10 seconds or so before dropping down to a tenth of that (40-50KB/s), and then after a few minutes to a truly miserable (4-5KB/s). This makes me think that the sysadmin has implemented some sort of a network throttling scheme.
Now I know that the network is not simply erratic, because if I start an apt-get install foo
, Ctrl-C
it after 10 seconds and immediately run apt-get install foo
again (by doing an up arrow and enter to use bash history), and then keep repeating this process for a few minutes till all packages are downloaded, I can download even large packages very fast. In particular, even after aborting a download with Ctrl-C, apt-get seems to be able to resume the download in the next invocation.
Of course, staring at the screen doing Ctrl-C Up Enter every 10 seconds gets really boring real fast, so I wrote a shell script -
#!/bin/sh
for i in `seq 1 100` ; do
sudo apt-get install foo -y &
sleep 10
sudo kill -2 $!
done
This seems to work. It spawns apt-get, runs it for 10 seconds and then kills (by sending a SIGINT) it and starts it up again. However, it doesn't really work because now apt-get does not resume downloads on subsequent invocations!
An an experiment I ran sudo apt-get install foo
from one terminal and then ran kill -2 <PID of apt-get>
from another terminal. And even in that case, when I restart apt-get, it does not resume the download.
So clearly a Ctrl-C is not equivalent to SIGINT. And something else is happening when I do Ctrl-C manually which gives apt-get a chance to save the state of the download. The question is - what is it?
Edit
These are the suggestions I have received so far, but no cigars. The mystery deepens! -
On
sudo kill -2 $!
the signal might be going tosudo
instead ofapt-get
. This is not the reason because as mentioned above I also tried sending SIGINT specifically to apt-get's PID and even that prevented apt-get from saving its state.Sudo catches the signal and sends some other signal to apt-get. I tried sending apt-get all the signals I can think of! It still does not resume the download for any of them. It only resumes downloads when I do Ctrl-C to kill it.
Apt-get handles SIGINT differently if it is from a script instead of an interactive shell. Again, the "experiment" above proves that this is not true.
Solution
Okay mystery solved! Thanks to the helpful folks over at the Indian Linux Users Group.
The answer here is two-fold -
Firstly, apt-get
invokes another program called http
for downloading data.
[~] ➔ file /usr/lib/apt/methods/http
/usr/lib/apt/methods/http: ELF 32-bit LSB executable, Intel 80386, version 1
(SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.15,
stripped
Note that that's an executable, not even a script, probably to support downloading files during system installation when none of perl/python/ruby etc. are yet available.
Secondly, when you press Ctrl-C after running apt-get
, the SIGINT gets sent to http
, and not to apt-get
. When http
receives the SIGINT, it saves the download state before shutting down.
Here's the updated script that works perfectly -
#!/bin/sh
for i in `seq 1 100` ; do
sudo apt-get install foo -y &
sleep 10
sudo kill -2 `ps -ae | grep " http" | awk '{print $1}'`
done
Answered By - Anupam Jain Answer Checked By - Clifford M. (WPSolving Volunteer)