Comments on: Download Files Async With Gio And Python

By: hb

hb — Sun, 21 Mar 2010 09:09:01 +0000

Is there a way to monitor % of download ?

By: Joe Shaw

Joe Shaw — Tue, 16 Mar 2010 01:05:04 +0000

One final comment: if you are using load_contents(), you are storing the entire contents of the file in memory. This is not a big deal if you are downloading a 1 or even a 100 kilobyte file, but if you’re talking about several megabytes, that is a considerable waste.

By: Joe Shaw

Joe Shaw — Tue, 16 Mar 2010 00:49:32 +0000

Oh well, that code formatting got butchered…

http://gist.github.com/333511

By: Joe Shaw

Joe Shaw — Tue, 16 Mar 2010 00:47:28 +0000

GIO has always felt a little awkward to me, and it seems especially so when translated into Python.

I'm rather fond of the Node.js way, but maybe it's because I love Javascript's nestable function literals:

var fs = require("fs"); var http = require("http");



var client = http.createClient(80, "google.com");
var request = client.request("GET", "/", {"host": "google.com"});
var file = fs.createWriteStream("google.html",
                                { flags: "w",
                                  mode: 0644 });

request.addListener("response", function(response) { response.addListener("data", function(chunk) { file.write(chunk); }); response.addListener("end", function() { file.close(); }); }); request.close();

The beauty part about this is that it's entirely async -- both reads via HTTP and writes to the file system. (Although this one ignores any errors, never mind about that. :)

And, well, Node doesn't integrate with GTK.

By: Derek

Derek — Tue, 16 Mar 2010 00:19:15 +0000

I like the example: very straightforward. I agree that asynchronously writing it to disk might be nice, but that could go in a different snippet.

One thing that I wonder is how many characters you have per line? Is it 80? The code is a little truncated on your website (though it’d be fine in Acire, of course).

Glad you figured it out. )

By: Rodney Dawes

Rodney Dawes — Mon, 15 Mar 2010 20:42:19 +0000

GIO is nice, but it's not always what you want to use, as it's not designed to be an HTTP library, but a filesystem abstraction. In this sense, it's usually ok to use it to copy from one location to another.

However, if to download a file, you need to do more complicated things with HTTP headers and auth, you will probably want to use libsoup (or httplib(2?) with GIOChannels).

Also, I know people are oft afraid to use threads, because they're "difficult" to get right and all, but using threads in Python is pretty trivial (even with glib/gtk+), and will make it easier to do the write to disk async as well.

By: jono

jono — Mon, 15 Mar 2010 20:00:45 +0000

Well, yeah…but I figured the write is pretty quick. I still need to port that to be async.

By: Jesse van den Kieboom

Jesse van den Kieboom — Mon, 15 Mar 2010 19:52:49 +0000

Of course you do realize that right now your writing to disk is done sync, and not async You could try the ‘splice’ API to write an input stream to an output stream.