Discussion of the download / upload tools
 

Please use this thread for technical discussion or problems with software being used.

Bug Reports vef-imagerescue 1.10
 

I got several bug reports for vef_imagerescue v 1.10:
If the thread title contains special characters the download folder cannot be created. Like quotation marks.

I will create a fix after I get back from work.
Downloading of thumbnails will also be removed in the next version. They are not necessary and this speeds up the whole process.

Thanks to the bug reporters for supplying failed threads. This makes reproducing, fixing and testing a lot easier.

Hal, messaged you yesterday about the slight inconsistent nature of downloads, sometimes it worked, sometimes it didn't, like the example where 107 files were downloaded by v1.9 and 108 by v1.7.

Not sure if you could possibly look at parsing the threads possibly twice, just to make sure images are definitely downloaded. It may have been a server issue on their end the first time it attempted to download it, but given time from not accessing the page and possibly on the second pass it may grab an image it failed to download the first time?

I don't know how workable this is, because I assume you create the .xxxxxx file to say you've attempted to download the image from that page, and I'm sorry I don't have concrete evidence to help pinpoint the issue.

It could just be pot luck whether it gets the image first time or not :(

It's a flipping really great tool. You have my highest level of respect here for your time and patience. I'm running it on 4 or 5 threads at a time, could that be an issue?

Cheers.

You are right, the .xxxxxx are to mark attempted downloads. I need a marker to not retry downloads indefinetly.
Would you please check if there are empty (zero bytes) .xxxxxxx in the folder where images are missing? If a download succeeds the file should contain the images file name.

Deleting the empty file and re-running the thread should retry to download the missing files.

I could add a rescan-feature that lools for zero byte files and retries them.
But this would take some time to implement and test.

This should not be a problem, but it could be that imagebam throttles parallel requests coming from the same ip address.

I'll check it out mate when I get home :) thanks as always.

In this case I think the images are from imgbox :)

I'll get back to you in around 4 hours or so :)

I suspect that the load on both imagebam and imgbox would be higher than normal if posters are scrambling to download content.

It may just be perception but I think both have slowed noticeably since their announcements.

However version 1.9 is working well for me.:thumbsup:

Just linking to halvar's vef-imagerescue 1.10 tool, so it's readily available/visible here for anyone who wants to use it.

vef-imagerescue 1.11 is available
 

download it here: https://1fichier.com/?6ui0ayo9ki

changes since 1.10
* Fixed Bug: Threads with special characters in the name could not be downloaded
* Failed image downloads are retried once
* Thumbnails are no more downloaded

Usage:
or start the GUI:
If anybody is interested in the source, it is included in the download and I just pushed the
project to teknik.io: https://git.teknik.io/halvar/vef-imagerescue

When are they​ retried hal? Straight away or does it go off and do other images before returning?

Cheers :)

@halvar: fantastic tool.:cool::thumbsup:

1) very minor suggestion, maybe too cosmetic to even bother?: you create a folder for the thread & inside that folders for posts with images to save.

I made the mistake of first testing your tool (1.10) on a thread with a very long title,:rolleyes: so I've got post folders each with very long titles inside the thread folder with a very long title, i.e.:

thread folder:
t33256-###_Marlon's_'Don't_know_where_else_to_put_them'_s mall_vintage_sets_Vintage_Erotica_Talk

post folder:
t33256-p15-###_Marlon's_'Don't_know_where_else_to_put_them'_s mall_vintage_sets_Vintage_Erotica_Talk-post-146-576111

The post folders only need: thread id, page, then post id.

Strictly, the post folders really only need the post id, as that is canonical on vef, but the thread id & page are perhaps helpful/reassuring for "humans"?:rolleyes:

When re-upping one can simply "visit" post ids:
so technically no need to "know" the page, etc.

2) a more serious problem: it seems to have found only 2 posts in that entire thread with imagebam or imgbox images?

p=576111, p=636725.

but there are posts immediately below that second post that also have imagebam images, for example p=636727, but they have not had their images saved?:confused:

I do not think I accidentally "stopped" your tool from completing its task--it got all 312 pages, but seemingly stopped with just 2 posts with images.

I tried to re-run the tool on the same thread but no change/additions.:confused: