Vintage Erotica Forums

Vintage Erotica Forums (http://vintage-erotica-forum.com/index.php)
-   Imagebam/Imgbox are no longer shutting down (http://vintage-erotica-forum.com/forumdisplay.php?f=88)
-   -   Discussion of the download / upload tools (http://vintage-erotica-forum.com/showthread.php?t=333326)

Jenny48549 06-14-2017 02:26 AM

Discussion of the download / upload tools
 
Please use this thread for technical discussion or problems with software being used.

halvar 06-14-2017 04:41 AM

Bug Reports vef-imagerescue 1.10
 
I got several bug reports for vef_imagerescue v 1.10:
If the thread title contains special characters the download folder cannot be created. Like quotation marks.

I will create a fix after I get back from work.
Downloading of thumbnails will also be removed in the next version. They are not necessary and this speeds up the whole process.

Thanks to the bug reporters for supplying failed threads. This makes reproducing, fixing and testing a lot easier.

BCFC_1982 06-14-2017 10:23 AM

Hal, messaged you yesterday about the slight inconsistent nature of downloads, sometimes it worked, sometimes it didn't, like the example where 107 files were downloaded by v1.9 and 108 by v1.7.

Not sure if you could possibly look at parsing the threads possibly twice, just to make sure images are definitely downloaded. It may have been a server issue on their end the first time it attempted to download it, but given time from not accessing the page and possibly on the second pass it may grab an image it failed to download the first time?

I don't know how workable this is, because I assume you create the .xxxxxx file to say you've attempted to download the image from that page, and I'm sorry I don't have concrete evidence to help pinpoint the issue.

It could just be pot luck whether it gets the image first time or not :(

It's a flipping really great tool. You have my highest level of respect here for your time and patience. I'm running it on 4 or 5 threads at a time, could that be an issue?

Cheers.

halvar 06-14-2017 01:02 PM

Quote:

Originally Posted by BCFC_1982 (Post 4067875)
I don't know how workable this is, because I assume you create the .xxxxxx file to say you've attempted to download the image from that page, and I'm sorry I don't have concrete evidence to help pinpoint the issue.

It could just be pot luck whether it gets the image first time or not :(

You are right, the .xxxxxx are to mark attempted downloads. I need a marker to not retry downloads indefinetly.
Would you please check if there are empty (zero bytes) .xxxxxxx in the folder where images are missing? If a download succeeds the file should contain the images file name.

Deleting the empty file and re-running the thread should retry to download the missing files.

I could add a rescan-feature that lools for zero byte files and retries them.
But this would take some time to implement and test.

Quote:

Originally Posted by BCFC_1982 (Post 4067875)
It could just be pot luck whether it gets the image first time or not :(

I'm running it on 4 or 5 threads at a time, could that be an issue?

This should not be a problem, but it could be that imagebam throttles parallel requests coming from the same ip address.

BCFC_1982 06-14-2017 01:10 PM

I'll check it out mate when I get home :) thanks as always.

In this case I think the images are from imgbox :)

I'll get back to you in around 4 hours or so :)

andw 06-14-2017 01:19 PM

Quote:

Originally Posted by halvar (Post 4068008)
...it could be that imagebam throttles parallel requests coming from the same ip address.

I suspect that the load on both imagebam and imgbox would be higher than normal if posters are scrambling to download content.

It may just be perception but I think both have slowed noticeably since their announcements.

However version 1.9 is working well for me.:thumbsup:

effCup 06-14-2017 01:22 PM

Just linking to halvar's vef-imagerescue 1.10 tool, so it's readily available/visible here for anyone who wants to use it.
Quote:

Originally Posted by halvar (Post 4067329)
The new version 1.10 only needs the thread number instead of the whole name with the dreaded -p1-.
https://1fichier.com/?zirwkstvbn

e.g.
Code:

java -jar vef-imagerescue-1.10.jar 115584 username password
or start the GUI without parameter:
Code:

java -jar vef-imagerescue-1.10.jar


halvar 06-14-2017 03:12 PM

vef-imagerescue 1.11 is available
 
download it here: https://1fichier.com/?6ui0ayo9ki

changes since 1.10
* Fixed Bug: Threads with special characters in the name could not be downloaded
* Failed image downloads are retried once
* Thumbnails are no more downloaded

Usage:
Code:

java -jar vef-imagerescue-1.11.jar 115584 username password
or start the GUI:
Code:

java -jar vef-imagerescue-1.11.jar
If anybody is interested in the source, it is included in the download and I just pushed the
project to teknik.io: https://git.teknik.io/halvar/vef-imagerescue

BCFC_1982 06-14-2017 03:13 PM

When are they​ retried hal? Straight away or does it go off and do other images before returning?

Cheers :)

effCup 06-14-2017 03:26 PM

@halvar: fantastic tool.:cool::thumbsup:

1) very minor suggestion, maybe too cosmetic to even bother?: you create a folder for the thread & inside that folders for posts with images to save.

I made the mistake of first testing your tool (1.10) on a thread with a very long title,:rolleyes: so I've got post folders each with very long titles inside the thread folder with a very long title, i.e.:

thread folder:
t33256-###_Marlon's_'Don't_know_where_else_to_put_them'_s mall_vintage_sets_Vintage_Erotica_Talk

post folder:
t33256-p15-###_Marlon's_'Don't_know_where_else_to_put_them'_s mall_vintage_sets_Vintage_Erotica_Talk-post-146-576111

The post folders only need: thread id, page, then post id.

Strictly, the post folders really only need the post id, as that is canonical on vef, but the thread id & page are perhaps helpful/reassuring for "humans"?:rolleyes:

When re-upping one can simply "visit" post ids:
Code:

http://vintage-erotica-forum.com/showthread.php?p=NNNNN
so technically no need to "know" the page, etc.

2) a more serious problem: it seems to have found only 2 posts in that entire thread with imagebam or imgbox images?

p=576111, p=636725.

but there are posts immediately below that second post that also have imagebam images, for example p=636727, but they have not had their images saved?:confused:

I do not think I accidentally "stopped" your tool from completing its task--it got all 312 pages, but seemingly stopped with just 2 posts with images.

I tried to re-run the tool on the same thread but no change/additions.:confused:


All times are GMT. The time now is 07:41 AM.



vBulletin Optimisation provided by vB Optimise v2.6.1 (Pro) - vBulletin Mods & Addons Copyright © 2018 DragonByte Technologies Ltd.