Discussion of the download / upload tools

[halvar]

Originally Posted by deezer

The poster set a thumbnail, which directs not to the full image, but to an imagebam gallery:-)

[URL=http://www.imagebam.com/gallery/cpnazhtdpdpc3lht4u54c0icw6f3rp54][IMG]http://thumbnails116.imagebam.com/49580/eb8796495795513.jpg[/IMG][/URL]

I will dl the gallery now with IHG. If you need the html-file, please let me know.

But I assume, this isn't a bug in the tool. It's more a very similar situation as with direct linked galleries, which we yet discussed here.

I have just added the downloading of linked galleries for imagebam. I will release after testing.

[halvar]

It seems imgbox is currently having troubles.

While testing the new version I experienced missing imgbox image. Sometimes I get the images sometimes not.

A HTTP status code indicates a server error:

2017-06-18 19:36:08 INFO: image-page: [ForkJoinPool-1-worker-1] RETRY http://imgbox.com/nMxJRmEp 
2017-06-18 19:36:09 SEVERE: http://imgbox.com/nMxJRmEpfailed HTTP/1.1 503 Service Unavailable

I also get sometimes an error page in the browser:

[deezer]

Originally Posted by halvar

It seems imgbox is currently having troubles.

While testing the new version I experienced missing imgbox image. Sometimes I get the images sometimes not.

A HTTP status code indicates a server error:

2017-06-18 19:36:08 INFO: image-page: [ForkJoinPool-1-worker-1] RETRY http://imgbox.com/nMxJRmEp 
2017-06-18 19:36:09 SEVERE: http://imgbox.com/nMxJRmEpfailed HTTP/1.1 503 Service Unavailable

I also get sometimes an error page in the browser:

Maybe they can't handle the traffic, as all try to dl their images....

[effCup]

Originally Posted by halvar

Thanks for the file. It seems that the login failed or got lost. The file contains text 'Register on the forum now ...' that is shown if you are not logged in. And if are not logged in you cannot report posts so the report post links are missing.

But it can also be that the login is lost after it initially succeeded. The login is kept in a session cookie. Sometimes these cookies get lost. This could be for various reasons and also happens sometimes in browsers.

I could change it to get the post number from the other links you mentioned. But I only want to do only really necessary changes now to avoid introducing new bugs. And besides I do not know if you really get the pages if you are not logged in.

I agree that it is quite unnecessary to fix this issue at present, or quite possibly ever as it is very infrequent. Just looking at it in problem-solving terms: the showpost buttons should always be present, even for anonym./guest accessors to the site. Much of vef is accessible such, but not everything--e.g. modern--so obviously incorporating a login is essential for the tool's wider operation. So: if it ever does seem like this problem should be solved then the showpost button is the candidate to target.

I suspect there may well be uses for your wonderful tool, or versions of it, even well after 30 June.

[halvar]

https://1fichier.com/?jdv58sbxlf

changes to 1.19 (1.20 is skipped)

• links to imagebam-galleries are processed. Gallery images will be downloaded.
• page nr limit is removed. There was a limit of 1000 for page numbers. This made it impossible to get pages > 1000
• page from and page upto can be optionally specified using the command line (4th and 5th argument: "... thread-id user password [from upto]")
• the retrying at the end of a thread is now also done in parallel with 2 workers

[effCup]

Would it be easy to omit saving images that form part of member's footers (when they've got more than a thumb, that is), or would that overly complicate matters? Not at all a biggie, more just curiosity.

----

For this thread, both deezer & I get the same error, in his case despite repeated attempts.

2017-06-19 22:43:58 INFO: 1 threads specified: [286103] 
2017-06-19 22:43:58 INFO: Starting Thread: 286103 
2017-06-19 22:43:59 INFO: HTTP/1.1 200 OK 
2017-06-19 22:43:59 INFO: IDstack cookie found. Successfully logged in to VEF 
2017-06-19 22:43:59 INFO: Download first thread page: http://vintage-erotica-forum.com/t286103-p1-x.html 
2017-06-19 22:43:59 INFO: HTTP/1.1 200 OK 
2017-06-19 22:44:00 INFO: Thread name: Sonia_Janet_in_red_lace_Model_ID_Request 
2017-06-19 22:44:00 INFO: Download forum page: http://vintage-erotica-forum.com/t286103-p1-x.html to D:\saved from vef\t286103-Sonia_Janet_in_red_lace_Model_ID_Request\t286103-p1-Sonia_Janet_in_red_lace_Model_ID_Request.html 
2017-06-19 22:44:00 INFO: End of Thread reached - no next page link found -  
2017-06-19 22:44:00 SEVERE: Index: 0, Size: 0 
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
	at java.util.ArrayList.rangeCheck(Unknown Source)
	at java.util.ArrayList.get(Unknown Source)
	at vef.imgrescue.ImageLinkProcessor.lambda$getPosts$10(ImageLinkProcessor.java:150)
	at java.util.stream.Collectors.lambda$toMap$58(Unknown Source)
	at java.util.stream.ReduceOps$3ReducingSink.accept(Unknown Source)
	at java.util.stream.ReferencePipeline$2$1.accept(Unknown Source)
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
	at java.util.stream.AbstractPipeline.copyInto(Unknown Source)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)
	at java.util.stream.AbstractPipeline.evaluate(Unknown Source)
	at java.util.stream.ReferencePipeline.collect(Unknown Source)
	at vef.imgrescue.ImageLinkProcessor.getPosts(ImageLinkProcessor.java:148)
	at vef.imgrescue.ImageLinkProcessor.processForumPages(ImageLinkProcessor.java:39)
	at vef.imgrescue.VEFImageRescue.start(VEFImageRescue.java:108)
	at vef.imgrescue.Gui.lambda$null$6(Gui.java:253)
	at java.lang.Thread.run(Unknown Source)

The html file here.

[effCup]

Originally Posted by halvar

• links to imagebam-galleries are processed. Gallery images will be downloaded.

There's a link to an imagebam gallery in the first post of this thread. 1.21 did not process it.

/shrug/ It's not a big deal to me if it doesn't; just reporting it for your interest.

[halvar]

Originally Posted by effCup

Would it be easy to omit saving images that form part of member's footers (when they've got more than a thumb, that is), or would that overly complicate matters? Not at all a biggie, more just curiosity.

Currently I would not like to do this. Try to avoidnew bugs...
Parsing the html is error prone as you see in the error you reported.

I more or less look at pages to figure out how to parse them. If there are variations I did not consider: bam!

It should be more resilient and just ignore posts that cannot be parsed....

Originally Posted by effCup

For this thread, both deezer & I get the same error, in his case despite repeated attempts.

2017-06-19 22:44:00 INFO: Download forum page: http://vintage-erotica-forum.com/t286103-p1-x.html to D:\saved from vef\t286103-Sonia_Janet_in_red_lace_Model_ID_Request\t286103-p1-Sonia_Janet_in_red_lace_Model_ID_Request.html 
2017-06-19 22:44:00 INFO: End of Thread reached - no next page link found -  
2017-06-19 22:44:00 SEVERE: Index: 0, Size: 0 
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
	at java.util.ArrayList.rangeCheck(Unknown Source)
	at java.util.ArrayList.get(Unknown Source)
	at

The html file here.

Thanks for the html, this time the reason is the deleted post:

the post count cannot be found.

I will try to make it more resilient, but today I am busy...

[deezer]

Originally Posted by effCup

There's a link to an imagebam gallery in the first post of this thread. 1.21 did not process it.

/shrug/ It's not a big deal to me if it doesn't; just reporting it for your interest.

I think the tool catches only galleries, which are linked with a thumbnail, which causes an error.
I reported this problem for v1.19 here

In your case the poster set a direct link to the gallery.
halvar mentioned here that he don't want to implement to search for direct links.
This links don't cause errors, but the tool don't find the images.

[effCup]

Originally Posted by halvar

Thanks for the html, this time the reason is the deleted post:

the post count cannot be found.

I will try to make it more resilient, but today I am busy...

Excellent spotting.

I'm afraid deleted posts are not uncommon/do happen on vef. But then again, this is the first I've encountered, and they can always be done "manually"/via IHG, or whatever.