June 20th, 2017, 01:53 AM | #111 |
Junior Member
Join Date: Dec 2007
Posts: 9
Thanks: 602
Thanked 95 Times in 7 Posts
|
...anyway, not only the shutdown news is frustrating but to make the thing worse a lot of galleries, while perfectly viewable as preview, are unreachable. The curious thing is the more recent posts seems to be more affected than older ones.
Some examples... http://vintage-erotica-forum.com/sho...4&postcount=31 http://vintage-erotica-forum.com/sho...8&postcount=41 http://vintage-erotica-forum.com/sho...7&postcount=31 |
The Following 11 Users Say Thank You to bugsbunny80 For This Useful Post: |
June 20th, 2017, 03:14 AM | #112 | |
Blocked!
Join Date: Jan 2008
Location: HH
Posts: 1,963
Thanks: 115,040
Thanked 32,801 Times in 1,955 Posts
|
Quote:
And the retry should to handle temporary glitches. |
|
June 20th, 2017, 03:27 AM | #113 |
Blocked!
Join Date: Jan 2008
Location: HH
Posts: 1,963
Thanks: 115,040
Thanked 32,801 Times in 1,955 Posts
|
Imagebam troubles
Currently I cannot download from imagebam anymore. Not only imagerescue is affected, but also Imagehost grabber.
When downloading the test thread http://vintage-erotica-forum.com/t333370-p1-x.html with IHG, only a few images can be downloaded. The others take a long time, 10 retries and fail. I hope that it's only temporary. At this failure rate it does not make sense to do mass downloads. I will wait for the evening and then try again. |
June 20th, 2017, 03:30 AM | #114 | |
Blocked!
Join Date: Jan 2008
Location: HH
Posts: 1,963
Thanks: 115,040
Thanked 32,801 Times in 1,955 Posts
|
Quote:
|
|
June 20th, 2017, 03:35 AM | #115 |
Blocked!
Join Date: Jan 2008
Location: HH
Posts: 1,963
Thanks: 115,040
Thanked 32,801 Times in 1,955 Posts
|
Recently I also noticed that retries mostly fail. But it is hard to say it is alway better to not retry. Maybe tomorrow we will have recoverable errors where the retry succeed...
|
June 20th, 2017, 06:17 AM | #116 |
Blocked!
Join Date: Jan 2008
Location: HH
Posts: 1,963
Thanks: 115,040
Thanked 32,801 Times in 1,955 Posts
|
It seems Imagebam is running better now, I could download the thread t121430-Girls_In_Pearls_General_Modern_Topics.
But imgbox seems to run better, here I see the two workers alternating, as one wouls expect: Code:
2017-06-20 08:16:52 INFO: [ForkJoinPool-1-worker-1]image-page: http://imgbox.com/adns1lhD 2017-06-20 08:16:53 INFO: [ForkJoinPool-1-worker-1]downloaded: https://i.imgbox.com/adns1lhD.jpg to storage/t229284-Chubby_But_Not_BBW_General_Modern_Topics/t229284-p4-Chubby_But_Not_BBW_General_Modern_Topics-post-33-2420539/20081031-IMG_2583.jpg 2017-06-20 08:16:53 INFO: [ForkJoinPool-1-worker-1]image-page: http://imgbox.com/adqMtAnb 2017-06-20 08:16:53 INFO: [ForkJoinPool-1-worker-0]downloaded: https://i.imgbox.com/abqpEesm.jpg to storage/t229284-Chubby_But_Not_BBW_General_Modern_Topics/t229284-p4-Chubby_But_Not_BBW_General_Modern_Topics-post-32-2420537/Laura_f07.JPG 2017-06-20 08:16:53 INFO: [ForkJoinPool-1-worker-0]image-page: http://imgbox.com/abj4TFol 2017-06-20 08:16:53 INFO: [ForkJoinPool-1-worker-1]downloaded: https://i.imgbox.com/adqMtAnb.jpg to storage/t229284-Chubby_But_Not_BBW_General_Modern_Topics/t229284-p4-Chubby_But_Not_BBW_General_Modern_Topics-post-33-2420539/20081031-IMG_2582.jpg 2017-06-20 08:16:53 INFO: [ForkJoinPool-1-worker-1]image-page: http://imgbox.com/abbwANK8 |
June 20th, 2017, 06:55 AM | #117 |
Blocked!
Join Date: Jan 2008
Location: HH
Posts: 1,963
Thanks: 115,040
Thanked 32,801 Times in 1,955 Posts
|
version 1.22 of imagerescue released
https://1fichier.com/?vrti7pxval
It seems that the success rate for imagebam varies from thread to thread or post to post. I would suggest removing the automatic retry if it does not provide any value. You can alway do a manual retry by deleting the empty dot files and rerunning the thread. I usually do this after a couple of hours. |
June 20th, 2017, 09:26 AM | #118 |
Super Moderator
Join Date: Mar 2009
Posts: 3,870
Thanks: 1,043,099
Thanked 52,360 Times in 3,827 Posts
|
Many thanks for the info and updates.
Using v1.22 what would be the command line arguments to filter by poster and thread range. I can see the object line as: public VEFImageRescue(int threadId, String username, String password, String storagePath, String filterPoster, int pageFrom, int pageUpto) however when instantiated it's: new VEFImageRescue(Integer.parseInt(args[0]), args[1], args[2], "storage", null, from, to).start(); which I think means the storagePath defaults to relative path "storage" and filterPoster is ignored and set to null? Wish I was an object-oriented programmer. |
The Following 10 Users Say Thank You to andw For This Useful Post: |
June 20th, 2017, 09:28 AM | #119 |
Vintage Idiot
Join Date: Feb 2012
Location: History
Posts: 22,143
Thanks: 226,791
Thanked 357,042 Times in 21,638 Posts
|
I understand the appeal of not wanting your tool to hang around waiting for images to download when they cannot. More troubling to me, however, is the lack of notice that I, as user, get of these issues.
Code:
2017-06-20 20:38:35 INFO: Usage without GUI: java -jar vef-imagerescue.jar thread username password 2017-06-20 20:39:50 INFO: 2 threads specified: [272029, 269539] [...] 2017-06-20 20:39:58 INFO: Starting Thread: 269539 2017-06-20 20:39:59 INFO: HTTP/1.1 200 OK 2017-06-20 20:40:00 INFO: IDstack cookie found. Successfully logged in to VEF 2017-06-20 20:40:00 INFO: Download first thread page: http://vintage-erotica-forum.com/t269539-p1-x.html 2017-06-20 20:40:01 INFO: HTTP/1.1 200 OK 2017-06-20 20:40:04 INFO: Thread name: Carmen_new_content_Mystery_Corner 2017-06-20 20:40:04 INFO: Download forum page: http://vintage-erotica-forum.com/t269539-p1-x.html to D:\saved from vef\t269539-Carmen_new_content_Mystery_Corner\t269539-p1-Carmen_new_content_Mystery_Corner.html 2017-06-20 20:40:05 INFO: NO DEADLOCK 2017-06-20 20:40:10 INFO: End of Thread reached - no next page link found - 2017-06-20 20:40:10 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/dcbf03331549696 2017-06-20 20:41:35 INFO: NO DEADLOCK 2017-06-20 20:42:15 INFO: [ForkJoinPool-1-worker-1]downloaded: http://111.imagebam.com/download/I0u15wnqG1tuhVUz-24pbQ/33155/331549696/IMG_20140607_0001.jpg to D:\saved from vef\t269539-Carmen_new_content_Mystery_Corner\t269539-p1-Carmen_new_content_Mystery_Corner-post-1-2930106\IMG_20140607_0001.jpg 2017-06-20 20:42:15 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/500b8d331549812 2017-06-20 20:43:05 INFO: NO DEADLOCK 2017-06-20 20:44:34 INFO: [ForkJoinPool-1-worker-1]downloaded: http://111.imagebam.com/download/uD2N4wHSk_20aWSQzEOR9w/33155/331549812/IMG_20140607_0002.jpg to D:\saved from vef\t269539-Carmen_new_content_Mystery_Corner\t269539-p1-Carmen_new_content_Mystery_Corner-post-1-2930106\IMG_20140607_0002.jpg 2017-06-20 20:44:34 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/42f53a331549922 2017-06-20 20:44:35 INFO: NO DEADLOCK 2017-06-20 20:46:05 INFO: NO DEADLOCK 2017-06-20 20:46:10 INFO: [ForkJoinPool-1-worker-1]downloaded: http://110.imagebam.com/download/fXp0GvgUNKChBUEVxY0hXA/33155/331549922/IMG_20140607_0003.jpg to D:\saved from vef\t269539-Carmen_new_content_Mystery_Corner\t269539-p1-Carmen_new_content_Mystery_Corner-post-1-2930106\IMG_20140607_0003.jpg 2017-06-20 20:46:10 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/ae0526331550076 2017-06-20 20:46:31 SEVERE: [ForkJoinPool-1-worker-1] Error downloading 'http://www.imagebam.com/image/ae0526331550076' java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at org.apache.http.impl.conn.LoggingInputStream.read(LoggingInputStream.java:84) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282) at org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:264) at org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:225) at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:184) at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135) at java.util.zip.InflaterInputStream.fill(Unknown Source) at java.util.zip.InflaterInputStream.read(Unknown Source) at java.util.zip.GZIPInputStream.read(Unknown Source) at org.apache.http.client.entity.LazyDecompressingInputStream.read(LazyDecompressingInputStream.java:70) at sun.nio.cs.StreamDecoder.readBytes(Unknown Source) at sun.nio.cs.StreamDecoder.implRead(Unknown Source) at sun.nio.cs.StreamDecoder.read(Unknown Source) at java.io.InputStreamReader.read(Unknown Source) at java.io.Reader.read(Unknown Source) at org.apache.http.util.EntityUtils.toString(EntityUtils.java:225) at org.apache.http.util.EntityUtils.toString(EntityUtils.java:306) at vef.imgrescue.ImageBam.extractImageURL(ImageBam.java:25) at vef.imgrescue.AbstactImageHost.download(AbstactImageHost.java:56) at vef.imgrescue.ImageLinkProcessor.lambda$downloadImages$8(ImageLinkProcessor.java:121) at java.util.ArrayList.forEach(Unknown Source) at vef.imgrescue.ImageLinkProcessor.downloadImages(ImageLinkProcessor.java:105) at vef.imgrescue.ImageLinkProcessor.lambda$null$1(ImageLinkProcessor.java:56) at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source) at java.util.TreeMap$EntrySpliterator.forEachRemaining(Unknown Source) at java.util.stream.AbstractPipeline.copyInto(Unknown Source) at java.util.stream.ForEachOps$ForEachTask.compute(Unknown Source) at java.util.concurrent.CountedCompleter.exec(Unknown Source) at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) at java.util.concurrent.ForkJoinTask.doInvoke(Unknown Source) at java.util.concurrent.ForkJoinTask.invoke(Unknown Source) at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(Unknown Source) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(Unknown Source) at java.util.stream.AbstractPipeline.evaluate(Unknown Source) at java.util.stream.ReferencePipeline.forEach(Unknown Source) at java.util.stream.ReferencePipeline$Head.forEach(Unknown Source) at vef.imgrescue.ImageLinkProcessor.lambda$processForumPages$2(ImageLinkProcessor.java:42) at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(Unknown Source) at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(Unknown Source) at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) 2017-06-20 20:46:31 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/62bac8331550167 2017-06-20 20:47:28 INFO: 1 threads specified: [269539] 2017-06-20 20:47:28 INFO: Starting Thread: 269539 2017-06-20 20:47:29 INFO: HTTP/1.1 200 OK 2017-06-20 20:47:29 INFO: IDstack cookie found. Successfully logged in to VEF 2017-06-20 20:47:29 INFO: Using existing target folder :D:\saved from vef\t269539-Carmen_new_content_Mystery_Corner 2017-06-20 20:47:29 INFO: Download forum page: http://vintage-erotica-forum.com/t269539-p1-x.html to D:\saved from vef\t269539-Carmen_new_content_Mystery_Corner\t269539-p1-Carmen_new_content_Mystery_Corner.html 2017-06-20 20:47:30 INFO: End of Thread reached - no next page link found - 2017-06-20 20:47:30 INFO: [ForkJoinPool-1-worker-0]image-page: http://www.imagebam.com/image/dcbf03331549696 2017-06-20 20:47:30 INFO: [ForkJoinPool-1-worker-0]http://www.imagebam.com/image/dcbf03331549696 is already processed 2017-06-20 20:47:30 INFO: [ForkJoinPool-1-worker-0]image-page: http://www.imagebam.com/image/500b8d331549812 2017-06-20 20:47:30 INFO: [ForkJoinPool-1-worker-0]http://www.imagebam.com/image/500b8d331549812 is already processed 2017-06-20 20:47:30 INFO: [ForkJoinPool-1-worker-0]image-page: http://www.imagebam.com/image/42f53a331549922 2017-06-20 20:47:30 INFO: [ForkJoinPool-1-worker-0]http://www.imagebam.com/image/42f53a331549922 is already processed 2017-06-20 20:47:30 INFO: [ForkJoinPool-1-worker-0]image-page: http://www.imagebam.com/image/ae0526331550076 2017-06-20 20:47:30 INFO: [ForkJoinPool-1-worker-0]http://www.imagebam.com/image/ae0526331550076 is already processed 2017-06-20 20:47:30 INFO: [ForkJoinPool-1-worker-0]image-page: http://www.imagebam.com/image/62bac8331550167 2017-06-20 20:47:35 INFO: NO DEADLOCK 2017-06-20 20:47:36 INFO: [ForkJoinPool-1-worker-1]downloaded: http://111.imagebam.com/download/CQ9gJ1IxenONoaU6N_ix-Q/33156/331550167/IMG_20140607_0005.jpg to D:\saved from vef\t269539-Carmen_new_content_Mystery_Corner\t269539-p1-Carmen_new_content_Mystery_Corner-post-1-2930106\IMG_20140607_0005.jpg 2017-06-20 20:47:36 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/e45d0d331550323 2017-06-20 20:48:32 INFO: [ForkJoinPool-1-worker-0]downloaded: http://111.imagebam.com/download/CQ9gJ1IxenONoaU6N_ix-Q/33156/331550167/IMG_20140607_0005.jpg to D:\saved from vef\t269539-Carmen_new_content_Mystery_Corner\t269539-p1-Carmen_new_content_Mystery_Corner-post-1-2930106\IMG_20140607_0005.jpg 2017-06-20 20:48:32 INFO: [ForkJoinPool-1-worker-0]image-page: http://www.imagebam.com/image/e45d0d331550323 2017-06-20 20:49:05 INFO: NO DEADLOCK 2017-06-20 20:49:33 INFO: [ForkJoinPool-1-worker-1]downloaded: http://109.imagebam.com/download/aq9YOoslW_bxMZaSYDr6cw/33156/331550323/IMG_20140607_0006.jpg to D:\saved from vef\t269539-Carmen_new_content_Mystery_Corner\t269539-p1-Carmen_new_content_Mystery_Corner-post-1-2930106\IMG_20140607_0006.jpg 2017-06-20 20:49:33 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/0775c5331550426 2017-06-20 20:50:35 INFO: NO DEADLOCK 2017-06-20 20:50:52 INFO: [ForkJoinPool-1-worker-1]downloaded: http://111.imagebam.com/download/1cKND9lGeZDXMhIeZOd-fA/33156/331550426/IMG_20140607_0007.jpg to D:\saved from vef\t269539-Carmen_new_content_Mystery_Corner\t269539-p1-Carmen_new_content_Mystery_Corner-post-1-2930106\IMG_20140607_0007.jpg 2017-06-20 20:50:52 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/dee561331550512 2017-06-20 20:50:55 INFO: [ForkJoinPool-1-worker-0]downloaded: http://109.imagebam.com/download/aq9YOoslW_bxMZaSYDr6cw/33156/331550323/IMG_20140607_0006.jpg to D:\saved from vef\t269539-Carmen_new_content_Mystery_Corner\t269539-p1-Carmen_new_content_Mystery_Corner-post-1-2930106\IMG_20140607_0006.jpg 2017-06-20 20:50:55 INFO: [ForkJoinPool-1-worker-0]image-page: http://www.imagebam.com/image/0775c5331550426 2017-06-20 20:50:55 INFO: [ForkJoinPool-1-worker-0]http://www.imagebam.com/image/0775c5331550426 is already processed 2017-06-20 20:50:55 INFO: [ForkJoinPool-1-worker-0]image-page: http://www.imagebam.com/image/dee561331550512 2017-06-20 20:51:12 SEVERE: [ForkJoinPool-1-worker-1] Error downloading 'http://www.imagebam.com/image/dee561331550512' java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at org.apache.http.impl.conn.LoggingInputStream.read(LoggingInputStream.java:84) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153) at org.apache.http.impl.io.SessionInputBufferImpl.read(SessionInputBufferImpl.java:206) at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:189) at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135) at java.util.zip.InflaterInputStream.fill(Unknown Source) at java.util.zip.InflaterInputStream.read(Unknown Source) at java.util.zip.GZIPInputStream.read(Unknown Source) at org.apache.http.client.entity.LazyDecompressingInputStream.read(LazyDecompressingInputStream.java:70) at sun.nio.cs.StreamDecoder.readBytes(Unknown Source) at sun.nio.cs.StreamDecoder.implRead(Unknown Source) at sun.nio.cs.StreamDecoder.read(Unknown Source) at java.io.InputStreamReader.read(Unknown Source) at java.io.Reader.read(Unknown Source) at org.apache.http.util.EntityUtils.toString(EntityUtils.java:225) at org.apache.http.util.EntityUtils.toString(EntityUtils.java:306) at vef.imgrescue.ImageBam.extractImageURL(ImageBam.java:25) at vef.imgrescue.AbstactImageHost.download(AbstactImageHost.java:56) at vef.imgrescue.ImageLinkProcessor.lambda$downloadImages$8(ImageLinkProcessor.java:121) at java.util.ArrayList.forEach(Unknown Source) at vef.imgrescue.ImageLinkProcessor.downloadImages(ImageLinkProcessor.java:105) at vef.imgrescue.ImageLinkProcessor.lambda$null$1(ImageLinkProcessor.java:56) at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source) at java.util.TreeMap$EntrySpliterator.forEachRemaining(Unknown Source) at java.util.stream.AbstractPipeline.copyInto(Unknown Source) at java.util.stream.ForEachOps$ForEachTask.compute(Unknown Source) at java.util.concurrent.CountedCompleter.exec(Unknown Source) at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) at java.util.concurrent.ForkJoinTask.doInvoke(Unknown Source) at java.util.concurrent.ForkJoinTask.invoke(Unknown Source) at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(Unknown Source) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(Unknown Source) at java.util.stream.AbstractPipeline.evaluate(Unknown Source) at java.util.stream.ReferencePipeline.forEach(Unknown Source) at java.util.stream.ReferencePipeline$Head.forEach(Unknown Source) at vef.imgrescue.ImageLinkProcessor.lambda$processForumPages$2(ImageLinkProcessor.java:42) at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(Unknown Source) at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(Unknown Source) at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) 2017-06-20 20:51:12 INFO: [ForkJoinPool-1-worker-1]image-page: RETRY http://www.imagebam.com/image/ae0526331550076 2017-06-20 20:51:33 INFO: 1 threads specified: [269539] 2017-06-20 20:51:33 INFO: Starting Thread: 269539 2017-06-20 20:51:34 INFO: HTTP/1.1 200 OK 2017-06-20 20:51:34 INFO: IDstack cookie found. Successfully logged in to VEF 2017-06-20 20:51:34 INFO: Using existing target folder :D:\saved from vef\t269539-Carmen_new_content_Mystery_Corner 2017-06-20 20:51:34 INFO: Download forum page: http://vintage-erotica-forum.com/t269539-p1-x.html to D:\saved from vef\t269539-Carmen_new_content_Mystery_Corner\t269539-p1-Carmen_new_content_Mystery_Corner.html 2017-06-20 20:51:35 INFO: End of Thread reached - no next page link found - 2017-06-20 20:51:43 INFO: 1 threads specified: [269539] 2017-06-20 20:51:43 INFO: Starting Thread: 269539 2017-06-20 20:51:44 INFO: HTTP/1.1 200 OK 2017-06-20 20:51:44 INFO: IDstack cookie found. Successfully logged in to VEF 2017-06-20 20:51:44 INFO: Using existing target folder :D:\saved from vef\t269539-Carmen_new_content_Mystery_Corner 2017-06-20 20:51:44 INFO: Download forum page: http://vintage-erotica-forum.com/t269539-p1-x.html to D:\saved from vef\t269539-Carmen_new_content_Mystery_Corner\t269539-p1-Carmen_new_content_Mystery_Corner.html 2017-06-20 20:51:44 INFO: End of Thread reached - no next page link found - 2017-06-20 20:51:53 INFO: Usage without GUI: java -jar vef-imagerescue.jar thread username password 2017-06-20 20:52:10 INFO: 1 threads specified: [269539] 2017-06-20 20:52:10 INFO: Starting Thread: 269539 2017-06-20 20:52:11 INFO: HTTP/1.1 200 OK 2017-06-20 20:52:11 INFO: IDstack cookie found. Successfully logged in to VEF 2017-06-20 20:52:11 INFO: Using existing target folder :D:\saved from vef\t269539-Carmen_new_content_Mystery_Corner 2017-06-20 20:52:11 INFO: Download forum page: http://vintage-erotica-forum.com/t269539-p1-x.html to D:\saved from vef\t269539-Carmen_new_content_Mystery_Corner\t269539-p1-Carmen_new_content_Mystery_Corner.html 2017-06-20 20:52:12 INFO: End of Thread reached - no next page link found - 2017-06-20 20:52:12 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/dcbf03331549696 2017-06-20 20:52:12 INFO: [ForkJoinPool-1-worker-1]http://www.imagebam.com/image/dcbf03331549696 is already processed 2017-06-20 20:52:12 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/500b8d331549812 2017-06-20 20:52:12 INFO: [ForkJoinPool-1-worker-1]http://www.imagebam.com/image/500b8d331549812 is already processed 2017-06-20 20:52:12 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/42f53a331549922 2017-06-20 20:52:12 INFO: [ForkJoinPool-1-worker-1]http://www.imagebam.com/image/42f53a331549922 is already processed 2017-06-20 20:52:12 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/ae0526331550076 2017-06-20 20:52:12 INFO: [ForkJoinPool-1-worker-1]http://www.imagebam.com/image/ae0526331550076 is already processed 2017-06-20 20:52:12 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/62bac8331550167 2017-06-20 20:52:12 INFO: [ForkJoinPool-1-worker-1]http://www.imagebam.com/image/62bac8331550167 is already processed 2017-06-20 20:52:12 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/e45d0d331550323 2017-06-20 20:52:12 INFO: [ForkJoinPool-1-worker-1]http://www.imagebam.com/image/e45d0d331550323 is already processed 2017-06-20 20:52:12 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/0775c5331550426 2017-06-20 20:52:12 INFO: [ForkJoinPool-1-worker-1]http://www.imagebam.com/image/0775c5331550426 is already processed 2017-06-20 20:52:12 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/dee561331550512 2017-06-20 20:52:12 INFO: [ForkJoinPool-1-worker-1]http://www.imagebam.com/image/dee561331550512 is already processed 2017-06-20 20:52:12 INFO: FINISHED! 2017-06-20 20:52:12 INFO: Finished Thread: 269539 2017-06-20 20:53:24 INFO: NO DEADLOCK 2017-06-20 20:54:54 INFO: NO DEADLOCK Code:
2017-06-20 21:02:12 INFO: Starting Thread: 307745 2017-06-20 21:02:13 INFO: HTTP/1.1 200 OK 2017-06-20 21:02:13 INFO: IDstack cookie found. Successfully logged in to VEF 2017-06-20 21:02:13 INFO: Download first thread page: http://vintage-erotica-forum.com/t307745-p1-x.html 2017-06-20 21:02:14 INFO: HTTP/1.1 200 OK 2017-06-20 21:02:14 INFO: Thread name: Mystery_Followup_Who_is_Maija_in_Scandinavian_Gir 2017-06-20 21:02:14 INFO: Download forum page: http://vintage-erotica-forum.com/t307745-p1-x.html to D:\saved from vef\t307745-Mystery_Followup_Who_is_Maija_in_Scandinavian_Gir\t307745-p1-Mystery_Followup_Who_is_Maija_in_Scandinavian_Gir.html 2017-06-20 21:02:15 INFO: End of Thread reached - no next page link found - 2017-06-20 21:02:15 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/8670c8281124113 2017-06-20 21:02:19 INFO: [ForkJoinPool-1-worker-1]downloaded: http://106.imagebam.com/download/1xTN4AZCFgt2jMST-bizEw/28113/281124113/newdoc482%20-%20Copy%20%282%29.jpg to D:\saved from vef\t307745-Mystery_Followup_Who_is_Maija_in_Scandinavian_Gir\t307745-p1-Mystery_Followup_Who_is_Maija_in_Scandinavian_Gir-post-1-2984985\newdoc482 - Copy (2).jpg 2017-06-20 21:02:19 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/bed0d4281124155 2017-06-20 21:02:24 INFO: NO DEADLOCK 2017-06-20 21:02:39 SEVERE: [ForkJoinPool-1-worker-1] Error downloading 'http://www.imagebam.com/image/bed0d4281124155' java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at org.apache.http.impl.conn.LoggingInputStream.read(LoggingInputStream.java:84) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282) at org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:264) at org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:225) at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:184) at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135) at java.util.zip.InflaterInputStream.fill(Unknown Source) at java.util.zip.InflaterInputStream.read(Unknown Source) at java.util.zip.GZIPInputStream.read(Unknown Source) at org.apache.http.client.entity.LazyDecompressingInputStream.read(LazyDecompressingInputStream.java:70) at sun.nio.cs.StreamDecoder.readBytes(Unknown Source) at sun.nio.cs.StreamDecoder.implRead(Unknown Source) at sun.nio.cs.StreamDecoder.read(Unknown Source) at java.io.InputStreamReader.read(Unknown Source) at java.io.Reader.read(Unknown Source) at org.apache.http.util.EntityUtils.toString(EntityUtils.java:225) at org.apache.http.util.EntityUtils.toString(EntityUtils.java:306) at vef.imgrescue.ImageBam.extractImageURL(ImageBam.java:25) at vef.imgrescue.AbstactImageHost.download(AbstactImageHost.java:56) at vef.imgrescue.ImageLinkProcessor.lambda$downloadImages$8(ImageLinkProcessor.java:121) at java.util.ArrayList.forEach(Unknown Source) at vef.imgrescue.ImageLinkProcessor.downloadImages(ImageLinkProcessor.java:105) at vef.imgrescue.ImageLinkProcessor.lambda$null$1(ImageLinkProcessor.java:56) at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source) at java.util.TreeMap$EntrySpliterator.forEachRemaining(Unknown Source) at java.util.stream.AbstractPipeline.copyInto(Unknown Source) at java.util.stream.ForEachOps$ForEachTask.compute(Unknown Source) at java.util.concurrent.CountedCompleter.exec(Unknown Source) at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) at java.util.concurrent.ForkJoinTask.doInvoke(Unknown Source) at java.util.concurrent.ForkJoinTask.invoke(Unknown Source) at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(Unknown Source) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(Unknown Source) at java.util.stream.AbstractPipeline.evaluate(Unknown Source) at java.util.stream.ReferencePipeline.forEach(Unknown Source) at java.util.stream.ReferencePipeline$Head.forEach(Unknown Source) at vef.imgrescue.ImageLinkProcessor.lambda$processForumPages$2(ImageLinkProcessor.java:42) at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(Unknown Source) at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(Unknown Source) at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) 2017-06-20 21:02:39 INFO: [ForkJoinPool-1-worker-1]image-page: http://www.imagebam.com/image/3a7762281124166 2017-06-20 21:03:00 SEVERE: [ForkJoinPool-1-worker-1] Error downloading 'http://www.imagebam.com/image/3a7762281124166' java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at org.apache.http.impl.conn.LoggingInputStream.read(LoggingInputStream.java:84) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282) at org.apache.http.impl.io.ChunkedInputStream.getChunkSize(ChunkedInputStream.java:264) at org.apache.http.impl.io.ChunkedInputStream.nextChunk(ChunkedInputStream.java:225) at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:184) at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135) at java.util.zip.InflaterInputStream.fill(Unknown Source) at java.util.zip.InflaterInputStream.read(Unknown Source) at java.util.zip.GZIPInputStream.read(Unknown Source) at org.apache.http.client.entity.LazyDecompressingInputStream.read(LazyDecompressingInputStream.java:70) at sun.nio.cs.StreamDecoder.readBytes(Unknown Source) at sun.nio.cs.StreamDecoder.implRead(Unknown Source) at sun.nio.cs.StreamDecoder.read(Unknown Source) at java.io.InputStreamReader.read(Unknown Source) at java.io.Reader.read(Unknown Source) at org.apache.http.util.EntityUtils.toString(EntityUtils.java:225) at org.apache.http.util.EntityUtils.toString(EntityUtils.java:306) at vef.imgrescue.ImageBam.extractImageURL(ImageBam.java:25) at vef.imgrescue.AbstactImageHost.download(AbstactImageHost.java:56) at vef.imgrescue.ImageLinkProcessor.lambda$downloadImages$8(ImageLinkProcessor.java:121) at java.util.ArrayList.forEach(Unknown Source) at vef.imgrescue.ImageLinkProcessor.downloadImages(ImageLinkProcessor.java:105) at vef.imgrescue.ImageLinkProcessor.lambda$null$1(ImageLinkProcessor.java:56) at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source) at java.util.TreeMap$EntrySpliterator.forEachRemaining(Unknown Source) at java.util.stream.AbstractPipeline.copyInto(Unknown Source) at java.util.stream.ForEachOps$ForEachTask.compute(Unknown Source) at java.util.concurrent.CountedCompleter.exec(Unknown Source) at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) at java.util.concurrent.ForkJoinTask.doInvoke(Unknown Source) at java.util.concurrent.ForkJoinTask.invoke(Unknown Source) at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(Unknown Source) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(Unknown Source) at java.util.stream.AbstractPipeline.evaluate(Unknown Source) at java.util.stream.ReferencePipeline.forEach(Unknown Source) at java.util.stream.ReferencePipeline$Head.forEach(Unknown Source) at vef.imgrescue.ImageLinkProcessor.lambda$processForumPages$2(ImageLinkProcessor.java:42) at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(Unknown Source) at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(Unknown Source) at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) 2017-06-20 21:03:00 INFO: [ForkJoinPool-1-worker-1]image-page: RETRY http://www.imagebam.com/image/bed0d4281124155 2017-06-20 21:03:20 SEVERE: [ForkJoinPool-1-worker-1] Error downloading 'http://www.imagebam.com/image/bed0d4281124155' java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at org.apache.http.impl.conn.LoggingInputStream.read(LoggingInputStream.java:84) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153) at org.apache.http.impl.io.SessionInputBufferImpl.read(SessionInputBufferImpl.java:206) at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInputStream.java:189) at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135) at java.util.zip.InflaterInputStream.fill(Unknown Source) at java.util.zip.InflaterInputStream.read(Unknown Source) at java.util.zip.GZIPInputStream.read(Unknown Source) at org.apache.http.client.entity.LazyDecompressingInputStream.read(LazyDecompressingInputStream.java:70) at sun.nio.cs.StreamDecoder.readBytes(Unknown Source) at sun.nio.cs.StreamDecoder.implRead(Unknown Source) at sun.nio.cs.StreamDecoder.read(Unknown Source) at java.io.InputStreamReader.read(Unknown Source) at java.io.Reader.read(Unknown Source) at org.apache.http.util.EntityUtils.toString(EntityUtils.java:225) at org.apache.http.util.EntityUtils.toString(EntityUtils.java:306) at vef.imgrescue.ImageBam.extractImageURL(ImageBam.java:25) at vef.imgrescue.AbstactImageHost.download(AbstactImageHost.java:56) at vef.imgrescue.ImageLinkProcessor.lambda$downloadImages$8(ImageLinkProcessor.java:121) at java.util.ArrayList.forEach(Unknown Source) at vef.imgrescue.ImageLinkProcessor.downloadImages(ImageLinkProcessor.java:105) at vef.imgrescue.ImageLinkProcessor.lambda$null$3(ImageLinkProcessor.java:62) at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source) at java.util.HashMap$EntrySpliterator.forEachRemaining(Unknown Source) at java.util.stream.AbstractPipeline.copyInto(Unknown Source) at java.util.stream.ForEachOps$ForEachTask.compute(Unknown Source) at java.util.concurrent.CountedCompleter.exec(Unknown Source) at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) at java.util.concurrent.ForkJoinTask.doInvoke(Unknown Source) at java.util.concurrent.ForkJoinTask.invoke(Unknown Source) at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(Unknown Source) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(Unknown Source) at java.util.stream.AbstractPipeline.evaluate(Unknown Source) at java.util.stream.ReferencePipeline.forEach(Unknown Source) at java.util.stream.ReferencePipeline$Head.forEach(Unknown Source) at vef.imgrescue.ImageLinkProcessor.lambda$processForumPages$4(ImageLinkProcessor.java:62) at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(Unknown Source) at java.util.concurrent.ForkJoinTask.doExec(Unknown Source) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(Unknown Source) at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source) at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source) 2017-06-20 21:03:20 INFO: [ForkJoinPool-1-worker-1]image-page: RETRY http://www.imagebam.com/image/3a7762281124166 2017-06-20 21:03:28 INFO: [ForkJoinPool-1-worker-1]downloaded: http://102.imagebam.com/download/I_k7Q-q_b-eCralKtZrYyQ/28113/281124166/newdoc482%20-%20Copy%20%284%29.jpg to D:\saved from vef\t307745-Mystery_Followup_Who_is_Maija_in_Scandinavian_Gir\t307745-p1-Mystery_Followup_Who_is_Maija_in_Scandinavian_Gir-post-1-2984985\newdoc482 - Copy (4).jpg 2017-06-20 21:03:28 INFO: FINISHED! 2017-06-20 21:03:28 INFO: Finished Thread: 307745 2017-06-20 21:03:54 INFO: NO DEADLOCK 2017-06-20 21:05:24 INFO: NO DEADLOCK 2017-06-20 21:06:54 INFO: NO DEADLOCK 2017-06-20 21:08:24 INFO: NO DEADLOCK 2017-06-20 21:09:54 INFO: NO DEADLOCK 2017-06-20 21:11:24 INFO: NO DEADLOCK 2017-06-20 21:12:54 INFO: NO DEADLOCK 2017-06-20 21:14:24 INFO: NO DEADLOCK 2017-06-20 21:15:54 INFO: NO DEADLOCK 2017-06-20 21:17:24 INFO: NO DEADLOCK Likewise with the second thread above. Your tool "bailed out" on the second of three images in that post. What troubles me, and maybe it's only me with my problem with "sticky" images, is that before I could tell your tool was having a problem with a given thread because its download had not completed. Above they were completed, so the only warning I have is: a) scrolling back through the output window to check for errors or b) comparing the no. of images downloaded to disk vs. those shown in the browser window for the post. Both are tedious manual "checks" to have to perform, compared to the previous version of your tool. For someone like deezer who is working through the many MIR queries (mostly very small threads), he's been able to chuck 20-50 threads at your tool and come back after, say, a half hour. The risk with v.1.22, at least in conditions like mine, are that there will be far more comparatively "silent" errors (not literally, because they're in the output window, but maybe more difficult to spot there?) that we humans won't easily catch? Maybe it is only me, in which case others can/should carry on with 1.22 whereas I perhaps should revert to 1.21? |
The Following 11 Users Say Thank You to effCup For This Useful Post: |
June 20th, 2017, 10:07 AM | #120 |
Vintage Idiot
Join Date: Feb 2012
Location: History
Posts: 22,143
Thanks: 226,791
Thanked 357,042 Times in 21,638 Posts
|
A further stray thought: @halvar, does your tool do anything to handle image filename clashes? I'm afraid that some posters don't give their scans unique filenames, so there's the potential for 2+ different scans to have the same filename.
I haven't noticed any such issues so maybe you've got it already sussed. I hadn't thought of the problem until just now. |
The Following 10 Users Say Thank You to effCup For This Useful Post: |
|
|