Register on the forum now to remove ALL ads + popups + get access to tons of hidden content for members only!
vintage erotica forum vintage erotica forum vintage erotica forum
vintage erotica forum

Go Back   Vintage Erotica Forums > Information & Help Forum > Help Section > Imagebam/Imgbox are no longer shutting down

Follow Vintage Erotica Forum on Twitter
Best Porn Sites Meet Our Girls Register FAQ Members List Calendar Mark Forums Read

Notices
Imagebam/Imgbox are no longer shutting down The day has been saved, the sites are not shutting, more info soon.


Reply
 
Thread Tools Display Modes
Old 07-23-2017, 04:15 PM   #171
halvar
Veteran Member
 
halvar's Avatar
 
Join Date: Jan 2008
Location: HH
Posts: 309
Thanks: 9,972
Thanked 3,581 Times in 303 Posts
halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+
Default Is the post date format different or same for all users?

I would like to parse and store the post date. I think the date format is the same for everyone, but I am not really sure. It could depend on system settings or browser language.

Does it look same to you?



My parse pattern is like this:
Code:
MM-dd-yyyy, hh:mm a
The parse pattern will be configurable. But if it is something fancy like '32 minutes ago' or 'a couple of hours ago', a simple configurable pattern will not suffice.

Just hit the thanks button if it is the same for you, reply if otherwise.

Thanks
halvar
halvar is offline   Reply With Quote
The Following 10 Users Say Thank You to halvar For This Useful Post:
Old 07-23-2017, 04:34 PM   #172
40plus
Moderator
 
Join Date: Aug 2007
Location: Offshore
Posts: 1,723
Thanks: 6,565
Thanked 15,342 Times in 1,663 Posts
40plus 50000+40plus 50000+40plus 50000+40plus 50000+40plus 50000+40plus 50000+40plus 50000+40plus 50000+40plus 50000+40plus 50000+40plus 50000+
Default

I believe the date format is the same for everyone and is USA (because a number of very respected members from that region could not cope with DD/MM). However, there is an option for members to change the time display to any time zone they choose. Assuming the script/code parses what is actually displayed then it will not be the same for everyone (ie two people could run the script and get different results depending on their settings).

Regards

40+
__________________
Like to be fondling.............Joanna Gosling
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
.
40plus is offline   Reply With Quote
The Following 8 Users Say Thank You to 40plus For This Useful Post:
Old 07-23-2017, 04:59 PM   #173
effCup
Vintage Idiot
 
effCup's Avatar
 
Join Date: Feb 2012
Location: Dildo Baggins' backyard
Posts: 14,669
Thanks: 162,476
Thanked 168,353 Times in 14,521 Posts
effCup 750000+effCup 750000+effCup 750000+effCup 750000+effCup 750000+effCup 750000+effCup 750000+effCup 750000+effCup 750000+effCup 750000+effCup 750000+
Default

Personally I would prefer: yyyy-mm-dd hh:mm AM as then there'd be no room for confusion, but vef/vBulletin determines that I must instead use: mm-dd-yyyy, hh:mm AM

See vef Control Panel -> Edit Options -> Date & Time Options
where time zone can be set & daylight savings time applied.
effCup is offline   Reply With Quote
The Following 9 Users Say Thank You to effCup For This Useful Post:
Old 07-23-2017, 05:07 PM   #174
halvar
Veteran Member
 
halvar's Avatar
 
Join Date: Jan 2008
Location: HH
Posts: 309
Thanks: 9,972
Thanked 3,581 Times in 303 Posts
halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+
Default

Quote:
Originally Posted by 40plus View Post
However, there is an option for members to change the time display to any time zone they choose. Assuming the script/code parses what is actually displayed then it will not be the same for everyone (ie two people could run the script and get different results depending on their settings).
Thanks for the hint. The different value for different users would not bother me. The important thing is that the parsing works.

The date is merely used as information. And to determine if the backup of a thread should be started. The longer the time since the last post, the longer the time to wait before backing it up.
halvar is offline   Reply With Quote
The Following 10 Users Say Thank You to halvar For This Useful Post:
Old 07-29-2017, 08:08 AM   #175
halvar
Veteran Member
 
halvar's Avatar
 
Join Date: Jan 2008
Location: HH
Posts: 309
Thanks: 9,972
Thanked 3,581 Times in 303 Posts
halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+
Default forum-backup 0.0.3 is released

get it here: https://1fichier.com/?vssj7pbs3p

howto install and use is documented here: https://git.teknik.io/halvar/forum-backup/wiki/Howto
release notes are here: https://git.teknik.io/halvar/forum-b...i/ReleaseNotes
additional documentation is here: https://git.teknik.io/halvar/forum-backup/wiki

Highlight in this version is the simple web console.

It looks just like the fist html page I made 20 years ago, but it works

Please backup your data (storage folder) before upgrading!
halvar is offline   Reply With Quote
The Following 9 Users Say Thank You to halvar For This Useful Post:
Old 10-21-2017, 10:28 AM   #176
halvar
Veteran Member
 
halvar's Avatar
 
Join Date: Jan 2008
Location: HH
Posts: 309
Thanks: 9,972
Thanked 3,581 Times in 303 Posts
halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+
Default forum-backup 0.0.4 is released

get it here: https://1fichier.com/?c5idd5kqmb

This version fixes a nasty bug concerning some ghost posts:
If a ghost post is encountered at the beginning of a page the page is downloaded in an infinite loop.
Thanks to Denaniel for pointing this out.

howto install and use is documented here: https://git.teknik.io/halvar/forum-backup/wiki/Howto
release notes are here: https://git.teknik.io/halvar/forum-b...i/ReleaseNotes
additional documentation is here: https://git.teknik.io/halvar/forum-backup/wiki

Please backup your data (storage folder) before upgrading!
halvar is offline   Reply With Quote
The Following 11 Users Say Thank You to halvar For This Useful Post:
Old 11-03-2017, 10:10 PM   #177
halvar
Veteran Member
 
halvar's Avatar
 
Join Date: Jan 2008
Location: HH
Posts: 309
Thanks: 9,972
Thanked 3,581 Times in 303 Posts
halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+
Default forum-backup 0.0.5 is released

get it here: https://1fichier.com/?s76xzf0ry6

I moved the project to gitlab:
https://gitlab.com/halvar/forum-backup

How It Works: https://gitlab.com/halvar/forum-backup/wikis/home
Howto: https://gitlab.com/halvar/forum-backup/wikis/howto
ReleaseNotes: https://gitlab.com/halvar/forum-back.../Release-Notes


Major Changes:
  • forum backup always starts the web console
  • the csv file for thread configuration is no longer used. Threads are configured in the database and can be added using the web console (see [Howto]).
  • progress bar is improved
  • local page file names and image folders are created with shorter names:
    t[thread-id]-p[pageno].html for html files
    t[thread-id]-p[pageno]-post-[postcount]-[postnr] for post folders.

As always: Please backup your data (storage folder) before upgrading!
halvar is offline   Reply With Quote
The Following 7 Users Say Thank You to halvar For This Useful Post:
Old 11-04-2017, 05:01 PM   #178
halvar
Veteran Member
 
halvar's Avatar
 
Join Date: Jan 2008
Location: HH
Posts: 309
Thanks: 9,972
Thanked 3,581 Times in 303 Posts
halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+
Default Future Plans for forum-backup

I will continue to improve it gradually. If there is something that could make things easier for you give me a note. Often little things achieve much.
Of course bugs have a priority if you find one, drop me a line and I will look into it.

Long term topics are:
Duplicate Detection
  • adding an MD5 hash for each image to the database
  • try image finger printing to find duplicates that have the same content but different resolution or quality. This is something that interests me from a mathematical point of view. Maybe http://phash.org/ could be of use here.
Dealing with edited/updated posts
I do not know of a way to specifically search or get a list of updated posts. So one way to deal with it could be
  • rename the old thread page files with a date-suffix like 't1234-p3-x.html.20171104
  • re-download all thread pages (this is something that should be avoided)
  • parse the posts again and compare the result.
  • If the result (imagelinks) differs then
  • add the links to the database, but with an offset at the index to distinguish them from the original links. like 1001, 1002, ...
  • an alternative to the offset could be storing thread edit date with the links
  • download the new links. This would probably lead to duplicates, but I want to keep the original file.
  • I could do an md5 or content compare of old and new files and delete the new files if they are identical. But what if the new files have nicer names? So I would rather keep all the files. Having is better than needing ;-)
  • Since this is a lengthy process that also causes load on the server this process shoul be started rarely (once a year per thread?). Or only started manually if one needs it.
  • Do edits matter anyway?
Dealing Deleted Posts
Last week a couple of pages were deleted from a thread due to a DMCA takedown:
http://vintage-erotica-forum.com/sho...&postcount=718
http://vintage-erotica-forum.com/t58...n-phoenix.html

So locally where pages 1 to 9, on the server only 1 to 6. If new posts where added the pages 6-8 would not have been downloaded again.

A workaround for this is
  • renaming page files 6 to 9 (append a date suffix like t1234-p6-x.html.2011104)
  • force the processing of the thread during the next run. This can be achieved by updating the thread using the webconsole without changing the page no. This deletes the last process date of the thread and the thread is processed during the next run.


I am planning to do some of this in my week off work in January.
halvar is offline   Reply With Quote
The Following 7 Users Say Thank You to halvar For This Useful Post:
Old 11-04-2017, 11:06 PM   #179
deezer
Veteran Member
 
deezer's Avatar
 
Join Date: Jan 2016
Posts: 1,016
Thanks: 7,864
Thanked 7,737 Times in 1,168 Posts
deezer 25000+deezer 25000+deezer 25000+deezer 25000+deezer 25000+deezer 25000+deezer 25000+deezer 25000+deezer 25000+deezer 25000+deezer 25000+
Default

Does the forum-backup detect merges?

E.g., this thread was originally started by SIT (now post #4).
Later were three posts of Jmailman merged into the thread, because
they were posted in the LKMM-thread, but belongs to this thread.
As Jmailman's post have a lower post-id, they are now on top of the thread.

If you now would parse the thread after the merge there would be no update detected,
because forum-backup parses the thread for the latest update.
Therefore the new added/merged posts wouldn't be saved, right?

If the above is correct, wouldn't it make sense, to compare the whole thread using the
post-id's, to detect updates, or would this be to time consuming?

Wouldn't this procedure solve the problem with Deleted Posts too?
__________________
.

Check this sub-forum:


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.
deezer is offline   Reply With Quote
The Following 6 Users Say Thank You to deezer For This Useful Post:
Old 11-05-2017, 04:04 PM   #180
halvar
Veteran Member
 
halvar's Avatar
 
Join Date: Jan 2008
Location: HH
Posts: 309
Thanks: 9,972
Thanked 3,581 Times in 303 Posts
halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+halvar 10000+
Default

I hesitate parsing whole threads. For a few threads it could be done. But it is too much if there are hundreds of threads configured with tens or hundreds of pages each.

A solution could be:
  • compare the post numbers on the last local page with the last thread page on the server
  • if the page does not exist on the server or if local posts are missing on the server's version of the page:
    move backwards to the previous and repeat comparison until a page is found where all posts match
    download all pages from this page forward.
Posts and links are stored in the database using the post number as key. A post would not be overwritten if it is found on another page.
But the page no, which is stored with the post, should be updated.

addendum:
But on problem persists: a thread is only processed, if the change results in an updated time stamp on the thread list:

Last edited by halvar; 11-05-2017 at 04:13 PM.. Reason: fixed typo, addendum
halvar is offline   Reply With Quote
The Following 6 Users Say Thank You to halvar For This Useful Post:
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump




All times are GMT. The time now is 12:12 AM.






vBulletin Optimisation provided by vB Optimise v2.6.1 (Pro) - vBulletin Mods & Addons Copyright © 2018 DragonByte Technologies Ltd.