Discussion of the download / upload tools

[halvar]

I have created a new version of the forum backup tool:
Version 0.0.18, https://1fichier.com/?p5izqj8leqw5m3c3jqjt

• usability is improved, editing of files is not needed anymore
• a bug was fixed that prevented the generation of reports under Windows
• there are three download modes supported: single post, threads, backup

I do not know if anybody is still interested. If not, I am not disappointed.
The tool is mainly created for my personal use. It would be nice somebody else finds it useful. But if not then not. I know that it is not very easy to use.

What does it do

• download pages from a forum, threads or posts
• store the information about these in an embedded database
• parse posts for links to image hosts
• download linked images

About

• this is a java program. The program requires a Java8 runtime environment
• the distributed zip contains a JRE for Windows
• it is developed on Linux, but it also run on Windows.
• I personally use it on Linux and only test it briefly under Windows
• configuration values are stored by default under <user home>/forum-backup/forum-backup.properties
• there are log files in <user home>/forum-backup/logs
• downloaded files and the embedded database are stored in configured location
• do not alter your thread display options like display mode and number of posts per page in your forum settings. It would mess up the downloaded pages.

Upgrade
New versions pick up settings and data from previous versions.

• unzip zip file
• start program
• goto settings and check configuration values
• do a check configuration

New Installation
unzip the zip file anywhere

start forum-backup.bat, e.g. by double click. A console window opens

open the url http://localhost:3137/ with your browser

click on the settings page link and enter your values

click on 'Check Configuration'. The output should look like this (every line starting with 'OK')

click on 'Save Configuration' to save the configuration. The configuration values are saved to <user-home>/forum-backup/forum-backup.properties.
e.g. 'C:\Users\<your login name>\forum-backup\forum-backup.properties' depending on your operating system.

Click on the 'Back to main view' link and start with downloading a single post

There are tree ways to download
posts
Only posts and linked images are downloaded. Nothing is stored in the database.
Enter one or more post numbers and click on 'Download Post'

The images are saved to <storage-path>/adhoc-posts/<post-folder>, e.g. 'C:\data\forum-storage\adhoc-posts\Judith_Ramirez-post-2230617'


threads
One or more threads are downloaded. For each thread

• thread pages are downloaded if they are not downloaded yet
• the last page of a thread is always downloaded, because it could contain new posts.
• the downloaded pages are parsed for posts and image links
• this information is stored in the database
• image links that have not been downloaded yet are downloaded
• failed image downloads are marked for retrying them later
• previously failed image downloads are retried

enter one ore more threads with optional start page and click 'Download Threads'

during the download a simple progress bar is shown

downloaded files are stored at the storage location:


thread backup

• for all active threads configured in the database the forum is determined
• forum pages are downloaded for these until a the last forum scan date is reached (maximum 14 days)
• if a thread was modified since that date it is downloaded
• due to the overhead of checking for modified threads it is only economically if you have many threads (> 100) configured and if the process is started regularly, e.g. once a day.