Register on the forum now to remove ALL ads + popups + get access to tons of hidden content for members only!
vintage erotica forum vintage erotica forum vintage erotica forum
vintage erotica forum

Go Back   Vintage Erotica Forums > Information & Help Forum > Help Section > Tutorials

Follow Vintage Erotica Forum on Twitter
Best Porn Sites Meet Our Girls Register FAQ Members List Calendar Mark Forums Read

Notices
Tutorials Step by step Guides and How to's with screengrabs.


View Poll Results: Is this tutorial usefull
Yes 8 61.54%
No, don't care about duplicates 4 30.77%
No, hard to follow. 1 7.69%
Multiple Choice Poll. Voters: 13. You may not vote on this poll

Reply
 
Thread Tools Display Modes
Old 12-13-2013, 12:46 PM   #11
Misrule
Vintage Member
 
Join Date: Jan 2008
Posts: 443
Thanks: 12,275
Thanked 12,690 Times in 762 Posts
Misrule 50000+Misrule 50000+Misrule 50000+Misrule 50000+Misrule 50000+Misrule 50000+Misrule 50000+Misrule 50000+Misrule 50000+Misrule 50000+Misrule 50000+
Default

For finding duplicate images, I use VisiPics. It's a great little program and will crunch through thousands of files identifying the ones that are identical. It can also be set to identify images that are similar and many other options.
And, once it has identified the duplicates, you can go through them and decide which one(s) to delete and which one to keep. All within the program. Thoroughly recommend it.
Misrule is offline   Reply With Quote
The Following 3 Users Say Thank You to Misrule For This Useful Post:
Old 12-14-2013, 02:50 AM   #12
DTravel
InterDimensional Traveler
 
DTravel's Avatar
 
Join Date: Apr 2008
Location: Between universes, lost in another TV series project
Posts: 11,469
Thanks: 8,504
Thanked 101,980 Times in 11,341 Posts
DTravel 500000+DTravel 500000+DTravel 500000+DTravel 500000+DTravel 500000+DTravel 500000+DTravel 500000+DTravel 500000+DTravel 500000+DTravel 500000+DTravel 500000+
Default

I use SimilarImages, have been for many years. I'm not sure where you could get a copy since I think the programmer isn't working on it anymore. I think you can find info on it in the Useful Programs thread.
__________________

To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.

If you have a request for reupping any of my screencaps, please send a PM.
DTravel is offline   Reply With Quote
The Following User Says Thank You to DTravel For This Useful Post:
Old 12-14-2013, 11:28 PM   #13
Al Gebra
Former Staff
 
Al Gebra's Avatar
 
Join Date: Jun 2007
Location: Germany
Posts: 11,692
Thanks: 19,208
Thanked 475,364 Times in 10,773 Posts
Al Gebra 1000000+Al Gebra 1000000+Al Gebra 1000000+Al Gebra 1000000+Al Gebra 1000000+Al Gebra 1000000+Al Gebra 1000000+Al Gebra 1000000+Al Gebra 1000000+Al Gebra 1000000+Al Gebra 1000000+
Default

Quote:
Originally Posted by karloschaos View Post
Highly recommended by me too.
__________________
m
Please add source, post complete photo and scan sets - with indexes, if available, preserve
genuine file names (that will help to ID sources and model names), thank, credit, and quote original posters.

I don't have time to spare for reuploads. Don't send reports (or PMs) of dead files or requests, please!
Once the files posted above are expired, please help each other, add the info I provided as well.
To view links or images in signatures your post count must be 0 or greater. You currently have 0 posts.

-> Underlined words in my posts are clickable. <-
Al Gebra is offline   Reply With Quote
The Following 2 Users Say Thank You to Al Gebra For This Useful Post:
Old 09-09-2014, 11:40 AM   #14
electile disfunction
Vintage Member
 
Join Date: Oct 2008
Location: Somewhere flat, that's either hot, cold, or windy ... Canada?
Posts: 1,758
Thanks: 39,573
Thanked 15,874 Times in 1,731 Posts
electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+
Default Anti-Dupl.net

I have been using Anti-Dupl.net for quite some time now.

It makes and stores databases of your collection(s), finds similarities & differences, remembers false positives, is very fast, etc.
http://antidupl.sourceforge.net/english/index.html

e.d.

P.S.: It is free & open source, too.

Last edited by electile disfunction; 09-10-2014 at 02:52 PM.. Reason: added P.S.
electile disfunction is offline   Reply With Quote
Old 09-15-2014, 09:13 AM   #15
Artsey
Member
 
Join Date: Aug 2011
Posts: 65
Thanks: 5,978
Thanked 369 Times in 64 Posts
Artsey 1000+Artsey 1000+Artsey 1000+Artsey 1000+Artsey 1000+Artsey 1000+Artsey 1000+Artsey 1000+Artsey 1000+Artsey 1000+Artsey 1000+
Default

Just want to thank the people who suggested software. I have checked some out. Using dupe detector reduces about 10% of the task. They all still have a quality control problem in not being able to zoom in to compare the quality of two similar images. While they do help in the detection and locating many dupes I still have to use the method described earlier to directly compare the images for quality. I can't just assume a small image size is no good. A larger version may be just a resampled re-jpeg enlargement of the smaller image. I've found that many times. Likewise a larger file size is not assumed to be better. I've found large file size images that were merely due to re-jpeg of a previous jpeg or the result of heavy flaws like moire patterns and excessive sharpening. I've even foung images that were poor quality but the same image dimensions, name and file size as another better quality that I assume to be an original. As if someone posted degraded images intentionaly trying to mislead a sort leaving one with a poor quality image. Not images I got here but elsewhere.

If anyone knows of a comparison software that allows for zoom in that would help a lot.
Artsey is offline   Reply With Quote
The Following User Says Thank You to Artsey For This Useful Post:
Old 09-15-2014, 02:58 PM   #16
electile disfunction
Vintage Member
 
Join Date: Oct 2008
Location: Somewhere flat, that's either hot, cold, or windy ... Canada?
Posts: 1,758
Thanks: 39,573
Thanked 15,874 Times in 1,731 Posts
electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+
Default

Quote:
Originally Posted by Artsey View Post
Just want to thank the people who suggested software. I have checked some out. Using dupe detector reduces about 10% of the task. They all still have a quality control problem in not being able to zoom in to compare the quality of two similar images. While they do help in the detection and locating many dupes I still have to use the method described earlier to directly compare the images for quality. I can't just assume a small image size is no good. A larger version may be just a resampled re-jpeg enlargement of the smaller image. I've found that many times. Likewise a larger file size is not assumed to be better. I've found large file size images that were merely due to re-jpeg of a previous jpeg or the result of heavy flaws like moire patterns and excessive sharpening. I've even foung images that were poor quality but the same image dimensions, name and file size as another better quality that I assume to be an original. As if someone posted degraded images intentionaly trying to mislead a sort leaving one with a poor quality image. Not images I got here but elsewhere.

If anyone knows of a comparison software that allows for zoom in that would help a lot.
Have you tried two (or more) dedicated, small, but excellent pieces of software used together, or do you want only a single program with compromises in seach and graphic quality for some reason?

Anti-Dupl is a stand-alone program but relies on whatever default viewing software you use for closer inspection. Example: When needed, I just double-click on the preview in AntiDupl and the full pic with all graphic data instantly opens in Irfanview (which you can close by pressing the [Esc] key just once!). This combination of A-D & Irfanview is extremely fast and powerful for finding the quality problems you outlined above. (99%+ of all the image files I download are poor, degraded, misleading images and I need to delete about 200 to 4000 of them every day!)

The programmer of A-D states outright that A-D is only for comparing thousands of images (even across many directories and and drives) and then listing possible duplicates for you to consider. Adding a redundant viewer software would take his time and effort away from creating good and fast comparison software.

Do you need something more?

e.d.

P.S.: I regularly search and compare graphic images individually as large as 140 MB, and neither A-D nor Irfanview has ever had problems with them.
electile disfunction is offline   Reply With Quote
Old 09-15-2014, 10:50 PM   #17
electile disfunction
Vintage Member
 
Join Date: Oct 2008
Location: Somewhere flat, that's either hot, cold, or windy ... Canada?
Posts: 1,758
Thanks: 39,573
Thanked 15,874 Times in 1,731 Posts
electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+electile disfunction 50000+
Lightbulb Alternate process

I have been considering your entire issue about finding and deleting duplicates. Mrs e.d. & I are in a very similar situation to you as we have hundreds of thousands of picture files, spread over many broad categories, collected over years.

I'll give you (and everyone reading) a brief outline of how we sort our files, and how we look for duplicate & similar pictures. This might give you a few ideas to get a good start at what you need and what will work for you!

In our collection we have: known duplicates which we want to keep for various reasons; many similar pictures--some to keep some to delete; pictures that are radically different but computers see as similar; and additions & deletions almost constantly. Files range in size from the smallest computer icons of 10s of bytes to deep space astronomy photographic tifs larger than 150 MB each. Picture subject and sort categories range from a small nephew making rude noises at his sister, to works of incredibly complex but sublime digital art--there is no "easy" way to sort our collection ... trust me, I've tried.

Tools:
You need some good but basic software to keep your sanity. All of it you can get and use for free, at some time you may wish to purchase some (and/or make donations to the creators).

a) Font--if you have any respect for your sanity, choose a well designed, sans serif, professional font that has all the letters and punctuation you use and (most importantly!!!) different glyphs for each of these characters:
1 i I L O 0 o F f $ S s T t W V U w v u
You need this to see thousands of file names easily and constantly. Use this font either universally on your computer or just set it up in your favourite file sorting program.
Every operating system comes with at least one (this is how programmers can see what they are typing): try Segoe UI, Trebuchet, or Tahoma and use them compare to others if you wish. (Note: Trebuchet is used by many hard-of-seeing people for all their computer and print needs.)

b) Graphic file viewer--it needs to load quickly and display accurately: jpg, gif, png, tif files and maybe others. It is best if you can have many instances of it running simultaneously (for visually comparing similar pictures). Irfanview is awesome, ACDsee used to do this (I haven't looked at it in a decade). The programs that come with Windows are mostly useless for this as they are slow, resource hogging, and try so hard to be "helpful" you don't know if what you are seeing is real or not most of the time.
You will likely want a program that allows you to quickly find the metadata and exif info of your picture files.

c1) A program to find duplicate computer files, regardless of file name; and
c2) A program to find similar picture files, that creates & reuses it's own databases.
There are hundreds of programs to find duplicate computer files spread of many drives. Find one you like; if it works with--or is part of--your favourite computer file sort & searcher (like Windows Explorer, etc.) so much the better. Before any visual-type search of new files, use this to find the obvious duplicates as programs are almost always infinitely faster that any graphic-search software.
Programs to find similar picture files are much more difficult to find--VEF member have been looking for and testing them for years with widely varying results. Many have been suggested for you in this thread already. I have found Anti-Dupl.net is very good and fast, it's also free of charge and reliable, and it works on our (3) computers without fussing with the OS, but YMMV.

d) (optional) Computer file renamer: you may not wish to fully rename any files but adding a couple words at the beginning of file names can infinitely cut down on your work. Examples: almost all of our visual artists have created self portraits--I use a renamer to append "Autoportrait_" to the beginning of those files' names ... Therefore it is easy to remove, the original file names remain intact, the files are easy to seach for and separate from the other works. Groups of pictures (from many different scanners or sources) will always be together, and any additional or new files can easily join the group with "copy and paste" commands.

e) (optional) Software to check csv files for known collections (if you need this you already know what it is). VEF posting Help Guides probably have suggestions for these, and they get reviewed/recommend regularly in other VEF threads.

Process:
1) Sort your collection of pics any way you wish. Our hierarchy usually includes these levels of folders (or more depending on many variables):
i. general subject (examples: our photos, animals, art, astronomy, hair styles, info.); then
ii. creator/artist/souce (examples: vacations, mammalia; Leonardo Da Vinci, NASA, Helmut Newton (photographer), wife's hair); then
iii. subsection(s) (examples: camping 2011, aquatic, architecture, solar system, photshoot name, long styles).
Also at thesubsection level we usually start including appropriate "miscellaneous" directories--examples: in nudes we sort whatever pictures remain by the models' surnames, but in astronomy we sort leftovers by wavelength and/or date.

2) Find any duplicate files. Keep or destroy then as necessary.
NEVER tell your programs to delete any file automatically unless YOU know exactly what you are doing!

3) Tell your "similar picture finder" software where it should look for and compare picture files. Sometimes I seach individual directories or drives, but usually (for nudes especially) I'll ask the computer to compare all new files to all known files on a regular basis--it will ignore mistakes it made in the past and add any new images to its databases as necessary.

4) You should be able to sort the "similar picture finder" results alphabetically and/or by directory and/or by % similarity and/or by type of difference so that your 'known & sorted' file names occur together and any new resultant names are easy-to-see beside them. Open any pictures you need to see for comparisons, tell your softwares of the results (Anti-Dupl continually deletes files, and updates filenames & directories if you change them, for example), and go on to the next results.

5) Once your "similar picture finder" has its initial database(s) and file of erroneous matches to ignore, all subsequent duplicate searches wil be much faster. (Right now, we can search 100,000 files over three hard drives in 3 or 4 minutes, where the initial database creation and search may have taken 20 minutes.)

6) Your intial review of the results found by your programs will take longer than any subsequent search, too but you will remove a lot of junk that first time. And you'll learn a lot, too!

That's the overview, I hope it gives you some ideas for what you want and need for your files. I will add corrections and other ideas as they occur to me.

Please feel free to send me a Private Message if you think I can help with something.

e.d.
electile disfunction is offline   Reply With Quote
The Following 3 Users Say Thank You to electile disfunction For This Useful Post:
Old 10-07-2015, 03:21 PM   #18
Artsey
Member
 
Join Date: Aug 2011
Posts: 65
Thanks: 5,978
Thanked 369 Times in 64 Posts
Artsey 1000+Artsey 1000+Artsey 1000+Artsey 1000+Artsey 1000+Artsey 1000+Artsey 1000+Artsey 1000+Artsey 1000+Artsey 1000+Artsey 1000+
Default

Want to thank everybody who contributed sorting ideas. That has increased my computer assisted dupe detection to about 20 to 30% of the dupes. Still, major differences in dupes prevents computer assistance from detecting them. Different color, cropping, scan pattern effects etc. causes it.

Working my files I have developed my criteria sort strategy to further speed up and improve the reliability of the dupe detection. My confidence is high that I get 100% of the dupes and with my newest strategies get the sort task to speeds that rival computer assisted detction. I have to time it sometime.

Anyway the additional strategies ar that after the innitial criteria sort that may reduce the size of a criteria, bikini/water/standing for example, to 150 to 200 images can be broken down into images with arms up leg up hat or head wear. Easy to see in thumbnail and geting a criteria down to 30 to 40 images to sort speeds it up hugely. It's a mathematical factoral thing. To explain; when looking at a folder with 200 bikini/water/standing images I look at the first image and scan the folder looking at 199 other images. Then the second image I have to scan 198 of them again. the third 197 etc. But if I get a clearly defined criteria, like bikini/water/standing/arms up, with 25 images I have reduced the scan time to an eighth of the time. Plus I easiliy recognize many images that don't have dupes so don't even have to scan the whole folder for many.

An example of the time demanded of the new strategy. I just sorted my miscilaneous movie star folder. It had 20,000 images. Deleted 15,000, spent about an hour a day for maybe a week and a half, and have huge confidence I got 100% of the unwanted dupes. In addition to the reduction of time demand it has reduced the fatigue and mental concentration demanded. It actually made it a moderately enjoyable task.

Last edited by Artsey; 10-07-2015 at 03:29 PM..
Artsey is offline   Reply With Quote
The Following 3 Users Say Thank You to Artsey For This Useful Post:
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump




All times are GMT. The time now is 12:19 AM.






vBulletin Optimisation provided by vB Optimise v2.6.1 (Pro) - vBulletin Mods & Addons Copyright © 2017 DragonByte Technologies Ltd.