Skip to content

Image deduplication tool, optimized for large amount of pictures and CG libraries.

License

Notifications You must be signed in to change notification settings

SecretNest/ImageStore

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ImageStore

Image deduplication tool, optimized for large amount of pictures and CG libraries.

This tool is built as PowerShell cmdlets, providing user a flexible way to deal with large amount of image files (~1 million). Specially, it is optimized for CG libraries storing by allowing user to suppress the comparing among files in the same folder.

Image Hashing Arithmetic

This tool use a forked version of priHash, which added methods optimized for ImageStore calling. priHash is a C# Implementation of pHash (http://phash.org). Based on phash-0.9.4 for Windows. In this tool, difference degree is based on the calculation of priHash.

Use Module in PowerShell

To use any module from dll in PowerShell, you just need 3 steps:

  1. Start PowerShell. Usually it is placed as C:\windows\System32\windowspowershell\v1.0\powershell.exe in Windows.
  2. Use command Import-Module to load module from dll file. The parameter of this command is the path of the dll file.
    Import-Module C:\ImageStore.dll will load the module file named as ImageStore.dll and placed in the root folder of drive C.
    Import-Module .\ImageStore.dll will load the module file named as ImageStore.dll from the current directory.
  3. Use cmdlets.

Also, you can combine the step 1 and 2 as one, by passing the command as a parameter while starting PowerShell.
C:\windows\System32\windowspowershell\v1.0\powershell.exe -noexit -command "Import-Module .\ImageStore.dll"

Enable Information and Verbose Output

By default, the information and verbose level output will be silenced. ImageStore will report key information as informational output and progress updates as verbose one. Thus, enabling information output is highly recommended. If ImageStore is dealing with large amount of files, turning on verbose output is advised.

  • Turn on information output: $InformationPreference="Continue"
  • Turn on verbose output: $VerbosePreference="Continue"
  • Turn off information output: $InformationPreference="SilentlyContinue", or simply restart PowerShell.
  • Turn off verbose output: $VerbosePreference="SilentlyContinue", or simply restart PowerShell.

These setting will not be preserved among instances of PowerShell. Every time the PowerShell started, there are set as "SilentlyContinue".

Database

ImageStore need Sql Server 2017 for hosting database. Each project should have a dedicated database.

All editions of Sql Server 2017 on Windows are supported, including LocalDb, Express, Standard, Enterprise and Developer. Linux versions are not tested.

Attached database file mode is supported and recommended.

To install Sql Server 2017, access Sql Server 2017 Homepage and download the edition you desired. LocalDb or Express edition will be a good choice IMHO.

You could download the empty database file DataStore.mdf and DataStore_log.mdf from Database folder, or create an empty database by script provided as CreateDatabase.txt in the samne folder.

Concepts

There are several concepts defined in ImageStore. Reading these docs will help you to understand the system.

Concept Description
Database A Sql Server 2017 database to save all records and settings.
Folder The root directory of your image library.
Extension Extension record for each kind of files.
File File record for each file.
Same File Exactly same files detected by Sha1 hashing.
Similar File Similar images detected by pHash algorithm.
Thumbprint Cache Cache for image thumbprints used in Similar File UI.
Difference Degree The difference between two image files.

Cmdlets

See Cmdlets.

Entity Types

These types of entities will be used while operating with cmdlets of ImageStore.

Type Description
ImageStoreFolder Represents a folder for storing image files.
ImageStoreIgnoredDirectory Represents an exclusion a directory from a folder.
ImageStoreExtension Represents an extension, a kind of file.
ImageStoreFile Represents a file stored in a folder.
ImageStoreSameFile Represents a record that a file detected to be the same as at least one other file.
ImageStoreSimilarFile Represents a similar relationship between two image files.

Other Types

These types will be used while operating with cmdlets of ImageStore.

Type Class Description
StringPropertyComparingModes enum The way to use the string provided as search condition.

Walkthrough

A walkthrough of use this product.

About

Image deduplication tool, optimized for large amount of pictures and CG libraries.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages