Skip to content

justinrobinson/totalrecall

 
 

Repository files navigation

Total Recall

Crawl and index your (static) asp.net website using an MSBuild target. Simple querying. Uses Lucene.Net and NCrawler.

totalrecall is a kind of wrapper for Lucene, but with simplified indexing that takes place during the build process. You give the build target the address of your integration or development server, and documents are crawled and indexed using HTTP. The resulting Lucene index is stored in your site (on the file system) — a simple query wrapper knows where to look for it.

Nuget Install

Step 1: Install package

nuget> install-package totalrecall

Step 2: Edit packages\totalrecall\tools\totalrecall.targets file:

<TotalRecall.MSBuild.IndexSiteTask PublishedWebsiteUrl="[YOUR NEWLY BUILT WEBSITE URL]" Optimize="True" IndexFolder="$(WebProjectOutputDir)\.totalrecall"/>

Step 3: Index your site:

MSBuild.exe /target:CrawlIndexSite mywebsiteproj.csproj

Step 4: Navigate to /search/query=something%20interesting

Building from source

Step 1: Run build.bat and reference binaries\merged\TotalRecall.dll. Copy the binaries\merged\totalrecall.targets file to your project.

Step 2: Import totalrecall targets somewhere into your project file:

<Import Project="..\your\path\to\totalrecall.targets" />

Step 3: Edit totalrecall.targets file:

<TotalRecall.MSBuild.IndexSiteTask PublishedWebsiteUrl="[YOUR NEWLY BUILT WEBSITE URL]" Optimize="True" IndexFolder="$(WebProjectOutputDir)\.totalrecall"/>

Step 4: Add config section to your web.config file:

<configSections>
   <section name="totalrecall" type="TotalRecall.Configuration.TotalRecallConfigurationSection,TotalRecall" />
</configSections>

<totalrecall indexfolder="~/.totalrecall" />

Step 5: Index your site:

MSBuild.exe /target:CrawlIndexSite yourwebsiteproj.csproj

Step 6: Query on your search page:

var engine = new TotalRecall.SearchEngine();
var hits = engine.Search("something interesting", 10);

web.config section documentation

All values are optional

<totalrecall
  indexfolder="~/.totalrecall"  <!-- The path where the Lucene index should be stored. Default: "~/.totalrecall" -->
  siterootdirectory="store"     <!-- The virtual directory of your development site if there is one. Default: Empty -->
  optimize="true"               <!-- Whether or not to optimize and compact the index after crawling: Default: true -->
/>

About

Crawl and index your (static) asp.net website for searching using an MSBuild target. Simple search query interface. Uses Lucene.Net and NCrawler.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C# 97.5%
  • PowerShell 1.5%
  • Other 1.0%