It's for searching the internet

If you don't know where you're going, this is a great place to start.

Useful concepts to understand it:

1. How do you carve an elephant out of marble? Start with a block of marble and cut away everything that doesn't look like an elephant.

Theres three types of filters - boolean, tagable and functional.

Boolean filters are usually related to page properties. For example 'has' is a boolean filter as it tests if a page has as a certain property like title (eg. has:title). Tagable filters are more complex and allow both filtering and tagging. For example lang:en will filter by english pages. If you just want to tag pages with their language use lang:*:* or lang::* Functional filters are expensive to run but the most powerful and include running machine learning models on the pages or running entire other queries on the pages. Usually to run these filters you need to know some information to pass into the filter. For example: ml-model(science, 0.99)::* will tag each page with the category predicted by the ML model science where that prediction is > 99% confidence threshold.

To filter run: ml-model(science,0.99):* To Tag include: ml-model(science,0.99)::*

Categories available:

• Health
• Environment
• Psychology
• Engineering
• Computer_Science
• Astronomy
• Physics
• Neuroscience
• Medicine
• Biology
• Animal_Science
• Anthropology

Traditional word search

burger

Include a random page for every 10000 looked at

random:0.0001 

Search for burgers and filter by city

burger geo-ip-city:Austin

Filter by language

lang:en

Tag the results by city where that city is in the United_States

geo-ip-city::* geo-ip-country:United_States

Exclude all english language pages

-lang:en

Remove 1 in 100000 random pages from the search even if all other pages match

burger -random:0.00001

Get English language burger articles from the science model with a 99% confidence threshold. Tag them with the og:type meta tag. Only include results with titles and descriptions. Apply a distinct window filter to the results so we only get one from the same domain every 1000 results.

burger ml-model(science,0.99):* lang:en property(og_type):*:* has:title has:description distinct-window(domain):1000

Use accesskeys if you're building a user facing tool
Use workspaces if you're building a server facing tool

• /api/v1/accounts - access all the data for an Account
• /api/v1/accesskeys - access all the data by an AccessKey
• /api/v1/workspaces - access all the data for a Workspace
• /api/v1/parser - access parsers for checking query text

Use this for user facing tools. You make up your own access keys minimum of 12 chars.

• GET /wellknown - get all well known WorkspaceAccessKeys

• GET /{accesskey} - list all the WorkspaceAccessKeys for an AccessKey

• GET /{accesskey}/workspace/{Id} - get a specific Workspace by its WorkspaceAccessKey id

• GET /{accesskey}/workspace/{workspaceAccessKeyId}/results - get the Workspace WorkspaceResults. Optional ?fromSequence=<number greater than 0>&count=<number:0-50>

• GET /{accesskey}/workspace/{workspaceAccessKeyId}/results/random - Get a random result from the Workspace;

• GET /{accesskey}/workspace/{workspaceAccessKeyId}/supportedFilters - Get the search and tagging filters supported by this Workspace;

• POST / - create an AccessKey

• POST /{accesskey}/workspace - create a Workspace and owner WorkspaceAccessKey

• PUT /{accesskey}/workspace/{workspaceAccessKeyId} - update a Workspace

• DELETE /accesskeys/{accesskey}/workspace/{workspaceAccessKeyId} - delete a WorkspaceAccessKey. Note the Workspace wont delete until all WorkspaceAccessKeys that reference it are deleted

• PUT,POST /{accesskey}/workspace/{workspaceAccessKeyId}/results - add or update WorkspaceResults for a Workspace

• DELETE /{accesskey} - delete an AccessKey

Use this for server facing tools

• GET /wellknown - get all well known Workspaces. Useful for publicly accessible workspaces and community workspaces
• GET /{workspaceId}/results - get WorkspaceResult records. Optional ?fromSequence=<number greater than 0>&count=<number:0-50>
• PUT,POST /{workspaceId}/results - add or update WorkspaceResult records
• DELETE /{workspaceId}/results - delete WorkspaceResult records

A personal group of access keys

{
	"Name": "PurpleMonkeyDishwasher",
	"Created": "2019-02-15T14:25:06.639414+08:00",  
	"IsActive": true
}

An access key to a Workspace Workspaces

{  
      "Id":"8cd40e60-5749-480a-a0e3-77d66f3bb5d6",  
      "AccountName":"PurpleMonkeyDishwasher",  
      "Expiry":"9999-12-31T00:00:00",  
      "IsActive":true,  
      "Permissions":1, // flags 0 = none, 1 = owner, 2 = Manager, 4 = Read 
	  "IsWellKnown":true,
      "Created":"2019-02-15T14:25:06.639414+08:00",
	  "Workspace": {...},
	  "Revision": 1
}  

Details of a running search query and it's results and other configuration for a persistent search

{  
   "WorkspaceId":"f5201ff7-ea59-4e00-87b9-af4a0a9c8e2e",  
   "Name":"random stuff",  
   "Description":"",  
   "QueryText":"-random",  
   "Created":"2019-02-15T06:25:06.639414+08:00",  
   "IsActive":true,  
   "IsWellKnown":false,
   "ResultCount":206,  
   "AccessKeys":[    
      {  
         "Id":"8cd40e60-5749-480a-a0e3-77d66f3bb5d6",  
         "AccessKey":"PurpleMonkeyDishwasher",  
         "Expiry":"9999-12-31T00:00:00",  
         "IsActive":true,  
         "Permissions":1,
		 "Created":"2019-02-15T14:25:06.639414+08:00"  
      },  
      {    
         "Id":"ab57d44d-2e6b-4404-adfa-0a8e393ffd45",  
         "AccessKey":"FaxesFavouriteKey",  
         "Expiry":"9999-12-31T00:00:00",  
         "IsActive":true,  
         "Permissions":1,
         "Created":"2019-03-08T17:39:50.25246+08:00"  
      }  
   ],
   "Revision": 1
}  

Individual search results

{   
    "Hash":"DDD2291FFF76A7209D5F8BF2FD5EFAA6",  
    "RefererUri":"",  
    "Uri":"https://www.alibris.com/",  
    "Title":"Alibris - Buy new and used books, textbooks, music and movies",  
    "Description":"",  
    "Tags":[ 
		"en",
		"Austin",
		"United_States"
    ],  
    "Created":"2019-03-07T20:17:38.838743+08:00",  
    "PageSize":62985,  
    "GlobalSequence":9241  
}