Skip to content

OffensiveK/ScrapeQL

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

18 Commits
 
 
 
 
 
 

Repository files navigation

ScrapeQL

Query language for webscraping

Dependencies

We are using the packages "HTMLAgilityPack" and "Csharpmonad" to realize this project.

Syntax

Regarding the syntax, ScrapeQL is much alike its famous paragon, SQL.

There are several queries.

  1. Load-Query: Loads a website or HTML File for further selection into a virtual workspace, later accessible by identifier.
LOAD "filename.fileExtension/http://websiteName.domain" AS Identifier
  1. Write-Query: Writes the finished selection into filename.filextension.
WRITE identifier TO "filename.fileextension"
  1. Select-Query: Selfexplanatory... Selects from identifier using given selector.
SELECT "selector" FROM identifier <WHERE attribute=value|identifier CONTAINS attribute>

Console Commands

Generally, console commands begin with a :.

  1. Load File Command: Loads a file of filetype ".scrapeql" and executes it. The file can contain both console commands and queries. (Not to be confused with Load-Query)
:load file.scrapeql
  1. Print Variable Command: Prints designated variable.
:printvar identifier
  1. Print Scope Command: Prints names of all loaded objects
:printscope
  1. Setprompt Command: Sets the prompt. Default is "ScrapeQL>".
:setprompt string
  1. Clear command: Clears the commandline.
:clear

(Note: Commands surrounded by <> are optional. Choose between Codeblocks, when they're separated by |. The language is case sensitive.)

About

Query language for webscraping

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages