Torrlient NoXs is an Extremely Powerful Distributed Crawler.
NoXs URL Crawler
Searches the World Wide Web for URLs and stores them in a database. This crawler does not read robots.txt directive, but also does not store anything else other than URLs. While we do not store information from websites, these URLs are used for serval purposes (see future down).
| Crawler Mode | User-agent |
| Torrlient NoXsURL-Crawler | Torrlient NoXs; https://torrlient.no/NoXs/ |
NoXs Console
The NoXs Console works similar to "Google Search Console", but controls the NoXs Crawler on a per domain basis. A client Account or registration is not required to utilize the NoXs Console, but the domain must be validated.
The NoXs Console allows website owners to control the NoXs Crawler, and how it will interact with their site. It's also possible to submit data to the crawler directly, see if the crawler is currently crawling the specific domain. And many other features.
NoXsDB
The NoXsDB is the largest URL Database in the world, where every URL only exist once. NoXsDB is a custom database system, designed and developed for high performance crawling. Access to NoXsDB is available through the NoXs Console for registered clients.
NoXsDB API
The NoXsDB API is available for any registered user with valid credit balance. And gives simple hasslefree access to queries and NoXsDB exports. NoXsDB API supports both JSON and XML results.
| Results | Price/Credit |
| 0 - 499 | 1 |
| < 1k | 2 Every 1k Results upto 9999 |
| < 10k | 4 Every 10k Results upto 99999 |
| < 100k | 16 |
Purposes
The NoXs Crawler allows us to find relatively specific data in massive quantities very fast. This is very useful for machine learning and of course scientific purposes.
Internet Science
Internet Sciences exists in serval sub fields and empowers curiosity. We can learn a lot about technology and even people by studying the internet.
Internet Security
Many websites contain security issues. Often only found by malicious parties and exploited, on the cost of operator's expense and credability. As technology evolve more and more crimal activity take place online, hidden from naive eyes.
NoXs finds everything: Malicious websites. Leaked image/video and personal information. We collect the required information for victims to take action against the responsible parties.
Machine Learning
Machine Learning require large datasets to learn and opperate. While no datasets are perfect, we utilize a pre-processor while fetching, to determinate a "type" of data that we desire.
Search and Content
Crawlers alike NoXs serves the largest Search Engines in the world.