Skip to content

Commit

Permalink
Merge pull request #471 from pzaino/develop
Browse files Browse the repository at this point in the history
Bug fixing after long testing of new features
  • Loading branch information
pzaino authored Oct 11, 2024
2 parents 28ddc73 + fd34bf3 commit 05e3c76
Show file tree
Hide file tree
Showing 14 changed files with 729 additions and 370 deletions.
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,10 +103,12 @@ For more information on the features, see the [features](doc/features.md) page.
### What problem does it solves?

The CROWler is designed to solve a set of problems about web crawling, content
discovery, and data extraction. It's designed to be able to crawl websites in a
respectful and efficient way. It's also designed to be able to crawl private
networks and intranets, so you can use it to create your own or company search
engine.
discovery, technology detection and data extraction.

While it’s main goal is to enable private, professional and enterprise users to
quickly develop their content discovery solutions, It’s also designed to be
able to crawl private networks and intranets, so you can use it to create your
own or your company search engine.

On top of that it can also be used as the "base" for a more complex cyber security
tool, as it can be used to gather information about a website, its network, its
Expand Down
Loading

0 comments on commit 05e3c76

Please sign in to comment.