Welcome here,Iveely search engine is a pure java realize search engine, try to directly hit the user's search for answers.
Iveely is an abbreviation from 'I void everything,except love you'.
Contact me if you have any questions, [email protected]. Auto build here.
Download iveely.crawler & iveely.search.api,and build to executable jar,you can use maven build them quickly.
Copy "example conf/for crawler/conf" to the same parent directory with 'iveely.crawler.jar'. Use 'java -jar iveely.search.api.jar' & 'java -jar iveely.crawler.jar' to execute them,please run iveely.search.api.jar first.
1. First run: java -jar iveely.search.api.jar
2. Next run: java -jar iveely.crawler.jar
Query with keyword 'java', you can open browser with http://127.0.0.1:8080/api/0.9.1/query?queryField=title&respFields=title%2Ccontent%2Curl&keywords=java&totalHits=10 to get the result.
If you get the response json,congratulations, you've successfully run.
Moreover API information were described using Swagger-UI. So you can visit http://127.0.0.1:8080/swagger-ui.html to get more api.
Iveely also was submitted to the maven central repositry. visit iveely@maven to get more.
With only document search is not the goal,to build more intelligent search engine is very important,so we have added a project named 'iveely.brain'.
iveely.brain has two mode, local debug & Remote network calls.
To run iveely.brain,you should do as follow:
Download iveely.brain and use maven to build,you also can run code by main class Progam.java.
Local operation does not require any arguments,but you need copy folder 'example conf/for brain/ai' to the same parent directory with 'iveely.brain.jar'.
When run local mode,you can enter a question on console like 'Which city is the capital of the United States?'
If console write 'Washington.',congratulations, you've successfully run.
Q:Which city is the capital of the United States?
A:Washington.
For more information see Distributed Artificial Intelligence Markup Language.
Modify file 'ai/property/branches.xml', configure the port number and offer to provide network services, so that the external system can access the service, which is important for distributed search engine.
In 2015, my friend and I began to research in the field of artificial intelligence, we need a lightweight computing framework to help us build data model in fast. Rapid deployment, rapid results, simple to fit is our original intention at that time. Even hope that any program can be distributed, such as crawler program of search engine. In the process of last year, Iveely Computing has given us a lot of help, so we decided to open source to more developers.
It is a very lightweight distributed real-time computing framework like storm,it has four very important components:
It is the input source data, can be gained by reading the file system data source, also can be achieved by other ways. It is also the place where the whole cluster program execution started
IOutput data sources can be from IOutput and IInput, but the output must IOutput or no output, cannot be directly output to a file.It is the middle of the data processing unit.
IInputReader is an input with the function of file read, any IReader implementation can be used in IInputReader, including Windows file system,Unix file system, Hadoop file system, etc.
It is a subclass of IOuput, used for writing data to a local file system or other file system.
Example can be found here.
iveely database is a mini data storage system,it's has two mode, as follow:
Local mode is easy to use, example code =>
final String houseName = "example_house";
final String tableName = "example_table";
Warehouse warehouse = LocalStore.getWarehouse(houseName);
warehouse.createTable(tableName);
int id = warehouse.insert(tableName,new Object[]{"1", "this is example"});
Object[] data = warehouse.selectById(tableName,id);
System.out.print(data);
warehouse.dropTable(tableName);
Use remote mode you can build a database server,and every application can access the database. example code =>
final String houseName = "example_house";
final String tableName = "example_table";
DbConnector connector = new DbConnector(houseName, "localhost", 4321);
final int id = connector.insert(tableName, new Object[]{"1", "this is example"});
Object[] data = connector.selectOne(tableName, id);
System.out.print(data);
connector.dropTable(tableName);
For a better understanding of the next generation of modern search engines, I wrote a book named "Big Data Search Engine Principle Analysis and Implementation", you can get this book on amazon.com or jd.com.
From 2009, I began to think search engine not just a simple search tool. I offer keywords to search engine, the search engine returns some documents, I think this is not smart enough.
I expect I'll give the search engines a question, it gives me an answer,this is my original goal.