http://regain.murfman.de/wiki/ 2012-01-31T10:21:46+01:00 http://regain.murfman.de/wiki/ http://regain.murfman.de/wiki/lib/images/favicon.ico text/html 2012-01-30T09:43:52+01:00 Benjamin http://regain.murfman.de/wiki/doku.php?id=components:crawler_plugins&rev=1327913032&do=diff Crawler Plugins hook into the crawling process in order to add advanced functionality. What can crawler plugins do? Some examples: * Modify the result of preparators * by specifying default-values if the chosen preparator does not fill in a certain field (onBeforePrepare) * by overriding or modyfing the results of whatever preparator was chosen (onAfterPrepare) text/html 2012-01-30T09:43:08+01:00 Benjamin http://regain.murfman.de/wiki/doku.php?id=project_info:crawling_process&rev=1327912988&do=diff How does the crawling process work? Where do the Crawler Plugins interact? At the beginning, onStartCrawling(Crawler) is called for all plugins. 1. Creating the job queue * According to the Crawling Whitelist and Blacklist, the start URLs are added to the job queue (Crawler::addJob) (onAcceptURL(String url, CrawlerJob job) or onDeclineURL(String url) is called to inform the plugin) * If a job would be accepted, the crawler plugins are asked if they want to blacklist it anyway (boolean c… text/html 2012-01-30T09:36:47+01:00 Benjamin http://regain.murfman.de/wiki/doku.php?id=features:white_and_black_list&rev=1327912607&do=diff Using the white and black list you can specify very precisely, what will get in the index and what not. The base rule is always: A document gets in the index, if its URL comes up to at least one entry from the white list, but no entry from the black list. text/html 2011-12-03T09:53:23+01:00 Benjamin http://regain.murfman.de/wiki/doku.php?id=project_info:sourcecode&rev=1322902403&do=diff If you are interested in details of regain or if you want to fix a bug, you'll need the sourcecode. There are several possibilites to get the sourcecode. You can either download the zipped source of a particular version or you can get the newest developer version directly from the SVN repository. text/html 2011-10-31T09:13:31+01:00 Benjamin http://regain.murfman.de/wiki/doku.php?id=project_info:howto_regain_in_an_ide&rev=1320048811&do=diff In any case, you need to install the JDK first: Download. You also should set the environment variable JAVA_HOME to the directory you installed the JDK into, e.g. C:/Programme/Java/jdk1.6.0_12. The variable PATH should contain the bin-Folder, e.g. %PATH%;%JAVA_HOME%\bin. text/html 2011-10-31T09:09:49+01:00 Benjamin http://regain.murfman.de/wiki/doku.php?id=de:project_info:building_regain&rev=1320048589&do=diff Diese Seite zeigt, wie Sie regain selbst aus dem Quellcode bauen können auf der Kommandozeile. (Sie können aber auch einen Code-Editor installieren und konfigurieren.) Entwicklungsumgebung einrichten Um regain bauen zu können, müssen Sie zuerst folgendes installieren: