|
-
-
- ', $title, ''; -} - -function template_sphinx_config_below() -{ - - echo ' - |
-
- 
-
-
- SMF Sphinx Configuration Utility
-
-
-
-
-
-';
-}
-
+
+ This configuration tool is designed to guide you through the installation of the Sphinx full-text search engine, specifically for Simple Machines Forum. Following the steps in this tool will tell how to install Sphinx, will configure SMF for using Sphinx, and will create a configuration file that will be needed for Sphinx based on SMF\'s settings. Make sure you have the latest version of this tool, so that the latest improvements have been implemented.
+
+ What is Sphinx?
++ Sphinx is an Open Source full-text search engine. It can index texts and find documents within fractions of seconds, a lot faster than MySQL. Sphinx consists of a few components: +
+ There\'s the indexer that creates the full-text index from the existing tables in MySQL. The indexer is run as a cron job each time, allowing it to update the index once in a while. Based on the configuration file, the indexer knows how to connect to MySQL and which tables it needs to query. +
+ Another important component is the search deamon (called searchd). This deamon runs as a process and awaits requests for information from the fulltext indexes. External processes, like the webserver, can send a query to it. The search deamon will then consult the index and return the result to the external process. +
+ +When should Sphinx be used for Simple Machines Forum?
++ Basically Sphinx starts to get interesting when MySQL is unable to do the job of indexing the messages properly. In most cases, a board needs to have at least 300,000 messages before that point has been reached. Also if you want to make sure the search queries don\'t affect the database performance, you can choose to put Sphinx on a different server than the database server. +
+ +Requirements for Sphinx
+-
+
- Root access to the server you\'re installing Sphinx +
- Linux 2.4.x+ / Windows 2000/XP / FreeBSD 4.x+ / NetBSD 1.6 (this tool will assume Linux as operating system) +
- A working C++ compiler +
- A good make program +
+ This tool will assume you will be installing Sphinx version ', $sphinx_ver, '. A newer version might be available and, if so, would probably be better. Just understand that the steps below and the working of the search engine might be different in future versions of Sphinx. Please note that Sphinx versions prior to 0.9.9 will not work properly. +
+Retrieving and unpacking the package
+ + Grab the file from the Sphinx website:+ [~]# wget http://www.sphinxsearch.com/downloads/sphinx-', $sphinx_ver, '.tar.gz
+
+ Untar the package:
+ [~]# tar -xzvf sphinx-', $sphinx_ver, '.tar.gz
+
+ Go to the Sphinx directory:
+ [~]# cd sphinx-', $sphinx_ver, ' + +
Compiling Sphinx
+ Configure Sphinx (generally no options are needed):+ [~]# ./configure
+
+ If everything went well, run the make tool:
+ [~]# make
+
+ If that went well too, make the install:
+ [~]# make install
+ + '; +} + + +function step_2() +{ + global $context, $modSettings, $txt; + + template_sphinx_config_above('Configure SMF for Sphinx'); + + echo ' + A few settings can be configured allowing to customize the search engine. Generally all options can be left untouched.
+
+ '; + + template_sphinx_config_below(); +} + +function step_3() +{ + global $context, $modSettings, $txt; + + checkSession(); + + updateSettings(array( + 'sphinx_data_path' => rtrim($_POST['sphinx_data_path'], '/'), + 'sphinx_log_path' => rtrim($_POST['sphinx_log_path'], '/'), + 'sphinx_stopword_path' => $_POST['sphinx_stopword_path'], + 'sphinx_indexer_mem' => (int) $_POST['sphinx_indexer_mem'], + 'sphinx_searchd_server' => $_POST['sphinx_searchd_server'], + 'sphinx_searchd_port' => (int) $_POST['sphinx_searchd_port'], + 'sphinxql_searchd_port' => (int) $_POST['sphinxql_searchd_port'], + 'sphinx_max_results' => (int) $_POST['sphinx_max_results'], + )); + + if (!isset($modSettings['sphinx_indexed_msg_until'])) + updateSettings(array( + 'sphinx_indexed_msg_until' => '1', + )); + + + template_sphinx_config_above('Configure SMF for Sphinx'); + echo ' + Your configuration has been saved successfully. The next time you run this tool, your configuration will automatically be loaded. +
Generating a configuration file
+ Based on the settings you submitted in the previous screen, this tool can generate a configuration file for you that will be used by Sphinx. Press the button below to generate the configuration file, and upload it to /usr/local/etc/sphinx.conf (default configuration).+
+
+ +
Some file actions
+ Create directories for storing the indexes:', strpos($modSettings['sphinx_data_path'], '/var/sphinx/') === false ? '' : ' + [~]# mkdir /var/sphinx
', ' + [~]# mkdir ' . $modSettings['sphinx_data_path'] . '
+ [~]# mkdir ' . $modSettings['sphinx_log_path'] . '
+
+ Make the data and log directories writable:
+ [~]# chmod 666 ' . $modSettings['sphinx_data_path'] . '
+ [~]# chmod 666 ' . $modSettings['sphinx_log_path'] . '
+ +
Indexing time!
+ It\'s time to create the full-text index:+ [~]# indexer --config /usr/local/etc/sphinx.conf --all
+
+ If that went successful, we can test run the search deamon. Start it by typing:
+ [~]# searchd --config /usr/local/etc/sphinx.conf
+
+ If everything worked so far, congratulations, Sphinx has been installed and works! Next step is modifying SMF\'s search to work with Sphinx. + +
Configuring SMF
+ Upload the SearchAPI-Sphinxql.php file to the \'Sources\' directory.+ Select \'Sphinx\' as database index below and press \'Change Search Index\'. Test your search function afterwards, it should work now!
+
+
+
+ +
Creating a cron job for the indexer
+ In order to keep the full-text index up to date, you need to add a cron job that will update the index from time to time. The configuration file defines two indexes: smf_delta_index, an index that only stores the recent changes and can be called frequently. smf_base_index, an index that stores the full database and should be called less frequently. + + Adding the following lines to /etc/crontab would let the index rebuild every day (at 3 am) and update the most recently changed messages each hour:+ # search indexer
+ 10 3 * * * /usr/local/bin/indexer --config /usr/local/etc/sphinx.conf --rotate smf_base_index
+ 0 * * * * /usr/local/bin/indexer --config /usr/local/etc/sphinx.conf --rotate smf_delta_index
+ + + '; + template_sphinx_config_below(); +} + +function step_888() +{ + global $modSettings; + + checkSession(); + + if (in_array($_REQUEST['search_index'], array('', 'fulltext', 'custom', 'sphinx', 'sphinxql'))) + updateSettings(array( + 'search_index' => $_REQUEST['search_index'], + )); + + echo 'Setting has been saved. This window can be closed.'; +} + + + +function step_999() +{ + global $context, $db_server, $db_name, $db_user, $db_passwd, $db_prefix; + global $db_character_set, $modSettings; + + $humungousTopicPosts = 200; + + ob_end_clean(); + header('Pragma: '); + if (!$context['browser']['is_gecko']) + header('Content-Transfer-Encoding: binary'); + header('Connection: close'); + header('Content-Disposition: attachment; filename="sphinx.conf"'); + header('Content-Type: application/octet-stream'); + + $weight_factors = array( + 'age', + 'length', + 'first_message', + 'sticky', + ); + $weight = array(); + $weight_total = 0; + foreach ($weight_factors as $weight_factor) + { + $weight[$weight_factor] = empty($modSettings['search_weight_' . $weight_factor]) ? 0 : (int) $modSettings['search_weight_' . $weight_factor]; + $weight_total += $weight[$weight_factor]; + } + + if ($weight_total === 0) + { + $weight = array( + 'age' => 25, + 'length' => 25, + 'first_message' => 25, + 'sticky' => 25, + ); + $weight_total = 100; + } + + + echo '# +# Sphinx configuration file (sphinx.conf), configured for SMF 1.1 +# +# By default the location of this file would probably be: +# /usr/local/etc/sphinx.conf + +source smf_source +{ + type = mysql + sql_host = ', $db_server, ' + sql_user = ', $db_user, ' + sql_pass = ', $db_passwd, ' + sql_db = ', $db_name, ' + sql_port = 3306', empty($db_character_set) ? '' : ' + sql_query_pre = SET NAMES ' . $db_character_set, ' + sql_query_pre = \ + REPLACE INTO ', $db_prefix, 'settings (variable, value) \ + SELECT \'sphinx_indexed_msg_until\', MAX(id_msg) \ + FROM ', $db_prefix, 'messages + sql_query_range = \ + SELECT 1, value \ + FROM ', $db_prefix, 'settings \ + WHERE variable = \'sphinx_indexed_msg_until\' + sql_range_step = 1000 + sql_query = \ + SELECT \ + m.id_msg, m.id_topic, m.id_board, IF(m.id_member = 0, 4294967295, m.id_member) AS id_member, m.poster_time, m.body, m.subject, \ + t.num_replies + 1 AS num_replies, CEILING(1000000 * ( \ + IF(m.id_msg < 0.7 * s.value, 0, (m.id_msg - 0.7 * s.value) / (0.3 * s.value)) * ' . $weight['age'] . ' + \ + IF(t.num_replies < 200, t.num_replies / 200, 1) * ' . $weight['length'] . ' + \ + IF(m.id_msg = t.id_first_msg, 1, 0) * ' . $weight['first_message'] . ' + \ + IF(t.is_sticky = 0, 0, 1) * ' . $weight['sticky'] . ' \ + ) / ' . $weight_total . ') AS relevance \ + FROM ', $db_prefix, 'messages AS m, ', $db_prefix, 'topics AS t, ', $db_prefix, 'settings AS s \ + WHERE t.id_topic = m.id_topic \ + AND s.variable = \'maxMsgID\' \ + AND m.id_msg BETWEEN $start AND $end + sql_attr_uint = id_topic + sql_attr_uint = id_board + sql_attr_uint = id_member + sql_attr_timestamp = poster_time + sql_attr_timestamp = relevance + sql_attr_timestamp = num_replies + sql_query_info = \ + SELECT * \ + FROM ', $db_prefix, 'messages \ + WHERE id_msg = $id +} + +source smf_delta_source : smf_source +{ + sql_query_pre = ', isset($db_character_set) ? 'SET NAMES ' . $db_character_set : '', ' + sql_query_range = \ + SELECT s1.value, s2.value \ + FROM ', $db_prefix, 'settings AS s1, ', $db_prefix, 'settings AS s2 \ + WHERE s1.variable = \'sphinx_indexed_msg_until\' \ + AND s2.variable = \'maxMsgID\' +} + +index smf_base_index +{ + html_strip = 1 + source = smf_source + path = ', $modSettings['sphinx_data_path'], '/smf_sphinx_base.index', empty($modSettings['sphinx_stopword_path']) ? '' : ' + stopwords = ' . $modSettings['sphinx_stopword_path'], ' + min_word_len = 2 + charset_type = ', isset($db_character_set) && $db_character_set === 'utf8' ? 'utf-8' : 'sbcs', ' + charset_table = 0..9, A..Z->a..z, _, a..z +} + +index smf_delta_index : smf_base_index +{ + source = smf_delta_source + path = ', $modSettings['sphinx_data_path'], '/smf_sphinx_delta.index +} + +index smf_index +{ + type = distributed + local = smf_base_index + local = smf_delta_index +} + +indexer +{ + mem_limit = ', (int) $modSettings['sphinx_indexer_mem'], 'M +} + +searchd +{ + listen = ', (int) $modSettings['sphinx_searchd_port'], ' + listen = ', (int) $modSettings['sphinxql_searchd_port'], ':mysql41 + log = ', $modSettings['sphinx_log_path'], '/searchd.log + query_log = ', $modSettings['sphinx_log_path'], '/query.log + read_timeout = 5 + max_children = 30 + pid_file = ', $modSettings['sphinx_data_path'], '/searchd.pid + max_matches = 1000 +} +'; + + flush(); +} + + + + + +function template_sphinx_config_above($title) +{ + global $smfsite, $settings; + + echo ' + + +
+
+
+
+ SMF Sphinx Configuration Utility
+
+
+
+
+
+';
+}
+
?>
\ No newline at end of file
diff --git a/ssi_examples.php b/ssi_examples.php
index 1aca176..0b5b2fe 100644
--- a/ssi_examples.php
+++ b/ssi_examples.php
@@ -419,7 +419,7 @@ function template_ssi_above()
|
+
+
+ ', $title, ''; +} + +function template_sphinx_config_below() +{ + + echo ' + |
+