=======
Indexes
=======

General
=======
MoinMoin relies strongly on indexes that accelerate access to item metadata and
data, and makes it possible to have simple backends, because the index layer
is doing all the hard and complex work.

Indexes are used internally for many operations like item lookup, history,
iterating over items, search, interactive search, etc.

MoinMoin won't be able to start with damaged, inaccessible, or non-existing indexes.
As a result, you will need to configure and initialize indexing correctly first.

Moin will automatically update the index when items are created, updated, deleted,
destroyed, or renamed via the storage API of Moin, the indexing layer, or above.

Configuration
=============
You need to have an ``index_storage`` entry in your wiki config.

We use Whoosh for indexing and, as Whoosh supports multiple storage backends,
this entry is made to potentially support any storage supported by Whoosh.

In general, this entry has the form of::

    index_storage = kind, (p1, p2, ...), {kw1=..., kw2=..., ...}

Currently, we only support the 'FileStorage' kind of index storage, which only
has one parameter - the index directory::

    index_storage = 'FileStorage', ("/path/to/moin-2.0/wiki/index", ), {}

**Notes for FileStorage:**

* The path MUST be absolute, writable and should be on a fast, local filesystem.
* Moin will use `index.temp` directory as well, if you build an index at
  the `temporary location`.


moin index subcommand reference
===============================
You can use the ``moin index-*`` group of CLI subcommands to manage indexes.

Many of the CLI commands for index management support a `--tmp` option to use
the temporary index location. This is useful if you want to do index operations
in parallel to a running wiki which is still using the index at the normal
index location.

moin index-create
-----------------
Creates an empty but valid index.

**Note:** The Moin WSGI application needs an index and storage to successfully start up.
Please see the command moin create-instance.

moin index-build
----------------
Process all revisions of the wiki and add the indexable documents to the index.

**Note:**

* For big wikis, this can take rather long; consider using --tmp.
* index-build does NOT clear the index at the beginning.
* index-build does not check the current contents of the index. Therefore you must not run
  index-build multiple times for the same data or the same wiki.

moin index-update
-----------------
Compare an index to the current storage contents and update the index as
needed (add, remove, update) to reflect the current storage contents.

**Note:** You can use this after building at the tmp location to get
the changes that happened to the wiki while building the index as well. You can run
index-update multiple times to keep even more caught up.

moin index-destroy
------------------
Destroy an index such that nothing is left at the respective location.

moin index-move
---------------
Move the index from the temporary location to the normal location.

moin index-optimize
-------------------
Optimize an index; see Whoosh docs for more details.

moin index-dump
---------------
Output index contents in human-readable form, e.g., for debugging purposes.

**Note:** only fields with attribute ``stored=True`` can be displayed.


Building an index for a single wiki
===================================

If your wiki is fresh and empty
-------------------------------
Use::

    moin index-create

Storage and index are now initialized and both empty.

If you add data to your wiki, the index will get updated automatically.


If your wiki has data and is shut down
--------------------------------------
If the index needs a rebuild for some reason (e.g., index lost, index damaged,
incompatible upgrade, etc.), use::

    moin index-destroy
    moin index-create
    moin index-build  # can take a while...


If your wiki has data and should stay online
--------------------------------------------
Use::

     moin index-create --tmp
     moin index-build --tmp  # can take a while...
     moin index-update --tmp  # should be quicker, make sure we have 99.x%
     # better shut down the wiki now or at least make sure it is not changed
     moin index-update --tmp  # make sure we have indexed all content, should be even quicker.
     moin index-move  # instantaneously
     # start the wiki again or allow changes now again

**Note:** Indexing puts load onto your server, so if you like to do regular
index rebuilds, schedule them at some time when your server is not too busy.


Building an index for a wiki farm
=================================
If you run a wiki farm (multiple related wikis), you may share the index
between the wikis so users will be able to search in one wiki
and also see results from the other wikis.

Before you start, you must prepare your wiki configs. For example, for a company
that uses two farm wikis, such as ``Sales`` and ``Engineering``. Their respective
wiki configs could look like:

``Sales``::

      interwikiname = "Sales"
      index_storage = 'FileStorage', ("/path/to/moin-2.0/wiki/index", ), {}


``Engineering``::

      interwikiname = "Engineering"
      index_storage = 'FileStorage', ("/path/to/moin-2.0/wiki/index", ), {}


Now do the initial index building::

     moin index-create  # create an empty index
     # now add the indexes from both other wikis:
     moin index-build  # with Sales wiki configuration
     moin index-build  # with Engineering wiki configuration

Now you should have a shared index for all wikis.

**Note:** Do not build indexes for multiple wikis in parallel. This is not
supported.