TN 005

Alpha Technical Note #005

How Alpha's help search engine works

Introduction
Where is the cache ?
Where are the search results ?
How indexing works
Indexing third party packages
Package keywords
Related Links

Introduction

Since version 9.0.8, Alpha provides a mechanism for searching its documentation and help files. The Help menu has a Search Field where users can enter a search string:

The results of the search are displayed in the Help Viewer as hyperlinks pointing to the corresponding locations.
This document explains how this functionality is implemented. In a nutshell, the help files and packages are indexed and a collection of indexed files is kept in a cache inside the application bundle. When the user enters a search string, Alpha looks up in the cache to find the targets associated with the entered word(s) and then assembles this information in a file in HTML format which is displayed in the help viewer.

Where is the cache ?

The helpsearch cache containing the index files is located inside the application bundle:

Alpha.app/Contents/Resources/Libraries/AlphaTcl/Cache/helpsearch

Where are the search results ?

For every search query, the results are assembled in an HTML file which is displayed by the Help Viewer. These result files are stored in a temporary folder inside Alpha's Preferences folder:

$PREF/tmp/search

where $PREF designates the Preferences folder (its location may vary).
The result files remain available during an Alpha session but are erased when the application quits.

How indexing works

Normally, there is nothing to do: Alpha comes with prebuilt indices. In some circumstances, one may want to regenerate the help search indices. This is quite simple:

activate the Alpha Developer Menu from the Menus Preferences dialog (opened via Alpha ↣ Global Setup ↣ Menus…) ;
then, from the AlphaDev menu, select the Rebuild AlphaTcl Indices menu item ;
in the dialog which is displayed, check the Packages Info and Help Search Info checkboxes. Also check the domains you are interested in (like Inside the Application), then press OK. It takes a few seconds to buid the indices.

The indexing procedure scans several elements:

all the help files contained inside the application bundle and inside the Application Support folders.
all the package names (modes, menus, features, etc.)
all the preindexed files found in Help/Index that may be provided by third party extensions. See the Indexing third party packages section. In particular, Alpha itself provides such a preindexed file obtained by scanning the documentation sources written with the Aida markup language.
all the keywords declarations that may be provided by packages (see the Package keywords section below)

Indexing third party packages

Third party packages (menus, modes, features) designed for Alpha may benefit from the help search mechanism if they conform to the following guidelines.
If their documentation is written using the Aida markup language (as it is the case for the major part of Alpha's documentation), then all section titles are automatically indexed, as well as many labels produced by the Aida macros (see Aida Macros Help). The documentation files should be installed in the Help subfolder of the Application Support folder where the third party package itself is installed.
Additionally, package writers may provide preindexed files that should be installed in the Index subfolder of the Help folder. Two files are required which should be named [packagename]_words and [packagename]_titles respectively, where [packagename] is a name identifying your package. For instance: MyPackage_words and MyPackage_titles.
The [packagename]_words file contains entries for all the indexed words (in lowercase): each line has the format:

word [tabulation] list of targets

where word is an indexed word followed by a tabulation followed by a space-separated list of "targets". The targets may be of different kinds: line numbers or positions in a text file, anchors in an HTML file, page numbers in a pdf file, etc.
The [packagename]_titles file contains entries for all the targets found in the [packagename]_words file. Each line has the form:

target [tabulation] actual title

For the mechanism to work properly, the [packagename]_titles file must also contain (usually near the top) a line of information of the form:

#info: format;relpath;kind

where format is the format of the help files (html, text or pdf), relpath is the subpath of the help files relative to the Help folder, and kind specifies how targets must be interpreted. The kind can be one of the following:

line
position
page[±offset]
anchor
relpath

The best way to understand this is to have a look at the preindexed files found in the Index subfolder of Alpha's Help folder (click here).
Here are a few examples:

#info: html;Extensions/TclAEHelp;relpath

#info: text;Extensions/TclXHelp;line

#info: pdf;latex_guide.pdf;page+4

In the last example, page+4 indicates that to reach page labelled 10 for instance, Alpha has to go to actual page 14 (because the 4 first pages correspond to the table of contents and are labelled in roman numbers)

Package keywords

The package declarations (made using the alpha::mode, alpha::menu, alpha::feature etc. procs) now support a keywords key. The value of this key is a list of words that the package wants to suggest. If the user performs a search on one of these words, the help search engine will provide a link to the package's help page. Here is a snippet coming from Julia mode's declaration:

alpha::mode Juli 0.1 source *.jl JuliaMenu {
    # Script to execute at Alpha startup
    [...]
} keywords {
    programming language numerical computing console
} description {
    Supports the editing and processing of Julia programming files
} help { 
    [...]
}