Skip to end of metadata
Go to start of metadata

Overview

This page describes how to configure the layout of fields in a search webapp.

Note that the format of the data has to be just right. You should test your changes before committing to GitHub! See below.

The CSpace Django Search webapp (aka "the Portal") comes in two or more varieties. Usually, there is a Public portal, which requires no authentication, and an Internal portal, which does.  Each variety requires three configuration files:

  • a small .cfg file which configures the variety itself.  Here's the one for the UCJEPS Public portal:
     [search]
# the following file is expected in the config directory. Don't forget to put it there!
FIELDDEFINITIONS = ucjepspublicparms.csv
  • a "field definitions file", in a minimal tab-separated .csv format, which describes all the fields that may appear in the Portal and how they are to be searched and displayed.
  • Parenthetically, there is also a third configuration file specifying parameters all search webapps have in common, such as program limits (e.g. the maximum number of results to return) and the location of services (such as bMapper and the CSpace server to point to for external links. You can find an example of the common configuration file here: common.cfg)

This document describes the second configuration file, the "field definitions file", which tells the program how to render fields for search and display.

In particular, it describes:

  • Where to find the Solr server and core to use for searches.
  • How to layout the search form: what fields to include, how to lay them out (rows and columns), and how to accept user input (input boxes, dropdowns, checkboxes, etc.)
  • How to render search results: what fields to include(and in what order) for the List, Facets, Full, and Grid displays, and what fields to include in downloaded .csv files.

The file is in the form of a 2-D table ("spreadsheet") with two sections, a Preamble (contain attribute-value pairs for a few search-specific parameters) and a Field Definition section, containing a row for each data field used in the Search app, each containing 15 columns of data about the function of the data field in the app.

NB: this file may be, indeed should be, edited as a spreadsheet; however, it must be saved in a text format, with the extension .csv. I must use tabs as field delimiters, the values must not be "encapsulated" (e.g. with quotes or other characters). See the versions in GitHub now for guidance on how they should look.

Here are the existing versions of these files for the various portals:

The first column of the table is special: it is a directive or label that determines how the rest of the columns are interpreted. It contains one of the following strings: (# date revision title server core header field)

#this value in column 1 indicates a comment, it may start any row and is ignored by the webapp

Preamble

These specifications are included in the top of the spreadsheet in Columns A and B.  These may appear only once in the file.

ParameterMeaningExample
datedate of last update (not used or displayed at the moment, maintained by developer)12/12/2015
revisionrevision number (not used or displayed at the moment, maintained by developer)1.3
titleTitle text that appears in the banner on the Search pageCollections Browser
serverURL of solr serverhttp://localhost:8983/solr
corename of the solr core to usepahma-internal

Field Definitions

Field definitions start with a header row ('header' in column A) followed by field definitions ('field' in column A).  The header row is special, unique, and must contain the column header values specified in the Header column below, in any order. The first eight columns provide some "housekeeping info" (e.g. a sequence number, for convenient reference) and functional info such as the name of the field in the Solr core.

ColumnHeaderMeaningRemarks
A 'header' or 'field' (indicates to the webapp how to treat the row, as the header or a field definition) 
Bsequenceunique sequence number of the file in this file; used only to help keep the file in order, e.g. if it gets sorted or something. 
CNamename of the field used internally by the webapp. Only 'id' is special and required. 
DLabellabel to use when the field is displayed in results or forms 
ESolrFieldname of the field in Solr (all dynamic, except id; types include _s, _ss, _p, _dt)(1)
FSearchabledetermines whether the field should be allowed to be search (currently not implemented) 
GSuggestionsfor suggestions from postgres, this two letter code indicates which authority term suggestions should come from(2)
HRolespecifies how the field is to be treated in forms and displays. Options: keyword, dropdown, etc. Some are special and required see below)(3)


The following seven columns specify field layout, which repeat, one for each variable that is included somewhere in the Portal. This row specifies how to layout the fields in forms, displays, and output files.

ColumnHeaderMeaningRemarks
ISearchrow and column (r,c) for the field in the search form.(4)
JFacetspecifies the order of fields in the Facets pane. Integers from 1 to N(5)
KbMapperspecifies the order of fields in the bMapper 'tabfile' pane. Integers from 1 to N.(6)
LlistDisplayspecifies the order of fields in the List pane. Integers from 1 to N(7)
MfullDisplayspecifies the order of fields in the Full pane. Integers from 1 to N(8)
NgridDisplayspecifies the order of fields in the Grid pane. Integers from 1 to N(9)

O

mapDisplayspecifies the order of fields in the Map subpane. Integers from 1 to N(10)
PinCSVspecifies the order of fields in the .csv output. Integers from 1 to N 
Qplaceholdervalue to make sure all columns, even if empty are present in this file.

(11)

Notes

(1)the Search app knows how to handle several of these dynamic types; see http://wiki.apache.org/solr/SchemaXml#Dynamic_fields
(2)Currently, the suggestposgres term-suggestion webapp supports about a dozen options: lo=location, ta=taxon, etc.
(3)Multiple roles may be specified, and some roles MUST be specified for the app to work. The required roles include objectno, accession, sortkey, mainentry, and location. Alas, there is currently no further discussion of what these roles actually do.
(4)The rows and columns must form a complete sequence (N x M rectangle). If you want to leave a hole somewhere, make a dummy field for it, with no label
(5)Maximum number and order of fields is an aesthetic choice – the app is not able to ensure a pretty layout.
(6)Do NOT include the field defined with Role=location; this field is added automatically. The layout here must match the layout specified in the bMapper config file. See the wiki page on bMapper configuration.
(7)Maximum number and order of fields is an aesthetic choice – the app is not able to ensure a pretty layout.
(8)Note that the fields defined with Roles = accession and mainentry are always shown, specially formatted, in this display, and so they should not be included here.
(9)Note that the fields defined with Roles = accession and mainentry are always shown, specially formatted, in this display, and so they should not be included here.
(10)The Map subpane appears in the Full display when the user has clicked the Show Map link. The values enumerated here (e.g Datum) are included in this pane.
(11)a value of 'x' has been placed in column P to guarantee the field count up to that point (some spreadsheet programs eliminate trailing empty cells from the saved file)

Example of a Field Definitions File

Below is an example of the preamble and first few rows of field definitions from pahmainternalparms.csv. Column P, which always contains an X in this example is optional – some spreadsheet programs eliminate trailing empty columns when saving a tab-separated file, and this can cause the file to be unparseable. Ensure that your file has all the columns!

ABCDEFGHIJKLMNOP
#parameter specification spreadsheet for pahma internal portal          x
#              x
date06/15/15            x
revision1             x
titleCollections (internal use only)           x
#              x
serverhttp://localhost:8983/solr           x
corepahma-internal           x
#              x
# Fields     row,[column]      x
#              x
header NameLabelSolrFieldSearchableSuggestionsRoleSearchFacetbMapperlistDisplayfullDisplaygridDisplayinCSVx
field1ididid  id       x
field2csidObject CSIDcsid_s  csid      31x
field3musnoMuseum Numberobjmusno_sx 

objectno,

accession

2,1 111 2x
field4sortnumSortable Museum Numberobjsortnum_s  string,sortkey      1x
field5nameObject Nameobjname_sxna

keyword,

mainentry

5,12222 10x
field6altnumAlternate Numberobjaltnum_ssx keyword3,1   3 11x
field7typeObject Typeobjtype_sx dropdown6,21  4 4x
field8countObject Countobjcount_sx string    5 5x

Suggestions and Reminders

Below is a short recipe for modifying and testing changes to the "field definitions file".

  1. Deploy and configure the Django project for your tenant, including search webapps, on your local machine (under PyCharm, for ease of development)
  2. Edit the "live" version of .csv file (i.e. the one in config/) using appropriate program (OpenOffice or LibreOffice, etc. Excel can be used, but u need to be careful).

  3. Save as .csv, "tabs only, no encapulation".
  4. Restart debugger/server in Pycharm, or dev server if you are using raw Django. This is needed to tell Django to re-read this configuration file.
  5. Check layouts in browser at localhost:8000 (or wherever you have it running)
  6. Rinse and repeat until desired layout and output is achieved.
  7. Copy the new, improved .csv file to the django_example_config repo  (or wherever) and commit it for safekeeping.
  8. Deploy to Dev and eventually Prod servers.
  • No labels