Repositories

A repository is a collection of something, and within Dynamicweb it refers to collections of settings related to indexing and querying different types of data on your solution – such as products, pages, users, files, etc.

This can be used to e.g.:

  • Implement free text search
  • Publish lists of users
  • Create searchable file collections/galleries.

Repositories are also used extensively by the system – e.g. in PIM to make sure product data is continually up to date, to create PIM queries finding products which need to be enriched, and queries belonging to a user-index can be used across the system to select e.g. recipients for an email. Our indexing engine is based on Lucene 3.0.3.

Repositories are created from Settings > Repositories (you must be an administrator user):

  • Right-click the Repositories node and select New repository
  • Give it a Name
  • Click OK

You will then see an empty repository (Figure 1.1).

Repositories can contain the following elements:

  • Indexes – a set of instructions for building an index
  • Queries – used to retrieve data based on the criteria you define
  • Facets – used to create filters in frontend
  • Tasks – used to rebuild indexes at scheduled intervals

They should be added in that order as queries depend on an index, facets depend on queries, and so on. You can read more about each type of element below.

An index is a data structure optimized for data retrieval – which means that it’s much faster to query an index than it is to search through a database directly.

You can create the following types of indexes in Dynamicweb:

Each of these is used to index a particular type of content on a solution, except for the SQL index which is a flexible option when you need to create non-standard indexes. This won’t happen very often – most solutions will need a Product index, a Content index, and maybe a User index. But it’s there for those edge-cases that always pop up.

An index contains the following elements:

  • Instances – the files being searched when a query is executed. You typically want at least 2, so there’s always one to query while the other one is being built-
  • Build configurations – instructions detailing exactly how you want the index to be built. Various settings allow you to tweak how the builder behaves in more advanced scenarios.
  • Field definitions – a way to specify exactly what should be a part of the index. In most cases you will use a default configuration based on a Schema extender we deliver out of the box, but you will also sometimes add fields manually when you want a different behavior than the one we deliver out of the box.
  • Field types – custom field types used when you want to analyze field content in a non-standard manner before adding it to the index

A query is a request for data from an index – such as return all products without a description or return all users who live in Denmark.

Queries are created by stringing together a set of expressions – as in Figure 3.1 where we’re asking for all products which are Active and in SHOP2.

You can think of each expression as being a kind of filter which limits what the query returns by only returning products which match the expression criteria, typically by checking if a field in the index has a specific value. The anatomy of an expression is this:

  • On the left side you have a field in the index – this can be a product field or a generated field
  • In the middle you have an operator – this is the part which defines what you’re actually testing for, e.g. GreaterThan, LessThan, Equal, or IsEmpty.
  • On the right side you have a test value – this is what you’re testing for

The operators available on a specific expression are dependent on the data type of the field your querying. For instance, a boolean is a data type which can only be true or false – and so the only operators available are Equal and In (for comparing against more than one test value). Test values can be constants you define, an term selected from the values in the database, a value returned by a macro, or a value passed to the query via a query parameter. You can read more about the different types of operators and test values in the queries article.

Queries can be used in various places around the system, most commonly:

Facets are controls which are used to filter a list of things published via the index, most commonly a product list (Figure 4.1).

Facets work by showing the user a list of values from a field as e.g. a list of checkboxes, and then passes any selected values to an underlying query which limits the content shown to the content matching all the selected facet values.

There are several ways to create facets:

  • Regular Facets are created under a repository and then applied to a specific Product Catalog/Product Catalog for ViewModel app which makes them available to render in frontend
  • Dynamic Facets are generated based on query string parameters, either from a list of pre-created facets or in a fully dynamic manner

Tasks are scheduled activities which rebuild an index at a predefined interval, e.g. every hour or every day. They are created on a repository and then executed by a standard scheduled task called Repository Task Handler.