Commit Graph

16 Commits

Author SHA1 Message Date
Ere Maijala be2bdfd511 Bug 19365: Fix several issues with the Elasticsearch code
Also optimize it so it's actually usable.

Test plan:

1. To test it properly you need biblio and authority data. You might get away with enabling AutoCreateAuthorities and BiblioAddsAuthorities so that authorities are created in the process. Another option would be to import authorities first e.g. from LoC.
2. Make sure the authority index has been properly created with "misc/search_tools/rebuild_elastic_search.pl -a -d"
3. Run "misc/link_bibs_to_authorities.pl -v -l" twice and observe the results.

Sponsored-by: National Library of Finland
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-11-26 16:11:01 +00:00
Julian Maurice ea184236a4 Bug 19502: Add POD for max_result_window
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-07-12 13:04:23 +00:00
Julian Maurice 95b1983a25 Bug 19502: Retrieve index.max_result_window from ES
This avoid hardcoding '10000' in two different places and allow users to
adjust this setting.

Also, this patch fixes a bug when the search return less than 10000
results

Test plan:
1. Do a search that returns 10000+ records.
2. Note the warning above the pagination buttons
3. Go to the last page, no error
4. Change the ES setting:
   curl -XPUT http://elasticsearch/koha_master_biblios/_settings -d \
     '{"index": {"max_result_window": 20000}}'
5. Do another search that returns more than 10000 but less than 20000
6. Note that the warning does not show up
7. Go to the last page, still no error

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
2018-07-12 13:04:21 +00:00
Tomas Cohen Arazi 377bd10e0c Bug 16660: (followup) Unit tests
This patch adds unit tests for the introduced changes in
build_query_compat.

It removes a warning too.

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2017-10-27 14:09:02 -03:00
Srdjan bf6caa81e5 Bug 16660: Moved Opac Supression filtering from opac-search.pl to Zebra::QueryBuilder
To test:
OPAC: Both SearchEngine "Elasticsearch" and "Zebra" should work with
OpacSuppression set to "yes"

NB: OPAC suppression is not implemented for Elasticsearch

Signed-off-by: David Bourgault <david.bourgault@inlibro.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2017-10-27 14:09:02 -03:00
Nick Clemens 9c8a04acb0 Bug 16976 - Authorities searches with double quotes gives ZOOM error 20003 [Alternate]
I dug into this a little bit, it seems the issue is that we were double
processing the query because of an extra call to
C4::AuthoritiesMarc::SearchAuthorities

I think we can simply remove this call

To test:
1 - Perform an authority search for a value that returns results
2 - Wrap it in quotes and search
3 - 'Internal server error'
4 - Apply patch
5 - Search with quotes succeeds

https://bugs.koha-community.org/show_bug.cgi?id=16796

Signed-off-by: Lee Jamison <ldjamison@marywood.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2017-08-15 12:17:40 -03:00
Marcel de Rooy 25c87a7542 Bug 9988: Add extract_biblionumber to Koha/SearchEngine
When we will replace the Zebra code in sub merge, we will call SearchEngine
to pass records and we need a routine to extract a biblionumber from
a search result record. A record from Zebra still must be converted to
MARC::Record. This is no longer needed for a ElasticSearch record.

Test plan:
Run t/db_dependent/Koha/SearchEngine/Search.t

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-04-13 08:53:46 -04:00
Tomas Cohen Arazi 9d9f2efa94 Bug 16489: ES code incorrectly refers to Moose
Probably a leftover from the original SolR code, there are unused references to Moose in the new Elasticsearch code.

To test:
- Run:
  $ git grep Moose
=> FAIL: There are unneeded references to Moose:
Koha/SearchEngine/Zebra/QueryBuilder.pm:use Moose::Role;
Koha/SearchEngine/Zebra/Search.pm:#use Moose::Role;
- Apply the patch
- Run:
  $ git grep Moose
=> SUCCESS: No references to Moose
- Run:
  $ prove t/Search* t/db_dependent/Search*
=> SUCCESS: Tests pass
- Sign off

Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
2016-05-12 16:06:18 -06:00
Jesse Weaver 37ea74669c Bug 12748: (QA followup) fix several tiny QA tools warnings
* admin/searchengine/elasticsearch/mappings.yaml:
    * Fix indentation so QA tools believe it's valid YAML
  * C4::AuthoritiesMarc: Remove tab character
  * C4::Search: Fix POD
  * Koha::BiblioUtils: remove "return undef"
  * Koha::Schema::Result::Search*: rename type to _type
  * Koha::SE::ES::Search:
    * Fix spelling/@_[0]
    * Add my to loop variable
    * Remove unused var $error
    * Add use Modern::Perl
    * Fix undefined variables
  * Koha::ES::Indexer: fix POD
  * Koha::SE::ES::QueryBuilder:
    * Don't define variables in conditionals
    * Fix POD
  * Koha::SE::Zebra::Search:
    * Fix debug warns/POD/barewords
    * Koha::SE::Zebra::Search: add use Modern::Perl
  * kohastructure.sql: remove old, unused elasticsearch_mappings table,
    add new tables
  * labels/label-item-search.pl: remove C4::Dates
  * opac/elasticsearch.pl: remove old file

Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-04-26 20:20:13 +00:00
Jonathan Druart 4b3d5ef686 Bug 12478: Define simple_search_compat for Zebra
When Zebra is defined as the search engine, the authority search is
broken (Can't locate object method "simple_search_compat" via package
"Koha::SearchEngine::Zebra::Search" at /home/koha/koha/C4/AuthoritiesMarc.pm line 357.)
simple_search_compat is not defined in
Koha::SearchEngine::Zebra::Search, it should be a wrapper for
C4::Search::SimpleSearch

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-04-26 20:20:12 +00:00
Robin Sheat 7e87829166 Bug 12478: fix issues caused by rebasing
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-04-26 20:20:06 +00:00
Robin Sheat 2f0fb1b844 Bug 12478: reinstate zebra compatibility for auth search
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-04-26 20:20:05 +00:00
Robin Sheat 7dbd13e66f Bug 12478 - pile of elasticsearch code
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
2016-04-26 20:20:03 +00:00
Jonathan Druart cf2eb49448 Bug 12538: Remove Solr without breaking anything else
Since nobody is currently working on the zebra layer introduced by bug
8233, Solr won't never work.
Some code has been introduced in 3.10 to prove several search engines
can cohabit into Koha but no help/fund has been found to go ahead.
It is useless to keep this code and to maintain an ambiguous situation.

I think the indexes configuration page could be restore later if someone
else introduces a new search engine into Koha.

Test plan:
Look at the code introduced by bug 8233 and verify all is removed.

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2014-10-11 16:59:04 -03:00
Jonathan Druart c3a55042e7 Bug 11015: add copyright headers to some files
This patch adds a copyright headers where they didn't exist in the Koha
namespace.

Signed-off-by: Robin Sheat <robin@catalyst.net.nz>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
2013-10-21 21:45:00 +00:00
Jonathan Druart 623f3a2c84 Bug 8233 : SearchEngine: Add a Koha::SearchEngine module
First draft introducing solr into Koha :-)

List of files :
  $ tree t/searchengine/
  t/searchengine
  |-- 000_conn
  |   `-- conn.t
  |-- 001_search
  |   `-- search_base.t
  |-- 002_index
  |   `-- index_base.t
  |-- 003_query
  |   `-- buildquery.t
  |-- 004_config
  |   `-- load_config.t
  `-- indexes.yaml
  just do `prove -r t/searchengine/**/*.t`

  t/lib
  |-- Mocks
  |   `-- Context.pm
  `-- Mocks.pm
  provide a mock to SearchEngine syspref (set_zebra and set_solr).

  $ tree Koha/SearchEngine
  Koha/SearchEngine
  |-- Config.pm
  |-- ConfigRole.pm
  |-- FacetsBuilder.pm
  |-- FacetsBuilderRole.pm
  |-- Index.pm
  |-- IndexRole.pm
  |-- QueryBuilder.pm
  |-- QueryBuilderRole.pm
  |-- Search.pm
  |-- SearchRole.pm
  |-- Solr
  |   |-- Config.pm
  |   |-- FacetsBuilder.pm
  |   |-- Index.pm
  |   |-- QueryBuilder.pm
  |   `-- Search.pm
  |-- Solr.pm
  |-- Zebra
  |   |-- QueryBuilder.pm
  |   `-- Search.pm
  `-- Zebra.pm

How to install and configure Solr ?
  See the wiki page: http://wiki.koha-community.org/wiki/SearchEngine_Layer_RFC

http://bugs.koha-community.org/show_bug.cgi?id=8233
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
2012-07-06 16:51:58 +02:00