This patch:
- makes the Koha::RecordProcessor code more clear
by removing the unnecessary newrecord variable.
- revises the filter to be more clear about the
expectation that operations are done directly on
the record parameter.
TEST PLAN
---------
prove t/RecordProcessor.t
prove t/db_dependent/Filter_MARC_ViewPolicy.t
run koha qa test tools
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Works as advertised. NO koha-qa errors
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
This patch:
- improves perlcritic messages in the filter and tests.
- changes should display logic to should hide logic
to simplify filter.
- perltidies the scripts
- debugs the issues outstanding on the comprehensive tests
provided in the second commit.
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
TEST PLAN
---------
0) Apply 15777 and 15871 as required.
1) Apply patch
2) prove -v t/db_dependpent/RecordProcessor_ViewPolicy.t
-- all should pass.
3) koha qa test tools.
BONUS
4) perlcritic -1 t/db_dependpent/RecordProcessor_ViewPolicy.t
-- using the koha qa test tools perlcriticrc, it should pass
though I am excluding checking print return values.
(see bug 15646)
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
Mainly a
perl -p -i -e 's/^.*3.07.00.049.*\n//' **/*.pm
Then some adjustements
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
The existing Koha::Authority does not use Koha::Objects and does not
exclusively deal with DB objects.
It makes much sense to move it to let the space free for a
Koha::Authority and Koha::Authorities modules based on Koha::Object.
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
If the system preference IncludeSeeFromInSearches is enabled, records
exported for zebra indexing are being additionally processed by
EmbedSeeFromHeadings record filter (right now used only in rebuild_zebra.pl
script). This filter embeds 'see from' fields (extracted from authority
records linked with the given biblio via $9 subfields) into target MARC
record, which is then subsequently indexed in zebra.
Currently all fields containing $9 are getting the same exact treatment
by this filter. But on the export stage when the filter is applied, MARC
record being processed already does have holdings data fields added in
the previous stage (usually 952 / 995, depending on the MARC format).
Problem is that holdings data fields use to have $9 subfields in them
as well (mapped to item.itemnumber by default). As a consequence, some
(great many in the typical setup) records exported for zebra indexing
may have surplus "see from" fields added erroneously in semi-random
fashion, so biblio searches would often return some completely
unexpected additional results.
EmbedSeeFromHeadings record filter should not process holdings fields
when dealing with MARC records intended for zebra indexing.
To reproduce:
1) database with as many sample or real-world biblio, item and authority
records as possible is recommended for testing purposes
2) enable IncludeSeeFromInSearches
3) export a bunch of biblio records for zebra (e.g.:
misc/migration_tools/rebuild_zebra.pl -I -b -x -k -length=1000),
inspect the result xml records in /tmp/<whatever> file; observe that at
the end of many records, here and there some extra "see from" (= 1st
indicator: 'z') fields tend to appear, which shouldn't be there ;)
To test:
4) apply patch
5) redo 3)
6) compare results from 3) and 5) with diff
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
I introduced a regression test for this. You should run the tests
without/with the patch and verify that the patch actually fixes the problem.
Good job Jacek! I'm sure writing the regression test would take less time
than such a detailed commit message!
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
http://bugs.koha-community.org/show_bug.cgi?id=9987
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
In Germany 020$9 is used to store the ISBN with hyphens. In rare cases this
can break indexing in combination with IncludeSeeFromInSearches.
To reproduce:
1) Change your frameworks:
* Make sure 084 $a is visible
* Create a 084 $9 that you link to PERSO_NAME authorities.
2) Create a record or edit an existing one:
* Use the plugin on 084 to link the record to an authority.
* Make sure 400 is filled in your authority record.
Preparations all done, now test indexing:
3) Make sure IncludeSeeFromInSearches is OFF
* Reindexing should work without any problems.
4) Activate IncludeSeeFromInSearches
* Reindexing should throw an error and stop:
201.Subfields are only for data fields (generally,
just tags >= 010 at /home/koha/Koha/Filter/MARC/EmbedSee
FromHeadings.pm line 94
5) Apply patch and reindex.
* No error should occur.
* Test 4xx fields from other tags were properly indexed.
What causes the problem is the fact, that Koha tries to store
the date from the 400 in the bibliographic record for indexing.
For this it calculates a tag in the bibliographic record from the
tag it found the authority link in and the 4xx tags used in the
authority record. If you have an authority linked to a 0xx field
and the matching authority has a 4xx where xx < 10, the calculated
tag will be a control field. The attempt to add subfields to this
control field, is causing the error and breaks the indexing process.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Comment: Work as described, and is a long description :)
Test:
1) Enabled 084 (all subfields visible), added 084_9 (missing from
default framework) linked to PERSO_NAME
2) Edited authority record and added some data to 400
3) Edited bib record, linked 084 to previous auth record
4) With IncludeSeeFromInSearches OFF, re-index without problem
5) With IncludeSeeFromInSearches ON, re-index problems!!!
====================
exporting biblio
====================
1...Subfields are only for data fields (generally, just tags >= 010) at...
Abort
6) Applied patch
7) With IncludeSeeFromInSearches ON, re-index without problem
...
====================
5001....................................................................................................
Records exported: 5060
====================
...
====================
2001....................................................................................................
Records exported: 2024
====================
CLEANING
====================
Good!
*) That's "works as described", don't know if other tests are needed.
No errors from koha-qa
Signed-off-by: Elliott Davis <elliott@bywatersolions.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
This patch adds the Koha::Indexer::RecordNormalizer and
Koha::Indexer::MARC::RecordNormalizer::EmbedSeeFromHeadings packages
to enable the inclusion of alternate forms of headings in bibliographic
searches. When the new syspref IncludeSeeFromInSearches is turned on
(default is off) rebuild_zebra.pl will insert see from headings from
authority records into bibliographic records when indexing, so that a
search on an obsolete term will turn up relevant records.
To test:
1) Enable IncludeSeeFromInSearches
2) Add a heading that has an alternate form to a record (for example,
"Cooking" has the alternate form "Cookery," if you have authority
records from LC)
3) Index the zebraqueue (or reindex if you haven't indexed your system
yet)
4) Confirm that if you search for "Cookery" you get the record you
just modified
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 5 August 2012
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 11 September 2012
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Also checked:
- Verified database update works correctly
- Checked system preference and its description
- Checked staff/opac detail pages with feature on/off
- Checked staff/opac search facets
- Downloaded and tested records in various formats
- Tried different searches for 'see from' entries of authorities
- Ran all unit tests
No problems found.
Mark Tompsett
Jonathan Druart
Jacek Ablewicz
Jonathan Druart
Katrin Fischer
Jared Camins-Esakov