This patch makes the ViewPolicy filter use the 'params' accessor instead
of relying of ->{options} which has no accessor. This will allow
interacting with the filter object be similar through all the filters in
the chain.
To test, we just need to verify no behaviour change takes place:
1. Run:
$ kshell
k$ prove t/db_dependent/Filter_MARC_ViewPolicy.t
=> SUCCESS: Tests pass
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests still pass!
4. Sign off :-D
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Joy Nelson <joy@bywatersolutions.com>
MARC::Record::append_fields takes a list of MARC::Field (not an arrayref)
Use $record->subfield() instead of $record->field()->subfield() to avoid errors
when field doesn't exist
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch makes the EmbedItemsAvailability filter handle the biblio.biblionumber > control field mapping gracefully.
Right now, it was assuming a regular data field was used, and such is not the case for UNIMARC (001).
To test:
- Apply the tests patch
- Run:
$ sudo koha-shell kohadev
k$ cd kohaclone
k$ prove t/db_dependent/Koha/Filter/EmbedItemsAvailability.t
=> FAIL: Tests fail to run
- Apply this patch
- Run:
k$ prove t/db_dependent/Koha/Filter/EmbedItemsAvailability.t
=> SUCCESS: Tests pass!
- Sign off :-D
Sponsored-by: ByWater Solutions
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Removes the loop by moving the onloan test to the query.
Test plan:
Run the test again.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch adds a new filter for MARC records to be used with
Koha::RecordProcessor. It's purpose is to inject the information about
items not-onloan in a fixed subfield, 999$x.
To test:
- Apply the patch
- Run:
$ prove t/db_dependent/Koha/Filter/EmbedItemsAvailability.t
=> SUCCESS: Tests pass!
- Sign off :-D
Sponsored-by: ByWater Solutions
Followed test plan, test passes OK
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
GetMarcStructure() currently uses Koha::Cache in the "safe" mode
(returning deep copy of the result data structure by default), which
causes numerous performance issues in many Koha scripts. Switching
it to the "unsafe" mode globally (2nd patch from Bug 16140) resolves
those issues, but ensuring that it is regression-free (and that it
will stay that way in the future) is far from easy. This patch
proposes a bit more manageable solution, it introduces
a possibility to use "unsafe" variant selectively (only in those
places in the code where GetMarcStructure() is called repetitively).
That way, amount of the code that needs to be audited for possible
problems gets vastly reduced, without any performance trade-offs.
Test plan:
1) Have a look at the code and audit the parts affected by this patch
for possible regressions
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Amended the POD of GetMarcStructure, removing a TODO.
NOTE: GetAuthorisedValueDesc, as called in C4::XSLT::transformMARCXML4XSLT
and by GetISBDView, GetMarcAuthors in C4::Biblio, may autovivify some hash
entries in tagslib.
Same for Koha/Filter/MARC/ViewPolicy.pm, sub filter.
No reason however to worry; our use of this structure in Koha does not
depend on the existence of intermediate hash keys. (We seem to be safe as
long as $tagslib->{$tag}->{$subfield}->{tab} and/or hidden are not filled.)
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Add should_hide_marc to ViewPolicy filter
Add should_hide_marc tests to t/db_dependent/Filter_MARC_ViewPolicy.t
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch:
- makes the Koha::RecordProcessor code more clear
by removing the unnecessary newrecord variable.
- revises the filter to be more clear about the
expectation that operations are done directly on
the record parameter.
TEST PLAN
---------
prove t/RecordProcessor.t
prove t/db_dependent/Filter_MARC_ViewPolicy.t
run koha qa test tools
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Works as advertised. NO koha-qa errors
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
This patch:
- improves perlcritic messages in the filter and tests.
- changes should display logic to should hide logic
to simplify filter.
- perltidies the scripts
- debugs the issues outstanding on the comprehensive tests
provided in the second commit.
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
TEST PLAN
---------
0) Apply 15777 and 15871 as required.
1) Apply patch
2) prove -v t/db_dependpent/RecordProcessor_ViewPolicy.t
-- all should pass.
3) koha qa test tools.
BONUS
4) perlcritic -1 t/db_dependpent/RecordProcessor_ViewPolicy.t
-- using the koha qa test tools perlcriticrc, it should pass
though I am excluding checking print return values.
(see bug 15646)
Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
Mainly a
perl -p -i -e 's/^.*3.07.00.049.*\n//' **/*.pm
Then some adjustements
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
The existing Koha::Authority does not use Koha::Objects and does not
exclusively deal with DB objects.
It makes much sense to move it to let the space free for a
Koha::Authority and Koha::Authorities modules based on Koha::Object.
Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
If the system preference IncludeSeeFromInSearches is enabled, records
exported for zebra indexing are being additionally processed by
EmbedSeeFromHeadings record filter (right now used only in rebuild_zebra.pl
script). This filter embeds 'see from' fields (extracted from authority
records linked with the given biblio via $9 subfields) into target MARC
record, which is then subsequently indexed in zebra.
Currently all fields containing $9 are getting the same exact treatment
by this filter. But on the export stage when the filter is applied, MARC
record being processed already does have holdings data fields added in
the previous stage (usually 952 / 995, depending on the MARC format).
Problem is that holdings data fields use to have $9 subfields in them
as well (mapped to item.itemnumber by default). As a consequence, some
(great many in the typical setup) records exported for zebra indexing
may have surplus "see from" fields added erroneously in semi-random
fashion, so biblio searches would often return some completely
unexpected additional results.
EmbedSeeFromHeadings record filter should not process holdings fields
when dealing with MARC records intended for zebra indexing.
To reproduce:
1) database with as many sample or real-world biblio, item and authority
records as possible is recommended for testing purposes
2) enable IncludeSeeFromInSearches
3) export a bunch of biblio records for zebra (e.g.:
misc/migration_tools/rebuild_zebra.pl -I -b -x -k -length=1000),
inspect the result xml records in /tmp/<whatever> file; observe that at
the end of many records, here and there some extra "see from" (= 1st
indicator: 'z') fields tend to appear, which shouldn't be there ;)
To test:
4) apply patch
5) redo 3)
6) compare results from 3) and 5) with diff
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
I introduced a regression test for this. You should run the tests
without/with the patch and verify that the patch actually fixes the problem.
Good job Jacek! I'm sure writing the regression test would take less time
than such a detailed commit message!
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
http://bugs.koha-community.org/show_bug.cgi?id=9987
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
In Germany 020$9 is used to store the ISBN with hyphens. In rare cases this
can break indexing in combination with IncludeSeeFromInSearches.
To reproduce:
1) Change your frameworks:
* Make sure 084 $a is visible
* Create a 084 $9 that you link to PERSO_NAME authorities.
2) Create a record or edit an existing one:
* Use the plugin on 084 to link the record to an authority.
* Make sure 400 is filled in your authority record.
Preparations all done, now test indexing:
3) Make sure IncludeSeeFromInSearches is OFF
* Reindexing should work without any problems.
4) Activate IncludeSeeFromInSearches
* Reindexing should throw an error and stop:
201.Subfields are only for data fields (generally,
just tags >= 010 at /home/koha/Koha/Filter/MARC/EmbedSee
FromHeadings.pm line 94
5) Apply patch and reindex.
* No error should occur.
* Test 4xx fields from other tags were properly indexed.
What causes the problem is the fact, that Koha tries to store
the date from the 400 in the bibliographic record for indexing.
For this it calculates a tag in the bibliographic record from the
tag it found the authority link in and the 4xx tags used in the
authority record. If you have an authority linked to a 0xx field
and the matching authority has a 4xx where xx < 10, the calculated
tag will be a control field. The attempt to add subfields to this
control field, is causing the error and breaks the indexing process.
Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>
Comment: Work as described, and is a long description :)
Test:
1) Enabled 084 (all subfields visible), added 084_9 (missing from
default framework) linked to PERSO_NAME
2) Edited authority record and added some data to 400
3) Edited bib record, linked 084 to previous auth record
4) With IncludeSeeFromInSearches OFF, re-index without problem
5) With IncludeSeeFromInSearches ON, re-index problems!!!
====================
exporting biblio
====================
1...Subfields are only for data fields (generally, just tags >= 010) at...
Abort
6) Applied patch
7) With IncludeSeeFromInSearches ON, re-index without problem
...
====================
5001....................................................................................................
Records exported: 5060
====================
...
====================
2001....................................................................................................
Records exported: 2024
====================
CLEANING
====================
Good!
*) That's "works as described", don't know if other tests are needed.
No errors from koha-qa
Signed-off-by: Elliott Davis <elliott@bywatersolions.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
This patch adds the Koha::Indexer::RecordNormalizer and
Koha::Indexer::MARC::RecordNormalizer::EmbedSeeFromHeadings packages
to enable the inclusion of alternate forms of headings in bibliographic
searches. When the new syspref IncludeSeeFromInSearches is turned on
(default is off) rebuild_zebra.pl will insert see from headings from
authority records into bibliographic records when indexing, so that a
search on an obsolete term will turn up relevant records.
To test:
1) Enable IncludeSeeFromInSearches
2) Add a heading that has an alternate form to a record (for example,
"Cooking" has the alternate form "Cookery," if you have authority
records from LC)
3) Index the zebraqueue (or reindex if you haven't indexed your system
yet)
4) Confirm that if you search for "Cookery" you get the record you
just modified
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 5 August 2012
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 11 September 2012
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Also checked:
- Verified database update works correctly
- Checked system preference and its description
- Checked staff/opac detail pages with feature on/off
- Checked staff/opac search facets
- Downloaded and tested records in various formats
- Tried different searches for 'see from' entries of authorities
- Ran all unit tests
No problems found.
Tomas Cohen Arazi
Josef Moravec
Julian Maurice
Marcel de Rooy
Jacek Ablewicz
Mark Tompsett
Jonathan Druart
Jonathan Druart
Katrin Fischer
Jared Camins-Esakov