Commit Graph

44 Commits

Author SHA1 Message Date
David Nind ef2f7524c0
Bug 41494: Rename "Koha administration" module name to "Administration"
This changes "Koha administration" to "Administration" on the
staff interface home page and the administration module home page.

Everywhere else in the staff interface, such as browser page titles
and breadcrumbs, administration is used.

This improves consistency, and is how it is commonly referred to.

It also change some other occurences in the code base, such
as code comments.

Test plan:
1. Note the inconsistencies in how we name and refer to the
   administration module:
   1.1 Staff interface home page: Koha administration
   1.2 Administration module home page: Koha administration, but note
       that the breadcrumb and browser page title is "Administration".
   1.3 Breadcrumbs and browser page titles for any of the administration
       areas are just "Administration".
   1.4 More > Administration.
2. Apply the patch.
3. Note that on the staff interface home page and the administration
   home page are now labelled "Administration".

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2026-01-06 07:45:37 -07:00
Tomas Cohen Arazi 1873d97c07
Bug 40163: Fix several http addresses in the code docs
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2025-09-23 16:21:29 -06:00
Janusz Kaczmarek d0ce815600
Bug 40128: StripWhitespaceChars can create empty subfields
When StripWhitespaceChars is enabled and the user by chance enters
some white spaces in the input field, Koha generates empty subfield
(or even entirely empty field).

Test plan:
==========
1. Enable StripWhitespaceChars system preference.
2. Edit an existing record, putting just some spaces into an empty
   subfield. Save the record.
3. Go to bibliographic record Normal view and in modal "MARC preview"
   notice that the subfield you entered spaces in exists as an empty
   subfield. If it was the only subfield in a field, the field exists
   with no data at all. This is wrong.
4. Apply the patch ; restart_all.
5. Repeat 2. and 3. You should not see any empty subfield now. If the
   empty subbfield was the only subfield in a field, the field should
   have been removed from the record.

Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2025-07-15 16:32:53 -06:00
Jonathan Druart 69b8341a8c
Bug 39325: Manual fixes
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2025-04-28 18:51:23 +02:00
Jonathan Druart 1d844a7d2f
Bug 39325: Automatic fix
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2025-04-28 18:51:22 +02:00
Jonathan Druart 15a030bc76
Bug 39319: Fixes for Pod::Checker
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2025-04-14 08:51:21 +02:00
Koha Development Team d659526b5a
Bug 38664: Tidy the whole codebase
This commit is generated using:
  % perl misc/devel/tidy.pl
*within* ktd, to get the same version of perltidy than what will be used
by our CI (currently v20230309).

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2025-02-11 14:58:24 +01:00
Kyle M Hall 52f94659c1
Bug 34481: Tidy altered code
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-06-27 14:04:50 +02:00
Fridolin Somers cb085034ff
Bug 34481: Add IncludeSeeAlsoFromInSearches like IncludeSeeFromInSearches
Like IncludeSeeFromInSearches adds see from heading in search (4xx),
new system preference IncludeSeeAlsoFromInSearches adds see also from heading (5xx).

Test plan:
Test on both Elasticsearch and Zebra:
1)
1.1) Create an authority record with heading, see from heading 4xx and see also from heading 5xx
1.2) Use this authority in a biblio record
2)
2.1) Set IncludeSeeFromInSearches and IncludeSeeAlsoFromInSearches to "Don't include"
2.3) Rebuild search engine
2.4) Perform a search on heading text => you find the biblio record
2.5) Perform a search on see from heading => you do not find the biblio record
2.6) Perform a search on see also from heading => you do not find the biblio record
3)
3.1) Set IncludeSeeFromInSearches and IncludeSeeAlsoFromInSearches to "Include"
3.2) Rebuild search engine
3.3) Perform a search on heading text => you find the biblio record
3.4) Perform a search on see from heading => you find the biblio record
3.5) Perform a search on see also from heading => you find the biblio record

Signed-off-by: Frédéric Demians <f.demians@tamil.fr>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-06-27 14:04:49 +02:00
Kyle M Hall ef9d9a006a
Bug 33749: (QA follow-up) Tidy code for qa script
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:08:07 -03:00
Tomas Cohen Arazi 54477ecc51
Bug 33749: Add TrimFields field class
This patch adds a RecordProcessor filter class that takes care of
performing the field cleanup that was originally done in the static
method `Koha::MetadataRecord::stripWhitespaceChars`.

In order to test the results, I adapted the tests to use the filter
instead of the original method.

To test:
1. Run:
   $ ktd --shell
  k$ prove t/Koha_MetadataRecord.t
=> SUCCESS: Tests pass
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests still pass
4. Sign off :-D

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-03 12:08:06 -03:00
Fridolin Somers 4cb5f36c2a
Bug 32464: Koha::Item->as_marc_field obsolete option mss
Since Bug 28445 the method Koha::Item->as_marc_field has the option mss not anymore.

This patch removes this in Koha::Filter::MARC::EmbedItems.

Test with :
prove t/db_dependent/Koha/Filter/EmbedItems.t

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-06-06 15:40:15 -03:00
Tomas Cohen Arazi c14951baee Revert "Bug 15187: Index 880 in Zebra the same as Elasticsearch"
This reverts commit 85690a14f8.
2022-10-24 14:03:44 -03:00
David Cook 85690a14f8
Bug 15187: Index 880 in Zebra the same as Elasticsearch
This patch modifies the MARC21 export to Zebra, so that 880 fields
are rewritten as their linked fields, in the same way that we
already do with Elasticsearch, so that the alternate graphic
representation of fields are indexed accordingly. (ie 880 $6245-01
Chinese titles will be indexed into the title index using the 245 rules)

Test plan:
0. Apply patch
1. Turn on ICU indexing
1b. vi /etc/koha/zebradb/etc/default.idx
1c. Replace charmap word-phrase-utf.chr with icuchain words-icu.xml
1d. Replace charmap word-phrase-utf.chr with icuchain phrases-icu.xml
1e. Restart Zebra server
1f. Re-index Zebra
2. Add record with a 880 $6 245-01 $a 教牧書信 field.
3. Search for this record using a title index with the Chinese title
4. Note that the record is correctly retrieved

(Note: This test probably works better using author or series as they
 present as links on the detail page which makes the fix more obviously
 useful.)

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-07-22 15:09:46 -03:00
Martin Renvoize 6c381c4b22
Bug 30848: Fix issue exposed by unit tests
This patch changes the 'map' to a simple for loop so that we can easily
map multiple subfields to a field without overwriting the first previous
subfields in the structure.

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-07-01 09:15:46 -03:00
Martin Renvoize 62bba6d9e1
Bug 30848: Add an ExpandCodedFields RecordProcessor filter
This patch introduces a RecordProcessor filter for MARC::Record objects
that replaces Koha codes with descriptions in the MARC::Record passed to the processor.

Target usage:

  my $biblio = Koha::Biblios->find(
      $biblio_id,
      { prefetch => [ metadata ] }
  );

  my $record = $biblio->metadata->record;

  my $processor = Koha::RecordProcessor->new(
    {
        filters => ('ExpandCodedFields'),
        options => {
            interface     => 'opac',
            frameworkcode => $biblio->frameworkcode
        }
    }
  );

  $processor->process( $record );

Test plan
* Read the included unit test and confirm it makes sense and passes

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2022-07-01 09:15:46 -03:00
Tomas Cohen Arazi 43abf06b64 Bug 29764: Fix incorrect EmbedItems RecordProcessor filter POD
This patch makes the usage POD for the EmbedItems filter actually be
usable. The filter is used in opac-ISBDdetail.pl and the POD is updated
to how it is used there.

Not much to test.

Signed-off-by: David Cook <dcook@prosentient.com.au>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2022-01-19 12:13:49 -10:00
Nick Clemens df63fe79e0 Bug 25030: (QA follow-up) Add POD
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-21 10:09:50 +02:00
Julian Maurice 2c0d09e993 Bug 25030: Implement IncludeSeeFromInSearches with Elasticsearch
Test plan:
1. Create an authority with at least a 1XX$a and a 4XX$a, for instance:
    100 $a Foo
    400 $a Bar
2. Create a biblio and add a link to this authority using the
   cataloguing plugin
3. Disable syspref IncludeSeeFromInSearches
4. Reindex the biblio. You should be able to find it when searching
   'Foo' but not when searching 'Bar'
5. Enable syspref IncludeSeeFromInSearches
6. Reindex the biblio. You should be able to find it when searching
   'Foo' and also when searching 'Bar'
   without the patch, 'Bar' doesn't yeld results. With it, it does.
7. prove t/db_dependent/Koha/SearchEngine/Elasticsearch.t

Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr>

Signed-off-by: Lucy Vaux-Harvey <lucy.vaux-harvey@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-10-21 10:09:50 +02:00
Jonathan Druart 9d6d641d1f Bug 17600: Standardize our EXPORT_OK
On bug 17591 we discovered that there was something weird going on with
the way we export and use subroutines/modules.
This patch tries to standardize our EXPORT to use EXPORT_OK only.

That way we will need to explicitely define the subroutine we want to
use from a module.

This patch is a squashed version of:
Bug 17600: After export.pl
Bug 17600: After perlimport
Bug 17600: Manual changes
Bug 17600: Other manual changes after second perlimports run
Bug 17600: Fix tests

And a lot of other manual changes.

export.pl is a dirty script that can be found on bug 17600.

"perlimport" is:
git clone https://github.com/oalders/App-perlimports.git
cd App-perlimports/
cpanm --installdeps .
export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib"
find . \( -name "*.pl" -o -name "*.pm" \) -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \;

The ideas of this patch are to:
* use EXPORT_OK instead of EXPORT
* perltidy the EXPORT_OK list
* remove '&' before the subroutine names
* remove some uneeded use statements
* explicitely import the subroutines we need within the controllers or
modules

Note that the private subroutines (starting with _) should not be
exported (and not used from outside of the module except from tests).

EXPORT vs EXPORT_OK (from
https://www.thegeekstuff.com/2010/06/perl-exporter-examples/)
"""
Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members.

@EXPORT and @EXPORT_OK are the two main variables used during export operation.

@EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace.

@EXPORT_OK does export of symbols on demand basis.
"""

If this patch caused a conflict with a patch you wrote prior to its
push:
* Make sure you are not reintroducing a "use" statement that has been
removed
* "$subroutine" is not exported by the C4::$MODULE module
means that you need to add the subroutine to the @EXPORT_OK list
* Bareword "$subroutine" not allowed while "strict subs"
means that you didn't imported the subroutine from the module:
  - use $MODULE qw( $subroutine list );
You can also use the fully qualified namespace: C4::$MODULE::$subroutine

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-07-16 08:58:47 +02:00
Tomas Cohen Arazi 58c275e04b Bug 23583: Handle OpacHiddenItems with yaml_preference
This patch makes all places in Koha that rely on OpacHiddenItems
actually use C4::Context->yaml_preference instead of manually calling
the YAML libraries and handling it.

To test:
1. Apply this patch
2. Run:
   $ kshell
  k$ prove t/db_dependent/Items* \
           t/db_dependent/Koha/Item* \
           t/db_dependent/Koha/Filter/EmbedItems.t
=> SUCCESS: Tests pass!
3. Try hiding things with opac-ISBDdetail.pl and opac-search.pl
=> SUCCESS: Things work the same!
4. Sign off :-D

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-04-21 15:25:07 +02:00
James O'Keeffe 2b607d8414 Bug 24000: Some modules do not return 1
This patch adds "1;" to the end of the following perl files:
Koha/Filter/MARC/EmbedItemsAvailability.pm
Koha/Filter/MARC/EmbedSeeFromHeadings.pm
Koha/Filter/MARC/Null.pm
Koha/Item/Search/Field.pm
Koha/SearchEngine.pm
misc/translator/VerboseWarnings.pm
t/lib/Koha/Plugin/Test.pm

This indicates the succesful execution of the initialization code.

Test plan:
Ensure that there are no other perl files that need "1;", but dont have
it.

Signed-off-by: David Nind <david@davidnind.com>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-04-06 15:56:30 +02:00
Jonathan Druart fafcbff015 Bug 27673: Fix encoding issues
There is a difference between YAML::Load and YAML::XS::Load
From YAML::XS pod:
"YAML::XS only deals with streams of utf8 octets"

Test plan:
We are going to test 1 occurence and QA will confirm others don't
contain typos.
0. Don't apply the patches
1. Create a new itemtype with code=❤️
2. Create a new item using this itemtype (to biblionumber=1 will work)
3. Fill OpacHiddenItems with
 itype: [❤️]
4. Search for "street shuffle" or any terms that will return the biblio
Notice that the item is there (there is an error in logs)
5. Apply the patches
6. Repeat 4 and confirm that the item is now hidden

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-02-16 14:54:50 +01:00
Jonathan Druart 46f7239b08 Bug 27673: Replace YAML with YAML::XS
From tht YAML pod:

"""
This module has been released to CPAN as YAML::Old, and soon YAML.pm will be changed to just be a frontend interface module for all the various Perl YAML implementation modules, including YAML::Old.

If you want robust and fast YAML processing using the normal Dump/Load API, please consider switching to YAML::XS. It is by far the best Perl module for YAML at this time. It requires that you have a C compiler, since it is written in C.
"""

See also
https://gitlab.com/koha-community/qa-test-tools/-/merge_requests/35

Test plan:
Try some place where YAML::XS is not used and confirm that it works
correctly

QA note: This patch removes some uses of YAML that were not useful

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2021-02-16 14:54:50 +01:00
Katrin Fischer ed066caf58 Bug 26015: Terminology: Fix tons more staff clients to staff interfaces
This takes care of more occurences of staff client and changes it to
staff interface, including in code comments.

To test:
- I think in this case careful code review is what we look for.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
2020-08-31 16:10:25 +02:00
Tomas Cohen Arazi df58c3e2c9
Bug 25019: Make the ViewPolicy filter initialization standard
This patch makes the ViewPolicy filter use the 'params' accessor instead
of relying of ->{options} which has no accessor. This will allow
interacting with the filter object be similar through all the filters in
the chain.

To test, we just need to verify no behaviour change takes place:
1. Run:
   $ kshell
  k$ prove t/db_dependent/Filter_MARC_ViewPolicy.t
=> SUCCESS: Tests pass
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests still pass!
4. Sign off :-D

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2020-04-08 11:45:47 +01:00
Tomas Cohen Arazi 6189f25dfa
Bug 23793: Add an EmbedItems RecordProcessor filter for MARC::Record objects
This patch introduces a RecordProcessor filter for MARC::Record objects
that embeds MARC::Field's for each item that is passed to the processor.

Target usage:

my $biblio = Koha::Biblios->find(
    $biblio_id,
    { prefetch => [ items, metadata ] }
);

my $opachiddenitems_rules;
eval {
    my $yaml = C4::Context->preference('OpacHiddenItems') . "\n\n";
    $opachiddenitems_rules = YAML::Load($yaml);
};

my @items  = grep { !$_->hidden_in_opac({ rules => $opachiddenitems_rules }) @{$biblio->items->as_list};
my $record = $biblio->metadata->record;

my $processor = Koha::RecordProcessor->new(
    {
        filters => ('EmbedItems'),
        options => {
            items        => \@items
        }
    }
);

$processor->process( $record );

Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-10-23 17:33:07 +01:00
Josef Moravec 68eeefa07e
Bug 22721: Remove frameworkcode parameter in GetMarcFromKohaField calls
Test plan:
Run tests, at least:
    t/db_dependent/Biblio.t
    t/db_dependent/Biblio/TransformHtmlToMarc.t
    t/db_dependent/Charset.t
    t/db_dependent/Circulation/GetTopIssues.t
    t/db_dependent/Filter_MARC_ViewPolicy.t
    t/db_dependent/ImportBatch.t
    t/db_dependent/Items.t
    t/db_dependent/Items/AutomaticItemModificationByAge.t
    t/db_dependent/Items/GetItemsForInventory.t
    t/db_dependent/Koha/Filter/EmbedItemsAvailability.t
    t/db_dependent/Serials.t
    t/db_dependent/XISBN.t
    t/db_dependent/FrameworkPlugin.t

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Bouzid Fergani <bouzid.fergani@inlibro.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2019-07-15 11:28:08 +01:00
Julian Maurice 61ca4deee3 Bug 18579: Fix call to append_fields, fix tests
MARC::Record::append_fields takes a list of MARC::Field (not an arrayref)

Use $record->subfield() instead of $record->field()->subfield() to avoid errors
when field doesn't exist

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-05-12 08:43:18 -04:00
Tomas Cohen Arazi 542ac21b94 Bug 18579: Make EmbedItemsAvailability handle control fields correctly
This patch makes the EmbedItemsAvailability filter handle the biblio.biblionumber > control field mapping gracefully.
Right now, it was assuming a regular data field was used, and such is not the case for UNIMARC (001).

To test:
- Apply the tests patch
- Run:
  $ sudo koha-shell kohadev
 k$ cd kohaclone
 k$ prove t/db_dependent/Koha/Filter/EmbedItemsAvailability.t
=> FAIL: Tests fail to run
- Apply this patch
- Run:
 k$ prove t/db_dependent/Koha/Filter/EmbedItemsAvailability.t
=> SUCCESS: Tests pass!
- Sign off :-D

Sponsored-by: ByWater Solutions

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-05-12 08:43:18 -04:00
Marcel de Rooy b038bb1310 Bug 18208: [QA Follow-up] Get count from Koha::Items
Removes the loop by moving the onloan test to the query.

Test plan:
Run the test again.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-04-13 08:51:04 -04:00
Tomas Cohen Arazi b27b77307c Bug 18208: Add RecordProcessor filter to inject not onloan count to MARC records
This patch adds a new filter for MARC records to be used with
Koha::RecordProcessor. It's purpose is to inject the information about
items not-onloan in a fixed subfield, 999$x.

To test:
- Apply the patch
- Run:
  $ prove t/db_dependent/Koha/Filter/EmbedItemsAvailability.t
=> SUCCESS: Tests pass!
- Sign off :-D

Sponsored-by: ByWater Solutions

Followed test plan, test passes OK
Signed-off-by: Marc Véron <veron@veron.ch>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2017-04-13 08:51:04 -04:00
Jacek Ablewicz 7bdc106c98 Bug 16365: Selectively introduce GetMarcStructure() "unsafe" variant for better performance
GetMarcStructure() currently uses Koha::Cache in the "safe" mode
(returning deep copy of the result data structure by default), which
causes numerous performance issues in many Koha scripts. Switching
it to the "unsafe" mode globally (2nd patch from Bug 16140) resolves
those issues, but ensuring that it is regression-free (and that it
will stay that way in the future) is far from easy. This patch
proposes a bit more manageable solution, it introduces
a possibility to use "unsafe" variant selectively (only in those
places in the code where GetMarcStructure() is called repetitively).
That way, amount of the code that needs to be audited for possible
problems gets vastly reduced, without any performance trade-offs.

Test plan:
1) Have a look at the code and audit the parts affected by this patch
for possible regressions

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Amended the POD of GetMarcStructure, removing a TODO.

NOTE: GetAuthorisedValueDesc, as called in C4::XSLT::transformMARCXML4XSLT
and by GetISBDView, GetMarcAuthors in C4::Biblio, may autovivify some hash
entries in tagslib.
Same for Koha/Filter/MARC/ViewPolicy.pm, sub filter.

No reason however to worry; our use of this structure in Koha does not
depend on the existence of intermediate hash keys. (We seem to be safe as
long as $tagslib->{$tag}->{$subfield}->{tab} and/or hidden are not filled.)

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-09-09 10:43:22 +00:00
Mark Tompsett dd8352bb8b Bug 11592: Add should_hide_marc method to filter
Add should_hide_marc to ViewPolicy filter
Add should_hide_marc tests to t/db_dependent/Filter_MARC_ViewPolicy.t

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
2016-09-08 11:54:16 +00:00
Mark Tompsett 4e1289a55d Bug 15870: potential follow up to comment #8
This patch:
- makes the Koha::RecordProcessor code more clear
  by removing the unnecessary newrecord variable.
- revises the filter to be more clear about the
  expectation that operations are done directly on
  the record parameter.

TEST PLAN
---------
prove t/RecordProcessor.t
prove t/db_dependent/Filter_MARC_ViewPolicy.t
run koha qa test tools

Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Works as advertised. NO koha-qa errors
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-03-24 20:15:40 +00:00
Mark Tompsett c9170233c9 Bug 15870: Follow-up of filter and tests
This patch:
- improves perlcritic messages in the filter and tests.
- changes should display logic to should hide logic
  to simplify filter.
- perltidies the scripts
- debugs the issues outstanding on the comprehensive tests
  provided in the second commit.

Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-03-24 20:15:40 +00:00
Mark Tompsett 25ffd8b215 Bug 15870: MARC Filter to exclude fields/subfields lacking visibility
TEST PLAN
---------
0) Apply 15777 and 15871 as required.
1) Apply patch
2) prove -v t/db_dependpent/RecordProcessor_ViewPolicy.t
   -- all should pass.
3) koha qa test tools.
BONUS
4) perlcritic -1 t/db_dependpent/RecordProcessor_ViewPolicy.t
   -- using the koha qa test tools perlcriticrc, it should pass
      though I am excluding checking print return values.
      (see bug 15646)

Signed-off-by: Hector Castro <hector.hecaxmmx@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-03-24 20:15:39 +00:00
Jonathan Druart 017699c345 Bug 16011: $VERSION - Remove the $VERSION init
Mainly a
  perl -p -i -e 's/^.*3.07.00.049.*\n//' **/*.pm
Then some adjustements

Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>

Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
2016-03-24 17:20:28 +00:00
Jonathan Druart 03c89fa02e Bug 15380: Move Koha::Authority to Koha::MetadataRecord::Authority
The existing Koha::Authority does not use Koha::Objects and does not
exclusively deal with DB objects.
It makes much sense to move it to let the space free for a
Koha::Authority and Koha::Authorities modules based on Koha::Object.

Signed-off-by: Marc Véron <veron@veron.ch>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Brendan Gallagher brendan@bywatersolutions.com
2015-12-31 18:50:41 +00:00
Jacek Ablewicz f3a8b7a0e1 Bug 14456: EmbedSeeFromHeadings record filter shouldn't process MARC holding fields
If the system preference IncludeSeeFromInSearches is enabled, records
exported for zebra indexing are being additionally processed by
EmbedSeeFromHeadings record filter (right now used only in rebuild_zebra.pl
script). This filter embeds 'see from' fields (extracted from authority
records linked with the given biblio via $9 subfields) into target MARC
record, which is then subsequently indexed in zebra.

Currently all fields containing $9 are getting the same exact treatment
by this filter. But on the export stage when the filter is applied, MARC
record being processed already does have holdings data fields added in
the previous stage (usually 952 / 995, depending on the MARC format).
Problem is that holdings data fields use to have $9 subfields in them
as well (mapped to item.itemnumber by default). As a consequence, some
(great many in the typical setup) records exported for zebra indexing
may have surplus "see from" fields added erroneously in semi-random
fashion, so biblio searches would often return some completely
unexpected additional results.

EmbedSeeFromHeadings record filter should not process holdings fields
when dealing with MARC records intended for zebra indexing.

To reproduce:

1) database with as many sample or real-world biblio, item and authority
records as possible is recommended for testing purposes
2) enable IncludeSeeFromInSearches
3) export a bunch of biblio records for zebra (e.g.:
misc/migration_tools/rebuild_zebra.pl -I -b -x -k -length=1000),
inspect the result xml records in /tmp/<whatever> file; observe that at
the end of many records, here and there some extra "see from" (= 1st
indicator: 'z') fields tend to appear, which shouldn't be there ;)

To test:

4) apply patch
5) redo 3)
6) compare results from 3) and 5) with diff

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
I introduced a regression test for this. You should run the tests
without/with the patch and verify that the patch actually fixes the problem.
Good job Jacek! I'm sure writing the regression test would take less time
than such a detailed commit message!

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-07-24 13:39:53 -03:00
Jonathan Druart a6c9bd0eb5 Bug 9978: Replace license header with the correct license (GPLv3+)
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>

http://bugs.koha-community.org/show_bug.cgi?id=9987

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
2015-04-20 09:59:38 -03:00
Katrin Fischer c5987b423b Bug 9620: IncludeSeeFromInSearches and data in $9 subfields can break indexing
In Germany 020$9 is used to store the ISBN with hyphens. In rare cases this
can break indexing in combination with IncludeSeeFromInSearches.

To reproduce:
1) Change your frameworks:
  * Make sure 084 $a is visible
  * Create a 084 $9 that you link to PERSO_NAME authorities.
2) Create a record or edit an existing one:
  * Use the plugin on 084 to link the record to an authority.
  * Make sure 400 is filled in your authority record.

Preparations all done, now test indexing:

3) Make sure IncludeSeeFromInSearches is OFF
  * Reindexing should work without any problems.

4) Activate IncludeSeeFromInSearches
  * Reindexing should throw an error and stop:
    201.Subfields are only for data fields (generally,
    just tags >= 010 at /home/koha/Koha/Filter/MARC/EmbedSee
    FromHeadings.pm line 94

5) Apply patch and reindex.
  * No error should occur.
  * Test 4xx fields from other tags were properly indexed.

What causes the problem is the fact, that Koha tries to store
the date from the 400 in the bibliographic record for indexing.
For this it calculates a tag in the bibliographic record from the
tag it found the authority link in and the 4xx tags used in the
authority record. If you have an authority linked to a 0xx field
and the matching authority has a 4xx where xx < 10, the calculated
tag will be a control field. The attempt to add subfields to this
control field, is causing the error and breaks the indexing process.

Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com>

Comment: Work as described, and is a long description :)
Test:
1) Enabled 084 (all subfields visible), added 084_9 (missing from
default framework) linked to PERSO_NAME
2) Edited authority record and added some data to 400
3) Edited bib record, linked 084 to previous auth record
4) With IncludeSeeFromInSearches OFF, re-index without problem
5) With IncludeSeeFromInSearches ON, re-index problems!!!
====================
exporting biblio
====================
1...Subfields are only for data fields (generally, just tags >= 010) at...
Abort
6) Applied patch
7) With IncludeSeeFromInSearches ON, re-index without problem
...
====================
5001....................................................................................................
Records exported: 5060
====================
...
====================
2001....................................................................................................
Records exported: 2024
====================
CLEANING
====================
Good!

*) That's "works as described", don't know if other tests are needed.

No errors from koha-qa

Signed-off-by: Elliott Davis <elliott@bywatersolions.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
2013-03-07 09:31:11 -05:00
Jared Camins-Esakov 340ba63758 Bug 7417 follow-up: fix a perlcritic violation
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
2012-09-14 17:22:11 +02:00
Jared Camins-Esakov bc05b5d163 Bug 7417: Include see from references in bibliographic searches
This patch adds the Koha::Indexer::RecordNormalizer and
Koha::Indexer::MARC::RecordNormalizer::EmbedSeeFromHeadings packages
to enable the inclusion of alternate forms of headings in bibliographic
searches. When the new syspref IncludeSeeFromInSearches is turned on
(default is off) rebuild_zebra.pl will insert see from headings from
authority records into bibliographic records when indexing, so that a
search on an obsolete term will turn up relevant records.

To test:
1) Enable IncludeSeeFromInSearches
2) Add a heading that has an alternate form to a record (for example,
   "Cooking" has the alternate form "Cookery," if you have authority
   records from LC)
3) Index the zebraqueue (or reindex if you haven't indexed your system
   yet)
4) Confirm that if you search for "Cookery" you get the record you
   just modified

Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 5 August 2012
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Rebased on master 11 September 2012

Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>

Also checked:
- Verified database update works correctly
- Checked system preference and its description
- Checked staff/opac detail pages with feature on/off
- Checked staff/opac search facets
- Downloaded and tested records in various formats
- Tried different searches for 'see from' entries of authorities
- Ran all unit tests

No problems found.
2012-09-13 14:19:28 +02:00