When running the script bulkmarcimport.pl, the number of MARC records reported at the end of the execution does not match the actual number of records processed from the input file.
This patch fixes 3 issues
- broken (invalid) records from batch get now counted (record_number)
- when using the number param (n) to limit the number of records to be processed valid and invalid records get counted
- when using th offset param (o) the scipt errored when broken records were part of the offset.
To test:
1. Download the MARC file with the broken record
2. Run the following import commands and compare the counted import numbers:
a) all records
perl misc/migration_tools/bulkmarcimport.pl -b -v -m MARCXML --file 2_records_1st_broken.xml
1 MARC records done in 0.0421221256256104 seconds
b) limit to 1 record in total
perl misc/migration_tools/bulkmarcimport.pl -b -v -n 1 -m MARCXML --file 2_records_1st_broken.xml
1 MARC records done in 0.0420489311218262 seconds
c) limit to 1 record offset 1 record
perl misc/migration_tools/bulkmarcimport.pl -b -v -n 1 -o 1 -m MARCXML --file 2_records_1st_broken.xml
:2: parser error : Opening and ending tag mismatch: leader2 line 2 and leader
<leader2>00216nam a22001097a 4500</leader>
3. Apply the patch
4. Repeat steps in 2 and compare the results
a) 2 MARC records done in 0.0405838489532471 seconds
b) 1 MARC records done in 0.00948596000671387 seconds *
c) 1 MARC records done in 0.0294840335845947 seconds
*though this one was skipped
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
When running the script bulkmarcimport.pl, the number of MARC records reported at the end of the execution does not match the actual number of records processed from the input file.
To test:
1. Download the joined MARC file containing 5 records.
2. Run the import command:
./misc/migration_tools/bulkmarcimport.pl -b --file=marc_file.mrc
3. Check the final script report
==> 11 MARC records done in 0.566349983215332 seconds
Expected output should show 5 records
4. Apply the patch
5. Repeat step 2, 3
==> The correct number is shown
Signed-off-by: Janusz Kaczmarek <januszop@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Remove start/end cronlogaction calls and unused C4::Log imports
from scripts that use Koha::Script, as these are now handled
automatically by the INIT/END blocks.
Scripts with mid-execution informational logging (plugins_nightly,
harvest_oai, fines, process_message_queue) retain their C4::Log
import and non-start/end cronlogaction calls.
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Correct genuine spelling errors identified by the codespell 2.4.x
full scan after the KTD container OS upgrade. Changes are in
comments, POD, strings, and template text across 55 files.
"SAN Ouest Provence" (the French library co-operative and Koha
sponsor) is preserved as a proper noun. Perl copyright lines
receive an inline # codespell:ignore annotation; about.tt files
(contributor attribution pages) are added to the Koha::Devel::Files
codespell exception list so that future uses of "Provence" as a
typo for "Province" are still caught in all other files.
Notable user-visible fixes:
- "submiting" -> "submitting" in authority and bibliographic merge screens
- "deleteable" -> "deletable" in patron exception messages
- "Acquistions" -> "Acquisitions" in reports dictionary
Test plan:
1. Apply patches
2. In the staff interface, go to Cataloguing > Merge records and
confirm the submit button reads "Submitting" (not "Submiting")
3. In the staff interface, go to Authorities > Merge and confirm
the same correction
4. Attempt to delete a patron with outstanding issues and confirm
the error message reads "deletable" (not "deleteable")
5. prove xt/author/codespell.t
=> All tests pass
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This looks trivial at first, but it's not, because of forking and the
comment in C4::Context
I don't think the comment makes sense, we need to retrieve a new dbh,
and so passing the 'new' parameters to C4::Context->dbh should do the
same.
Test plan:
Try rebuild_zebra.pl and confirm there is no error
% perl misc/migration_tools/rebuild_zebra.pl --processes 2 -b
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
When using bulkmarcimport with the -d switch should not only delete
biblio, biblioitems and items, but also biblio_metadata.
I haven't checked if a FK takes care of the data, but if it does,
we should still also reset the AUTO_INCREMENT:...
Test plan:
==========
1. Check (and notice) the number of records in the biblio (bCount) and
biblio_metadata (bmCount) tables. Check the AUTO_INCREMENT value
for these tables (bAI, bmAI).
2. Import bibliographic records with delete, turn off foreign key checks
and non-existent filename:
misc/migration_tools/bulkmarcimport.pl -b -d -fk -file=non-exist-file
==> "Could not open..." is OK, file doesn't exist.
3. Repeat p. 1 and compare with previously noticed:
bCount => 0 (OK)
bmCount => unchanged (NOT OK)
bAI => 1 (OK)
bmAI => unchanged (NOT OK)
4. Apply the patch.
5. Repeat p. 2-3:
bCount => 0 (OK)
bmCount => 0 (changed, OK)
bAI => 1 (OK)
bmAI => 1 (changed, OK)
Sponsored-by: Ignatianum University in Cracow
Signed-off-by: Janusz Kaczmarek <januszop@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
We sent a SIGKILL to the daemon which does not let it cleanup
Updating to SIGTERM (15) and handle the dbh disconnect gracefully in the
daemon.
Also handle SIGINT (ctrl+c) only because I think it can be useful (?)
Test plan:
prove t/db_dependent/www/search_utf8.t
=> Without this patch there are 3 warnings in the DB container log
db-1 | 2025-10-03 12:30:57 362 [Warning] Aborted connection 42 to db: 'koha_kohadev' user: 'koha_kohadev' host: '172.19.0.6' (Got an error reading communication packets)
=> With this patch applied there are none.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Generated using
`perl misc/devel/tidy.pl --perl`
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
After bug 35659 that added the OAI harvester to Koha the -d switch
for deleting authorities before importing with bulkmarcimport.pl
was broken and resulted in a FK error.
Switching from truncate to DELETE FROM like we already did for items,
biblio etc. fixes the issue.
To test:
- Make sure you have some authorities in your database
- Export some into a file for importing
- Run bulkmarcimport -a -d --file <filename>
- Verify you get the a DB FK constraint error.
- Apply patch
- Repeat steps, verify the script now works as expected again
* Existing authorities are deleted
* File is imported
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jan Kissig <bibliothek@th-wildau.de>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
If using ES, the script will collect the newly created records and biblionumbers in atwo arrays (@search_engine_record, @search_engine_record_ids) and pass that to the indexer when $commitnum records are read. The arrays are cleaned afterwards.
BUT: The records and biblionumbers were always stored, even if we're not using Elasticsearch. And as the cleaning of those arrays only happens if we are in fact using ES, all the data is added but never cleaned.
This patch fixes this by only storing record and biblionumber if we have an $indexer (i.e. if we're using Elasticsearch).
I have no idea though how/if the records are indexed when using Zebra.
After applying this patch and importing a large enough file without using Elasticsearch, the RAM usage also stays more or less constant.
@Jan, please verify!
Signed-off-by: Jan Kissig <bibliothek@th-wildau.de>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
So, I reviewed the code, which has a bunch of issues:
* there's a call to `next RECORD`, but no `RECORD` loop label.
* $record_number is incremented twice
* and the calls to txn_begin is moved from the outside the loop into each record, starting one transaction per record. Not sure how mysql handles this, but postgres would not like it.
This patch fixes those issues:
After I fixed the first two issues, I ran the patched script on a file containing 75 authorities (from GND), and not one was added to the DB, presuambly because nested transactions don't work in mysql?
So I also moved the BEGIN transaction outside of the main loop and started a new one after each commit, and added a final commit after to loop.
Signed-off-by: Jan Kissig <bibliothek@th-wildau.de>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Include checking for invalid marc data in main loop to avoid
loading all marc data into memory at once.
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
The first part of the Zebra rebuild is the exporting. This part is
made faster. The second part with zebraidx is not changed.
A new commandline parameter -forks is added to the rebuild_zebra.pl
script. A subroutine export_marc_records is added between index_records
and export_marc_records_from_sth. The last routine has a new parameter,
the sequence number of the export file.
NOTE: This report does not touch koha-rebuild-zebra yet! This will be
done on a follow-up.
Test plan:
Note that the number of forks/records below can be adjusted
according to your server and database setup.
[1] Reindex a subset of 100 records without forks:
su [YOUR_KOHA_USER]
misc/migration_tools/rebuild_zebra.pl -a -b -r -d /tmp/rebuild01 -k --length 100
Check if /tmp/rebuild01/biblio contains one export file for auth/bib.
Verify that max. 100 auth and bib were indexed (check Auth search, Cataloguing)
[2] Reindex an additional subset of 100 recs with forks (remove -r, add -forks):
su [YOUR_KOHA_USER]
misc/migration_tools/rebuild_zebra.pl -a -b -d /tmp/rebuild02 -k --length 100 --offset 100 -forks 3
Check if /tmp/rebuild02/biblio contains 3 export files for auth/bib.
Verify that max. 200 auth and bib were indexed (check Auth search, Cataloguing)
[3] Run a full reindex with forks:
su [YOUR_KOHA_USER]
misc/migration_tools/rebuild_zebra.pl -a -b -d /tmp/rebuild03 -k -forks 3
Check both searches again.
[4] Bonus: To get a feeling of improved speed, reindex a larger production db with and
without using -forks. (Use something like above.) You may add -I to skip indexing
in order to better compare both exports.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Reindexed a prod db in 96 mins instead of 150 mins (3 forks, 4 cores). Main gain in
biblio export; complete export took 35 mins, zebraidx 61 mins.
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This commit is generated using:
% perl misc/devel/tidy.pl
*within* ktd, to get the same version of perltidy than what will be used
by our CI (currently v20230309).
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Testing plan:
1. set CronjobLog to "Log"
2. Run the three cron jobs
`perl misc/link_bibs_to_authorities.pl; perl misc/cronjobs/merge_authorities.pl -b; perl misc/migration_tools/remove_unused_authorities.pl --confirm`
3. Visit the log viewer /cgi-bin/koha/tools/viewlog.pl, select the "Cron jobs" module, & note it's empty
4. apply the patch
5. Repeat steps 2 & 3 but note the cron logs aren't empty
squash
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
The system is vulnerable to Formula Injection attacks as the data
stored within the database and exported as CSV/Excel is not being
sanitized or validated against implanted formula payloads
This patch modifies all uses of Text::CSV and derived classes to pass
the "formula" parameter with value of "empty" which replaces formulas
by empty string.
Test Plan:
1) Apply this patch
2) For guided_reports.pl, attempt to export CSV where you've set a column to a formula somehow
( such as "=1+3" )
3) Export that CSV file
4) Note the formula has not been exported
5) Repeat this plan for the remaining scripts that export CSV files
where users can define the outputted data
Signed-off-by: Magnus Enger <magnus@libriotech.no>
Fixed two conflicts. I have tested that this works as advertised on:
- Reports (Download > Comma separated text (.csv)) [Text::CSV::Encoded]
- Circulation > Overdues > Download file of all overdues [Text::CSV_XS]
- misc/export_borrowers.pl [Text::CSV]
This covers all modules used, and both GUI and command line.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Change none to empty in the commit message ! None is the default,
doing nothing. Empty clears the formulas.
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Option name changed, POD updated.
The reason not to do this by default is that it is an extra record conversion
to check the validity, so may slow down the import.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds a conversion from MARC -> XML -> MARC to catch any parsing errors.
If errors are found, we then lint the record to catch any problems, output the
warnings, and skip the record.
To test:
1 - Download the sample records from this bug report
2 - perl misc/migration_tools/bulkmarcimport.pl -b --file=520_nosubfield.mrc -v
fails!
3 - perl misc/migration_tools/bulkmarcimport.pl -b --file=003_subfielda.mrc -v
fails!
4 - Apply patch
5 - Repeat 2 & 3 - no change
6 - Add -st switch to the commands:
perl misc/migration_tools/bulkmarcimport.pl -b --file=520_nosubfield.mrc -v -st
perl misc/migration_tools/bulkmarcimport.pl -b --file=003_subfielda.mrc -v -st
7 - The records are now skipped, and the script completes
8 - Confirm the warnings generated are useful
9 - Sign off!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
To test:
1 - Grab the sample file on this bug report
2 - perl misc/migration_tools/bulkmarcimport.pl -b --file=spaceditems.pl -v
3 - An exception for 'Duplicate ID' is thrown and script dies
4 - Apply patch
5 - run script again
6 - The script finishes, no items are added, but record is
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
We can now call metadata_record directly on the Koha::Biblio object.
This aptch updates all modules and controllers to use
Koha::Biblio->metadata_record directly where appropriate. The
exceptions are where we don't require any filtering or the filtering
makes sense to do after the initial fetch.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Test plan:
1) Create an authority with a unique value in a searchable field, for instance a PERSO_NAME authority with 'BZ18273' in the 100$a field.
2) Create a second authority with the same value in the same field.
3) Export this second authority as iso2709 (Save -> MARC (Unicode/UTF-8))
4) Run bulkmarcimport.pl with the following parameters:
perl bulkmarcimport.pl -a --file=/kohadevbox/koha/auth-xxxx.utf8 --match='Heading,100$a' -l /kohadevbox/koha/bulk.log
5) Check in the log file and/or through search that a third authority has been added (despite using the --match option)
6) Apply the patch
7) Delete the 3 authorities and repeat step 1 to 4.
8) Check that a third authority has not been added (--match worked)
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Add documentation for the new -si, --skip_indexing parameter
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Before the previous patch we were indexing with every AddAuthority/ModAuthority call and I assumed
we were also indexing during the commity, however, it appears we were not. This patch ensures we push
the record and ids into the arrays for indexing during commit.
Additionally I add a skip to next record on error, to match biblio behaviour.
I also correct a log referring to biblios during authority importing
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds a new optoin to skip indexing to bulkmarcimport.
It also fixes a bug where authorities were being indexed multiple times
during import.
To test:
1 - Apply patch
2 - Download the sample files on this bug
3 - Perform asearch engine search that should retrieve all records( 'a' in Zebra, '*' in ES)
4 - Note the total (435 in KTD)
5 - perl misc/migration_tools/bulkmarcimport.pl -b -v --file=bug_35466_b.mrc
6 - Search again, note increaed by 100
7 - perl misc/migration_tools/bulkmarcimport.pl -b -v --file=bug_35466_b.mrc --skip_indexing
8 - Search again, no increase
9 - perl misc/search_tools/rebuild_elasticsearch.pl -v
10 - Search again, increase, records were added but not initially indexed
11 - Browse to authorities and search as for biblios
12 - perl misc/migration_tools/bulkmarcimport.pl -a -v --file=bug_35466_a.mrc
13 - Search again, note increase
14 - perl misc/migration_tools/bulkmarcimport.pl -a -v --file=bug_35466_a.mrc --skip_indexing
15 - Search again, no increase
16 - perl misc/search_tools/rebuild_elasticsearch.pl -v
17 - Search again, increase, records were added but not initially indexed
18 - Sign off! Hi5!
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
1 - perl misc/migration_tools/bulkmarcimport.pl -b --file=notafile.mrc -v
2 - Output:
Characteristic MARC flavour: MARC21
0 MARC records done in 0.00804400444030762 seconds
3 - Apply patch
4 - perl misc/migration_tools/bulkmarcimport.pl -b --file=notafile.mrc -v
5 - Output:
Characteristic MARC flavour: MARC21
Could not open notafile.mrc: No such file or directory at misc/migration_tools/bulkmarcimport.pl line 262.
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Script migration_tools/build_oai_sets.pl is missing ORDER BY
biblionumber.
This is a problem when using OFFSET and LIMIT, the results may be
differenly orderder between two calls of this script.
Test plan :
1) Create a OAI SET with all records
2) Run migration_tools/build_oai_sets.pl with the first half of records
3) Run migration_tools/build_oai_sets.pl with the second half of records
4) Check all records are in the OAI set
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This change consistently sends the Csrf-Token in the request header.
Previously, one POST sent it in the request body, while the other POST
sent it in the request header. Since we're using an API, it's best
for us to always send it in the request header
Test plan:
0. Apply the patch
1. perl ./misc/migration_tools/koha-svc.pl \
http://localhost:8081/cgi-bin/koha/svc koha koha 29 > bib-29.xml
2. perl ./misc/migration_tools/koha-svc.pl \
http://localhost:8081/cgi-bin/koha/svc koha koha 29 bib-29.xml
3. Note that the following appears in STDOUT and there is no 403 error:
"update 29 from bib-29.xml"
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This change fixes the Koha::SVC to store the CSRF token for
the authenticated session for further POSTing.
Test plan:
0. Apply the patch
1. perl ./misc/migration_tools/koha-svc.pl \
http://localhost:8081/cgi-bin/koha/svc koha koha 29 > bib-29.xml
2. perl ./misc/migration_tools/koha-svc.pl \
http://localhost:8081/cgi-bin/koha/svc koha koha 29 bib-29.xml
3. Note that the following appears in STDOUT and there is no 403 error:
"update 29 from bib-29.xml"
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Like IncludeSeeFromInSearches adds see from heading in search (4xx),
new system preference IncludeSeeAlsoFromInSearches adds see also from heading (5xx).
Test plan:
Test on both Elasticsearch and Zebra:
1)
1.1) Create an authority record with heading, see from heading 4xx and see also from heading 5xx
1.2) Use this authority in a biblio record
2)
2.1) Set IncludeSeeFromInSearches and IncludeSeeAlsoFromInSearches to "Don't include"
2.3) Rebuild search engine
2.4) Perform a search on heading text => you find the biblio record
2.5) Perform a search on see from heading => you do not find the biblio record
2.6) Perform a search on see also from heading => you do not find the biblio record
3)
3.1) Set IncludeSeeFromInSearches and IncludeSeeAlsoFromInSearches to "Include"
3.2) Rebuild search engine
3.3) Perform a search on heading text => you find the biblio record
3.4) Perform a search on see from heading => you find the biblio record
3.5) Perform a search on see also from heading => you find the biblio record
Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Items are no longer embedded in the MARCXML and because of this the MARC data
does not need to be saved once more after changing record items data.
The "defer_marc_save" is no longer needed since bulkmarcimport.pl was the only
place this option was utilized in order to resave MARC data after possibly changing
items data. There is also a bug bulkmarcimport.pl where the record data is re-saved
without stripping items if duplicate items are found and the dedup barcodes
option is enabled that is resolved by this change.
This change enables MARC Overlay rules to be enabled for bulkmarkcimport.pl as using
the defer_marc option would previously effectively bypass any defined rules.
To test
1) Apply patch 29440 (which this depends on), but no not yet apply this
patch
2) Remove comments around "bulkmarcimport: _("bulkmarcimport.pl"),"
in koha-tmpl/intranet-tmpl/prog/en/modules/admin/marc-overlay-rules.tt
line 463
3) Enable MARC overlay rules and add a rule (with source => "*" or
'bulkmarcimport') to protect some field.
4) Import a biblio using the bulkmarcimport script. Overlay rules will
only be applied on updates so a match condition matching the
record in Koha when bulkmarcimport.pl is run the next time for the same
record needs to be supplied (for example -match "control-number,001")
5) Edit the imported biblio and change the value of the protected field
6) Run bulkmarcimport with the same parameters as in 4) and verify the
the field that should have been protected has been overwritten
7) Revert changes for line 463 in marc-overlay-rules.tt (or will result in
conflict applying the patch), apply the patch and repeat steps 4-5,
verify that the field now was protected from being overwritten.
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This variable actually contains ModBibio options and
ModifyBiblioMarc accepts a subset of these
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Jan Kissig
Hammat Wele
Nick Clemens
Martin Renvoize
Jonathan Druart
Marcel de Rooy
Roman Dolny
Katrin Fischer
Tomas Cohen Arazi
Thomas Klausner
David Gustafsson
Koha Development Team
phette23
Kyle M Hall
Martin Renvoize
Matthias Meusburger
Fridolin Somers
David Cook
David Schmidt
David Gustafsson