Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Authority MODIFY action log diffs were emitting bogus entries for every
field with a default ' ' indicator on every save, producing output like:
_marc.fields.1.040.ind1 ' ' ' '
_marc.fields.1.040.ind2 ' ' ' '
_marc.fields.2.150.ind1 ' ' ' '
_marc.fields.3.667.ind1 ' ' ' '
...
even when only a single subfield was edited.
Root cause: Struct::Diff treats a utf8-flagged ' ' as different from a
non-utf8 ' ' SV, even though Perl's eq returns true. ModAuthority was
sourcing the two sides of the diff from different parse paths:
- $oldrecord = GetAuthority($authid)
-> MARC::Record->new_from_xml(...) -> indicators arrive utf8-flagged
- $record (the "after" side) was the in-memory MARC built by the
editor from CGI parameters -> indicators are bare byte strings
The XML round-trip through AddAuthority normalises the stored copy, but
the in-memory $record we passed to logaction had not been re-parsed, so
every default indicator looked like a change to Struct::Diff.
Fix: after AddAuthority commits the new state, re-read it via
$new_authority->record and pass that to _authority_log_data. Both sides
of the diff now traverse the same new_from_xml path, utf8 flags match,
and Struct::Diff falls silent on unchanged indicators.
This is the symmetric counterpart of how biblios already work in
C4::Biblio: ModBiblio sources both $original_marc and $updated_marc_log
through Koha::Biblio::Metadata::record_strip_nonxml (which also calls
new_from_xml), so biblios don't hit this.
The deeper asymmetry between in-memory editor MARC and stored MARC
(default indicators only get normalised on the XML round-trip) is left
for a separate bug; this patch keeps the action log diff clean without
touching the storage path.
Test plan:
1. prove t/db_dependent/AuthoritiesMarc.t
2. Enable AuthoritiesLog. Edit any authority via the staff UI making a
single subfield change. Open Tools -> Log viewer for that authority
and verify the diff column lists only the changed subfield (plus the
005 timestamp), with no rows for ind1/ind2 of untouched fields.
3. Re-run the same edit a second time changing nothing else; the diff
should again be limited to 005 + the touched subfield.
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Mirrors bug 42188's biblio info-column convention. Generalise the
CATALOGUING-only info-column rewrite branch in C4::Log::logaction to
cover AUTHORITIES as well, using a module-aware prefix map so each
module gets its own prefix ("biblio" / "authority"). For AUTHORITIES
this means MODIFY now stores "authority {pre-change JSON}" and DELETE
stores "authority {final-state JSON}", while ADD stays bare
"authority" (the call site passes the string directly and ADD is
guarded anyway). The biblio behaviour is preserved bit-for-bit.
Test plan:
1. prove t/db_dependent/AuthoritiesMarc.t
2. Enable AuthoritiesLog. Create / modify / delete an authority via
the staff UI, then check action_logs:
- ADD row's info column reads just "authority"
- MODIFY row's info column reads "authority { ... pre-change JSON ... }"
- DELETE row's info column reads "authority { ... final-state JSON ... }"
3. prove t/db_dependent/Koha/Biblio.t
4. prove t/db_dependent/Log.t
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Mirrors the bug 42188 main commit for biblios. Replace the custom
_marc_record_to_diffable helper with MARC::Record::MiJ's to_mij_structure
so AUTHORITIES action log diffs use the same MARC-in-JSON shape as
CATALOGUING diffs. Add Koha::Authority::_unblessed_for_log to expose
only the auth_header columns that exist independently of the MARC
record (authtypecode, origincode, linkid), avoiding duplication of
MARC-derived data (heading, authtrees), raw marcxml, and system
timestamps. The explicit whitelist also removes the need for the
post-hoc empty-field stripping added in a previous follow-up.
Test plan:
1. prove t/db_dependent/AuthoritiesMarc.t
2. Enable AuthoritiesLog. Create, edit, and delete an authority via
the staff UI, then inspect action_logs.diff in the DB:
- _marc contains MiJ shape { leader: "...", fields: [ { tag: {...} }, ... ] }
- Changes to heading fields produce diffs on
_marc.fields[N].TAG.subfields[0].a
- The diff no longer carries a redundant top-level heading entry
(it's MARC-derived and now only appears in _marc).
3. prove t/db_dependent/Log.t
4. prove t/db_dependent/Koha/Authority.t
5. prove t/db_dependent/Koha/Biblio.t
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The AUTHORITIES action log entries for authority operations previously
stored minimal data: ADD and DELETE logged just the string "authority",
while MODIFY logged a raw MARC-formatted dump of the pre-change record
as a plain text string ("authority BEFORE=>..."). None of these populated
the diff column.
This patch updates AddAuthority, ModAuthority and DelAuthority in
C4/AuthoritiesMarc.pm to follow the same pattern introduced for biblios
in bug 40135: passing combined hashrefs to logaction so that the diff
column is automatically populated with Struct::Diff JSON.
The combined hashref contains the auth_header table columns (authtypecode,
heading, origincode, etc. — excluding the raw marcxml blob) plus a _marc
key built by the new _marc_record_to_diffable helper, which converts a
MARC::Record into a tag => array-of-formatted-strings structure readable
by the existing diff viewer.
A new _authority_log_data helper packages the authority unblessed data
and MARC record into the combined hashref, keeping the three call sites
concise.
For ADD, the logaction is now emitted after the authority record has been
fully stored (after _after_authority_action_hooks), so the diff captures
the complete saved state. For MODIFY, the before-state is captured from
Koha::Authorities before calling AddAuthority, and the after-state is
read back once AddAuthority has committed the new data. For DELETE, the
MARC is captured before the authority is removed.
The diff column for MODIFY will now show both authority table column
changes (e.g. heading) and MARC field-level changes (e.g. _marc.151.0)
in the existing diff viewer table in the log viewer.
Tests are added to verify that ADD, MODIFY, and DELETE all populate the
diff column and that _marc content is captured correctly.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The info column for CATALOGUING action logs is, by convention, the
record state before modification -- MODIFY writes the pre-change JSON
payload, DELETE writes the state at time of deletion, but ADD has no
pre-change state. Writing the new record state into info on ADD
duplicates the state that the subsequent MODIFY (if any) captures as
its "before" snapshot, and conflicts with that convention.
Skip the JSON payload on ADD so info degrades to the bare "biblio"
prefix. The diff column already captures the full creation (every
field "added" against an empty hashref), so no information is lost.
Test plan:
1. prove t/db_dependent/Koha/Biblio.t
2. Observe that the "CataloguingLog MARC-in-JSON diff tests" subtest
now also asserts:
- ADD info == "biblio"
- MODIFY info matches /^biblio \{/
- DELETE info matches /^biblio \{/
3. Create / modify / delete a record via the staff UI with
CataloguingLog on, then check action_logs: the ADD row's info
column should read just "biblio".
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Replace Dumper with to_json for the biblio info column in logaction,
fixing UTF-8 encoding and extending coverage to ADD and DELETE actions
which previously logged only the literal string "biblio".
All three actions (ADD, MODIFY, DELETE) now log "biblio " followed by
a pretty-printed JSON representation of the relevant state: pre-change
for MODIFY, new state for ADD, and deleted state for DELETE.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Replace the custom _marc_record_to_diffable helper with MARC::Record::MiJ's
to_mij_structure method, which produces standard MARC-in-JSON. This fixes
three problems with the old approach:
- Dollar signs used as subfield delimiters in the formatted string were
ambiguous when subfield content itself contained " $x " sequences.
- Grouping fields by tag into a hash lost inter-field ordering, making
lossless reconstruction of the MARC record impossible.
- Non-ASCII characters were subject to Perl string context encoding
issues; JSON serialisation of the MiJ structure handles UTF-8 cleanly.
_unblessed_for_log in Koha::Biblio is updated to return only the three
biblio columns that are genuinely independent of the MARC record
(frameworkcode, opac_suppressed, serial). The remaining columns
(author, title, subtitle, notes, etc.) are denormalised copies of MARC
data already captured in the _marc key, so including them produced
redundant and noisy diffs.
The fmt() helper in viewlog.js is updated to render JavaScript arrays
as ordered lists rather than objects with numeric keys, improving the
display of MARC subfield arrays in the diff table.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The "Cite" option in the OPAC is missing the subtitle (245) in MARC21.
To recreate:
1. Add a record or find a record with a subtitle (in KTD, biblionumber 58 has a 245 http://localhost:8081/cgi-bin/koha/catalogue/MARCdetail.pl?biblionumber=58
2. In the OPAC, go to the record
http://localhost:8080/cgi-bin/koha/opac-detail.pl?biblionumber=58
3. Click "Cite"
--> The subtitle is not in any of the citations
4. Apply the patch
5. Repeat step 2 and 3
--> The subtitle is now shown
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Test plan:
Read the patch.
Search for both parameters to verify that they are not used.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
UNIMARC instances include the following two authority types
by default (among others): Corporate Body Name (code: 'CO')
and Collective Title (code: 'CO_UNI_TI').
The problem is that when searching for Corporate Body Name
authorities (in the OPAC or in the Staff interface), the
results will include Collective Title authorities as well.
It has to do with Corporate Body Name authorities using
an authtypecode (CO) that happens to be the prefix of the
authtypecode (CO_UNI_TI) used by Collective Titles.
NOTE: Elasticsearch is not affected by this.
The fix is to additionally pass the completeness attribute
to the RPN query sent to Zebra so that it will try to match
on what is literally found in the entire field's phrase
index (i.e. the value of UNIMARC 152$b), instead of performing
a partial match.
More information can be found at the official documentation:
https://software.indexdata.com/zebra/doc/querymodel-rpn.html#querymodel-bib1-completeness
Test plan (tailored for KTD):
0) Launch a UNIMARC Koha instance using Zebra:
ktd --marcflavour unimarc --search-engine zebra up
1) Import the Sample MARC record attached to this Bug,
containing a single Collective Title authority
(adding one manually would take a long time due to
the many mandatory subfields).
2) Perform a generic authority search (without entering
anything in the search box) for all 'Corporate Body
Name' authorities. Note the number of results returned
(83), and also notice that in page 4 of the results you
get a stray 'Collective Title' authority ("Selected
works", the one you imported in Step 1).
3) Apply this patch (and `restart_all`).
4) Repeat the same search as in Step 2. This time, the
number of results returned should be one less than
before (82), and you should only see 'Corporate Body
Name' authorities in the results.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch updates the diff for ADD/CREATE and DELETE logactions to
exclude undefined fields from the diff.
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Update suggestion action logging to populate the diff column using the
correct before/after semantics:
- CREATE: empty before-state ({}), new suggestion data as after-state
- MODIFY: pre-change unblessed data as before, post-store object as after
- DELETE: log before the SQL DELETE so the object can still be fetched;
pass the suggestion as both $infos (info column) and $original (diff
before-state), logaction sets after-state to {}
Also add tests verifying the diff column is populated for all three actions.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch extends the SIP checkout handler to honour the new
three-valued AllowItemsOnHoldCheckoutSIP preference:
* 0 - Don't allow checkout of any held item
* 1 - Allow checkout when the hold is still pending (not attached)
* 2 - Also allow checkout when the hold is waiting, in transit or
being processed for another patron
Previously the preference was a simple YesNo toggle that, when
enabled, allowed the checkout regardless of whether the hold was
pending or already attached to an item on the hold shelf. Libraries
could not block the more disruptive case (checking out a waiting
hold destined for another patron) without also blocking pending
holds.
The existing post-confirmation filter is reworked into a single
branch per confirmation type that consults the preference value.
Test plan:
1. prove t/db_dependent/SIP/Transaction.t
2. With AllowItemsOnHoldCheckoutSIP=0, place a pending hold for
patron A on an item, attempt SIP checkout to patron B: refused.
3. Repeat with the hold marked waiting: refused.
4. Set AllowItemsOnHoldCheckoutSIP=1, pending hold: allowed;
waiting hold: refused.
5. Set AllowItemsOnHoldCheckoutSIP=2, waiting hold: allowed.
Signed-off-by: Kristi Krueger <kkrueger@cuyahogalibrary.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
With the ability to customise which columns are hidden by default as
part of "Table settings", this syspref is redundant. As part of the
atomicupdate, we incorporate the previous syspref value into
columns_settings.
As there are six possible states the migration could start from (the
syspref can be 0 or 1, and the table setting can be 0, 1, or NULL) and
two final states it must be idempotent from, the test plan is
unfortunately quite cumbersome.
Test plan:
1. Reset the database, to clear the table settings.
2. Apply patch and update the database.
3. Navigate to Administration > System preferences > Cataloging.
4. Verify that "SpineLabelShowPrintOnBibDetails" does not appear.
5. Navigate to a catalogue detail page.
6. Verify that the "Spine label" column is hidden.
7. Update the database again.
8. Navigate to a catalogue detail page.
9. Verify that the "Spine label" column is hidden.
10. Revert the patch.
11. Reset the database, to clear the table settings.
12. Navigate to Administration > System preferences > Cataloging.
13. Set "SpineLabelShowPrintOnBibDetails" to "Show" and save.
14. Apply patch and update the database.
15. Navigate to a catalogue detail page.
16. Verify that the "Spine label" column is visible.
17. Update the database again.
18. Navigate to a catalogue detail page.
19. Verify that the "Spine label" column is visible.
20. Revert the patch.
21. Reset the database, to clear the table settings.
22. Navigate to Administration > Table settings.
23. Under Catalog > holdings_table, check "Is hidden by default" for
"spinelabel" and save.
24. Apply patch and update the database.
25. Navigate to a catalogue detail page.
26. Verify that the "Spine label" column is hidden.
27. Revert the patch.
28. Reset the database, to clear the table settings.
29. Navigate to Administration > System preferences > Cataloging.
30. Set "SpineLabelShowPrintOnBibDetails" to "Show" and save.
31. Navigate to Administration > Table settings.
32. Under Catalog > holdings_table, check "Is hidden by default" for
"spinelabel" and save.
33. Apply patch and update the database.
34. Navigate to a catalogue detail page.
35. Verify that the "Spine label" column is hidden.
36. Revert the patch.
37. Reset the database, to clear the table settings.
38. Navigate to Administration > Table settings.
39. Under Catalog > holdings_table, uncheck "Is hidden by default" for
"spinelabel" and save.
40. Apply patch and update the database.
41. Navigate to a catalogue detail page.
42. Verify that the "Spine label" column is hidden.
43. Revert the patch.
44. Reset the database, to clear the table settings.
45. Navigate to Administration > System preferences > Cataloging.
46. Set "SpineLabelShowPrintOnBibDetails" to "Show" and save.
47. Navigate to Administration > Table settings.
48. Under Catalog > holdings_table, uncheck "Is hidden by default" for
"spinelabel" and save.
49. Apply patch and update the database.
50. Navigate to a catalogue detail page.
51. Verify that the "Spine label" column is visible.
Thanks-to: Katrin Fischer
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Adds minimal POD stubs for all private (underscore-prefixed) subroutines
in C4/Matcher.pm and C4/ImportBatch.pm to satisfy the QA script's
pod_coverage check.
Sponsored-by: Main Library Alliance <https://www.mainlib.org/>
Signed-off-by: Jeanne Mauriello <jeanne.mauriello@mainlib.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Extends C4::Matcher::get_matches() to track each matchpoint's individual
score contribution alongside the aggregate total. A new `composite_scores`
hashref (keyed by search_index) is included in every result entry.
Updates C4::ImportBatch::SetImportRecordMatches() to JSON-encode and
persist the composite scores in the new column, and
GetImportRecordMatches() to decode and return them.
Sponsored-by: Main Library Alliance <https://www.mainlib.org/>
Signed-off-by: Jeanne Mauriello <jeanne.mauriello@mainlib.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This commit improves upon the original fix by making the intent more
explicit when handling empty string date parameters in AddRenewal and
AddIssue functions.
WHY THIS IS BETTER:
The original fix (changing `defined $datedue` to just `$datedue`)
works correctly but relies on Perl's implicit truthy/falsy semantics:
if ( $datedue && ref $datedue ne 'DateTime' )
This is subtle because:
- Empty string "" is falsy, so parsing is skipped ✓
- But it's not immediately obvious that empty strings are being
handled specially
- The code doesn't explicitly show its intent
THIS APPROACH:
if ( defined $datedue && $datedue ne '' && ref $datedue ne 'DateTime' )
Benefits:
1. EXPLICIT: Clearly states "only parse if defined AND not empty
AND not already a DateTime"
2. SELF-DOCUMENTING: Intent is obvious to future developers
3. NO RELIANCE on truthy/falsy semantics
4. CONSISTENT: Same pattern applied across all date parameter handling
COMPREHENSIVE CHANGES:
1. AddRenewal (C4/Circulation.pm:3449)
- Made datedue empty string check explicit
2. AddIssue (C4/Circulation.pm:1611)
- Made datedue empty string check explicit
- Also updated issuedate handling (line 1616) to explicitly handle
empty strings
3. Test Coverage (t/db_dependent/Circulation/issue.t)
- Added test for AddIssue with empty string datedue
- Added test for AddIssue with empty string issuedate
- Ensures both functions properly calculate dates from circulation
rules instead of using "now"
The existing test for AddRenewal already covered empty string datedue
(added in the original Bug 37966 commit). These new tests ensure the
same correct behavior for AddIssue.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
When renewing a book on hold with the RenewalOnHoldOverride sys pref set
to "Allow", if the due date is not specified then it becomes "now". By
checking the input is set rather than just defined the due date is set as
expected based on circulation rules.
Also change the description of the RenewalOnHoldOverride sys pref as the
due date would no longer be required to be entered with this change.
Test plan:
1. make sure RenewalOnHoldOverride is set to 'Allow'
2. check out a book, then place a title-level hold on it
3. renew the book, see the hold override dialog
4. leave the due date empty and hit the override button
5. observe the new due date is "now"
6. apply patch and restart plack
7. renew the book again, overriding the hold, and see the expected due date
as per circulation rules.
Signed-off-by: Marion Durand <marion.durand@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This is not used!
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This looks trivial at first, but it's not, because of forking and the
comment in C4::Context
I don't think the comment makes sense, we need to retrieve a new dbh,
and so passing the 'new' parameters to C4::Context->dbh should do the
same.
Test plan:
Try rebuild_zebra.pl and confirm there is no error
% perl misc/migration_tools/rebuild_zebra.pl --processes 2 -b
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Some services ( such as CloudFlare ) use alternative heads to X-Forwarded-For. Koha should be able to use a custom header instead of X-Forwarded-For if needed.
This patch updates the feature to allow both the custom header and X-Forwarded-For to work harmoniously
For example, CloudFlare has it's own custom header.
This patch allows a site to be configured for Cloudflare's proxy before the proxy is enabled.
Test Plan:
1) prove t/Koha/Middleware/RealIP.t
Signed-off-by: Lin Wei Li <lin-wei.li@inLibro.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The previous POD follow-up added headings for the private helper subs but
introduced several typos and used =head2, which puts them at the same level
as the public API in the generated docs. This patch:
- Fixes typos: matchpint, pionts, march matcher, mathcpoints, etc.
- Clarifies a couple of descriptions.
- Demotes each private helper from =head2 to =head3 so they render as
implementation notes rather than peers of the public methods, matching
the pattern used in Koha::Authority's "Internal methods" block.
Helper subs remain interleaved with the public API in source order to keep
the diff minimal; a wholesale move into a dedicated "Internal functions"
section would be a larger refactor outside the scope of this bug.
Test plan:
1 - perldoc C4/Matcher.pm
2 - Confirm no typos are rendered and the private helpers appear one level
deeper than the public methods.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The previous defensive block still dereferenced $target_auth unconditionally
when attempting to recover via record_strip_nonxml. If Koha::Authorities->find
returned undef (stale search index pointing at a deleted authid) the eval
around ->record would catch the first failure, but the recovery call on the
next line would die uncaught.
This patch:
- Skips silently when the authid returned by the search engine no longer
exists in the database, so get_matches cannot die on a stale index.
- Wraps record_strip_nonxml in eval so an exception there is also handled
and the record is simply skipped - matching the stated intent of the
"skip when not possible" path.
Test plan:
1 - Run t/db_dependent/Matcher.t - still passes.
2 - Simulate a stale index: mock search_auth_compat to return an authid
that is not present in auth_header. Confirm get_matches no longer dies
and returns without error.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
When matching authorities we can die on a bad record, we should add protection against this
on top of bug 41843
To test:
1 - Apply index patch and rebuild elastic
perl misc/search_tools/rebuild_elasticsearch.pl -r -v
2 - Make a new record matching rule
threshold: 1000
type: authority
index: authid
score: 1000
tag: 001
3 - Choose an authority record and corrupt it:
sudo koha-mysql kohadev
UPDATE auth_header SET marcxml = UpdateXML(marcxml, '//datafield[@tag="040"]/subfield[@code="c"]',CONCAT('<subfield code',CHAR(27),'="c">OSt</subfield>')) WHERE auth_header.authid=92;
4 - View and save the record as marcxml
http://localhost:8081/cgi-bin/koha/authorities/detail.pl?authid=92
5 - Cataloging - stage marc records - authority/marcxml/no matching
6 - Confirm it stages
7 - Attempt to match using your new rule
8 - Get an error parsing the marc
9 - Apply patch, restart all
10 - Relaod the staged batch
11 - Attempt matching agaion
12 - Success!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The import result hash key was inconsistent: the module sets
$result->{overwrote}, but the test checked $result->{overwrite} and the
template wrapper used import_overwrite. The uninitialized key also
triggered a "Use of uninitialized value" warning when no templates were
overwritten.
- Initialize overwrote => 0 in ImportModificationTemplates result hash
- Update test to check $result->{overwrote}
- Fix template success-alert wrapper to gate on import_overwrote
Test plan:
- prove t/db_dependent/MarcModificationTemplates.t
- all 134 subtests pass
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch adds export and import functions to the MarcModificationTemplates tools
The different templates can be exported either individually or all of them.
On import you can choose to either skip or overwrite existing templates with the same name.
Test plan:
- create a Marc Modification templates with one field creation action.
- click on "Export all" in the menu and save the file on your computer.
- add a new action in the template (ie, update field).
- click on "Import" in the menu and choose the downloaded file.
- by default the import tool doesn't overwrite
- there should be a message saying "skipped 1 template"
- the template should still have 2 actions.
- import the same file again but uncheck the "skip existing template".
- this time the template should be back to only one action.
- delete the template
- import the same file again, the template should be back with a single action.
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Add POD documentation for private subroutines in C4/Letters.pm and
fix the POD heading for _type in Koha/Notice/Message.pm to satisfy
the pod_coverage QA check.
Also fixes a typo in the existing _add_attachments POD heading
(_add_attachements → _add_attachments).
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch applies the new style preferences to the preview display and
email sending code.
Sponsored-by: Open Fifth <https://openfifth.co.uk>
Signed-off-by: Marion Durand <marion.durand@biblibre.com>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The EAN-13 barcode query used max(abs(barcode)) which allowed non-EAN-13
barcodes (e.g. 14-digit KTD sample barcodes) to dominate the result.
The saved 13-digit EAN-13 barcodes were always smaller, so $nextnum
never advanced and the same barcode was generated on every page load.
Fix by extracting EAN-13 logic into C4::Barcodes::ValueBuilder::EAN13
(consistent with other barcode type packages) and scoping the query to
WHERE length(barcode) = 13 so only valid-length barcodes contribute to
the max. Both barcode.pl and Koha::REST::V1::Biblios now call the shared
function, eliminating the duplicated inline code.
Tests: adds EAN-13 test cases to t/db_dependent/Barcodes_ValueBuilder.t
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Reports may consume too much resources of the db. To avoid that a new config param
'report_sql_max_statement_time_seconds' was created, which sets the maximum time a report statement may
take to execute in seconds. If it takes longer it will be aborted.
reflowed the code, so that is is executed after is_sql_valid, and put
the code in a subroutine
Test plan:
0. if you run this test in ktd you should check if you are running
mysql or mariadb, and then be able to switch database
1. create a new report by going to
More > Reports > New SQL Report
(/cgi-bin/koha/reports/guided_reports.pl?op=add_form_sql)
enter the following SQL (which only purpose is to run long)
select ExtractValue(bm1.metadata,'//controlfield[@tag="001"]')
from biblio_metadata bm1
join biblio_metadata bm2 on
ExtractValue(bm1.metadata,'//controlfield[@tag="001"]') =
ExtractValue(bm2.metadata,'//datafield[@tag="773"]/subfield[@code="w"]')
give the Report a name eg "extract_long"
2. run the report
3. report should be executed and return several 1000 lines on ktd
4. apply patch
5. add a the parameter to koha-conf.xml to the <config> </config> part
<report_sql_max_statement_time_seconds>0.001</report_sql_max_statement_time_seconds>
OR use the following statemtn
sed -i '/<biblioserver>/a <report_sql_max_statement_time_seconds>0.001<\/report_sql_max_statement_time_seconds>' /etc/koha/sites/kohadev/koha-conf.xml
6. restart koha
koha-plack --restart kohadev
7. go back to the report page and rerun report which now should lead to an error
The following error was encountered:
The database returned the following error:
Query execution was interrupted, maximum statement execution time exceeded
Please check the log for further details.
8. in koha-conf.xml set
<report_sql_max_statement_time_seconds>20</report_sql_max_statement_time_seconds>
9. restart koha
10. rerun query, which now should execute as before
11. stop ktd, and restart with the "other" database
DB_IMAGE=mysql:8.0 ktd up
DB_IMAGE=mariadb:10.5 ktd up
12. repeat from step 1 (omit applying the patch)
13. Sign off
14. Thx
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This silences warnings about undefinved values in the cache key
This seems to come from some calls in _build_languages_arrayref where we might not
have all variants of a language - the call could be prevented, but I am not sure
what the effects would be, so I simply convert undef (null value) to "undef" (string)
and use that as the cache key
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This should have no visible effect, just internal caching
To test:
1 - Do a search, check the language facets
2 - Apply patch, restart all
3 - Do a search, check the language facets
4 - No change
5 - Sign off
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The restore functionality is a cataloguing operation and should live
under the editcatalogue permission module (bit 9) rather than tools
(bit 13), consistent with where it appears in the UI.
Update all references:
- userpermissions.sql and atomicupdate: module_bit 13 → 9
- Swagger x-koha-authorization: tools → editcatalogue
- CGI flagsrequired: tools → editcatalogue
- Templates: CAN_user_tools_records_restore → CAN_user_editcatalogue_records_restore
- C4::Auth::get_cataloguing_page_permissions: remove from tools
sub-perms list (editcatalogue '*' already covers it)
- Tests: module_bit 13 → 9
Signed-off-by: Michaela <michaela.sieber@kit.edu>
Sponsored-by: ByWater Solutions
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Koha's XSLT display path now calls a plugin hook so plugins can
extend the record processor filter list before MARC is transformed.
This enables prototype punctuation filters (and similar) to run
as plugins without core changes.
Test plan:
1) Enable plugins in koha-conf.xml.
2) Install the TestPunctuation plugin from https://github.com/HKS3/koha-test-punctuation
3) Enable the plugin in Koha admin.
4) View a biblio in staff or OPAC with XSLT display enabled and confirm
the filter runs (e.g., 245$a gets a trailing period if missing).
Signed-off-by: Alexander Wagner <alexander.wagner@desy.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
When merging (or updating) authorities, the biblios using the auths might also be updated. If this happens, Koha used to create one background job for each biblio. If an auth is used in a lot of biblios, this creates a lot of background jobs, each to index one single biblio.
This patch fixes that by collecting the affected biblios and putting all of them into one reindex background job.
Test plan:
* Apply the patch
* run the unit test t/db_dependent/Authority/Merge.t
* the test should now have 16 tests in total, with a new test #15 output like this:
1..3
ok 1 - 3 bibliographic record updated while merging two authorities
ok 2 - we have only one more background job
ok 3 - we have the correct biblio ids in the job to be indexed
ok 15 - merge with ModBiblio only creates one indexer background job for multiple biblios
ok 16 - no warnings
Manual test plan (rather cumbersome):
* Start KTD with ElasticSearch
* Create an PERSON authority (eg "100a Test")
* Create three biblios which use that authority in eg 100
* Access the DB to get the count of background jobs
* `ktd --dbshell`
* `select count(*) from background_jobs;`
* the count should be the same as the number of biblios you created (eg 3). But whatever it is, remeber it
* Edit the authority (change 100 to "Test new")
* check if the change is reflected in the linked biblios (100a = "Test new")
* Check the Background jobs in the DB again:
* `select count(*) from background_jobs;`
* should be three more than previously (eg 6)
* You can use this SQL to verify that the last three jobs each contain one biblionumber (which is what we want to fix):
* `select json_extract(data,'$.record_ids'),enqueued_on from background_jobs order by enqueued_on desc limit 3;`
+-----------------------------------+---------------------+
| json_extract(data,'$.record_ids') | enqueued_on |
+-----------------------------------+---------------------+
| [2299] | 2026-03-25 13:19:34 |
| [2300] | 2026-03-25 13:19:34 |
| [2301] | 2026-03-25 13:19:34 |
+-----------------------------------+---------------------+
* Now apply the patch and run restart_all
* Edit the authority again, setting 100a to "Test new patched".
* The biblios should change accordingly
* Check the Background jobs in the DB again:
* `select count(*) from background_jobs;`
* This should now be only 1 more then the last time (so eg 7)
* Check the details of the background jobs:
* `select json_extract(data,'$.record_ids'),enqueued_on from background_jobs order by enqueued_on desc limit 3;`
* The most recent entry should now contain three items in one row, and the following two should be the same as in the previous query (i.e. the old jobs from the edit before the patch)
+-----------------------------------+---------------------+
| json_extract(data,'$.record_ids') | enqueued_on |
+-----------------------------------+---------------------+
| [2299, 2300, 2301] | 2026-03-25 13:29:17 |
| [2300] | 2026-03-25 13:19:34 |
| [2299] | 2026-03-25 13:19:34 |
+-----------------------------------+---------------------+
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This has the same goals of bug 24975 but with a different approach.
Bug 24975 tried the "POT/PO files as database tables" approach but
it appeared that is not what best fits our needs.
Instead of a "source text / translated text" mapping, we need
translations to be attached to a specific "object" (or table row), which
has the added benefit of keeping translations unaffected when the
original string changes. It may or may not be what you want but at least
this avoids the problem where fixing an error in the original string
forces users to re-translate everything.
We also need to be able to translate all translatable properties of a
particular entity.
The solution proposed here is to add a DBIC component that will add
several DBIC relations to the `localization` table:
* a relation named `localizations` that will return all translations for
all translatable properties for all languages.
* for each translatable property:
* a relation named `{$property}_localizations` that will return all
translations for a given property, for all languages
* a relation named `{$property}_localization` that will return the
translation for a given property, for the current language
To enable translations for a particular entity, we will need to:
* add two method calls in the "result class" corresponding to the
localizable object type
* in the admin interface, add a link to the the translation popup.
Unlike bug 24975 there is not a page where you can translate
everything. I don't know if it is really useful (probably better to
translate itemtypes when you are on the itemtype page). It can be
added later if needed.
To give an example, this is what needs to be added manually in
Koha/Schema/Result/Itemtype.pm:
__PACKAGE__->load_components('+Koha::DBIx::Class::Localization');
__PACKAGE__->localization_add_relationships(
'itemtype', # the PK column
'description' # a list of translatable columns
);
(see POD in Koha/Schema/Component/Localization.pm for more information)
Final notes:
* I wanted this patch to have as few changes as possible so I kept
methods like `search_with_localization`, `children_with_localization`
even if I believe they can be removed (replaced by search/children)
Localizations do not need to be prefetched/joined unless you want to
sort on them.
For the same reason, $itemtype->unblessed now always return a
`translated_description` key.
This can be fixed in followup patches
* There is cache involved (see
Koha::Schema::Component::Localization::localization).
Localizations in cache are grouped by object type and language (keys
look like 'KOHA:localization:Itemtype:en')
* By keeping a single table for all translations of all entities, we
cannot have data integrity at the RDBMS level, but automatic deletion
of translations is handled by the DBIC `cascade_delete` feature, so
integrity should be fine as long as no raw SQL update/delete operation
is involved.
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Pedro Amorim
Martin Renvoize
Lisette Scheer
Hammat Wele
Marcel de Rooy
Andreas Roussos
Olivia Reynolds
Michael Hafen
Jonathan Druart
David Cook
Lucas Gass
Kyle M Hall
Nick Clemens
Paul Derscheid
Arthur Suzuki
Martin Renvoize
Mark Hofstetter
Jacob O'Mara
Thomas Klausner
Julian Maurice