In preparation to implement 'hold for supply ILL'
This DRYs the existing functionality, meaning the new similar functionality will only be added once in the Controller::Catalogue
IMPORTANT:
The following command must only return occurrences in
Koha::Controller::Catalogue before this is pushed to main:
git grep "\->cookie(\"holdfor\")"
Signed-off-by: Jason Robb <jrobb@sekls.org>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Convert JSON response to XML if header content_type: application/xml
Signed-off-by: Jason Robb <jrobb@sekls.org>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Building on bug 41918, it would be good to limit the number of reports
a given user can run at a time. This control is config-file based as
it is tied to the server resources provided by the administrator.
Adds a new koha-conf.xml option total_running_reports_per_user_limit
(0 = disabled). When the configured limit is reached, the user is told
"You have too many reports running at the moment. Please wait for some
of them to finish before running this report again."
Note on implementation: the original patch on bz (attachment 198034)
performed a second pre-flight Koha::Reports->running call in
reports/guided_reports.pl. The bug 41918 QA follow-ups already pushed
the duplicate-running guard into Koha::Report::prep_report so it would
fire across the staff UI, svc/report and opac/svc/report. This patch
extends that same model-layer architecture rather than reintroducing
controller-layer enforcement:
- New Koha::Exceptions::Report::TotalRunning subclass of
Koha::Exceptions::Report.
- prep_report now performs a second check after the duplicate one and
throws TotalRunning when the user has total_running_reports_per_user_limit
reports of any kind already in flight.
- All three callers (guided_reports.pl, svc/report, opac/svc/report)
catch the new exception alongside DuplicateRunning, mapping it to a
new template error key total_running_reports_limit_exceeded for the
staff UI and a JSON {error: total_running_reports_limit_exceeded}
with HTTP 429 for the JSON endpoints.
Test Plan:
1) Set total_running_reports_per_user_limit to 3 in your koha-conf.xml.
2) Create 3 long-running reports like:
SELECT COUNT(*) FROM items i1 JOIN items i2 JOIN items i3 JOIN items i4 JOIN items i5 JOIN items i6
3) Run each report in turn by opening a new tab, browsing to the
reports module, and choosing to run that report.
4) Note the first three continue to run, but a fourth attempt gives
the new "too many reports running" alert in the staff UI.
5) Repeat against /cgi-bin/koha/svc/report?id=N and confirm an HTTP
429 response with JSON {error: total_running_reports_limit_exceeded}.
6) Set total_running_reports_per_user_limit back to 0 and confirm
normal behaviour resumes.
7) prove t/db_dependent/Koha/Reports.t -> 13 tests pass, including a
new subtest for TotalRunning.
Signed-off-by: Alex Carver [Acerock7] <alex@rcls.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Koha::Reports->running queries information_schema.processlist. Without
the PROCESS privilege MariaDB/MySQL already restrict the view to the
caller's own threads, so the previous code worked correctly on stock
deployments. However, if a site has granted broader privileges (eg. a
shared MariaDB server where the Koha DB user happens to also have
PROCESS, or a custom role) the query would happily count threads
belonging to other database users - another Koha instance, a reporting
tool, or any unrelated application sharing the server.
Add an explicit
user = SUBSTRING_INDEX(CURRENT_USER(), '@', 1)
clause so the query is always scoped to the connection's MySQL user,
regardless of what privileges that user has been granted. The
processlist.user column holds just the username with no host, so we
strip the '@host' off CURRENT_USER() before comparing.
The new clause has no bind parameters - it is a constant per
connection - so the existing parameterised-bind assertions in the
running subtest still hold. A small additional assertion verifies the
clause is present in the SQL text.
Test plan:
1) prove t/db_dependent/Koha/Reports.t (now 12 tests, all pass).
2) Optional: in a multi-tenant MariaDB, grant PROCESS to the Koha DB
user, start a long query as a different DB user with the marker
/* { saved_sql.id: 1 } { user_id: 1 } */ in its info, then call
Koha::Reports->running. Confirm the foreign thread is NOT counted.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The previous patches enforced duplicate_running_reports_per_user_limit
only in reports/guided_reports.pl. Two other code paths run saved SQL
reports for an authenticated user and bypassed the guard entirely:
- svc/report (staff JSON endpoint, eg. used for chart widgets and any
caller hitting /cgi-bin/koha/svc/report?id=N directly)
- opac/svc/report (OPAC JSON endpoint)
A user who hit their staff-UI limit could still spawn additional copies
of the same long-running report via either svc endpoint, defeating the
guard.
Push the check into Koha::Report::prep_report itself, which is the one
function all three callers go through. When the limit is reached
prep_report now throws a new Koha::Exceptions::Report::DuplicateRunning
(carrying report_id, user_id and the configured limit). The guided
reports CGI catches it and re-renders the existing "report is already
running" alert; both svc/report endpoints catch it and return HTTP 429
Too Many Requests with a JSON { error: "duplicate_running_report" }
payload.
Notes:
- The pre-flight Koha::Reports->running call is removed from the
guided-reports CGI; the model now owns that policy and the
controller is just rendering. The TOCTOU window remains identical
in size and is the same intentional trade-off as before.
- Limit=0 (the default) and absent userenv both bypass the check, so
CLI cronjobs (misc/cronjobs/runreport.pl) and direct DB users are
unaffected.
Test plan:
1) prove t/db_dependent/Koha/Reports.t - now includes a new subtest
covering limit=0 (lives), under limit (lives) and at limit (throws
Koha::Exceptions::Report::DuplicateRunning).
2) Set duplicate_running_reports_per_user_limit=1 in koha-conf.xml,
restart_all.
3) Open a long-running report in two staff tabs - second tab shows
the "report is already running" alert (unchanged behaviour).
4) While one run is in flight, hit /cgi-bin/koha/svc/report?id=N as
the same user - now returns HTTP 429 with the JSON error payload
(previously: ran a second copy of the query).
5) Same check on /cgi-bin/koha/opac/svc/report?id=N.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Two small documentation improvements following on from the duplicate-
running guard:
- Koha::Report::prep_report now spells out the tracking-comment format
in its POD. Koha::Reports->running parses this comment, and external
log/audit tooling that consumes Koha SQL relied on the previous
format (/* saved_sql.id: <id> */). Calling out the new format
explicitly makes the public contract visible to anyone reading the
POD or grepping for it, and warns future contributors that changing
the wrapper will break running() too.
- reports/guided_reports.pl gets an inline comment explaining the
TOCTOU window between the count check and the actual report
execution. The window is intentional - the guard exists to stop
accidental double-clicks and two-tab submissions, not to be a
serializing lock - so worst case is one extra concurrent run, which
is acceptable and cheaper than the alternative.
No behaviour change.
Test plan:
1) prove t/db_dependent/Koha/Reports.t still passes.
2) perldoc Koha::Report -> prep_report section now mentions the
saved_sql.id / user_id wrapper format.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Adds a 'running' subtest covering the new method:
- Returns a Koha::Reports resultset (object identity, not opaque ids).
- Returns an empty resultset when nothing is in flight, and when the
user_id / report_id filter does not match any process.
- Parses saved_sql.id markers out of synthetic processlist rows and
resolves them to real Koha::Report objects, ignoring rows without
the marker.
- Sends parameterised LIKE binds for user_id (proves the WHERE clause
is built without string interpolation).
- Falls back to an empty resultset and does not propagate the
exception when the underlying selectall_arrayref call dies, eg.
when the database user lacks PROCESS privilege.
To make the failure path mockable without trying to mock DBI::db
(which is not loadable as a separate file), the dbh selectall is
extracted into a tiny Koha::Reports::_processlist_rows helper so
Test::MockModule can replace it cleanly. The helper is otherwise
behaviour-preserving.
Test plan:
1) prove t/db_dependent/Koha/Reports.t -> 11 tests, all pass.
2) Existing prep_report subtest still passes.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Koha::Reports->running queries information_schema.processlist. A MySQL
user without the PROCESS privilege still sees its own threads, but if
the privilege check fails entirely (or the engine returns an unexpected
error) the previous code would let the DBI exception bubble up and
abort the calling report run.
Wrap the selectall in eval and on failure log via Koha::Logger and
treat the result as empty. This means a misconfigured database
degrades to "no enforcement" rather than to "no reports can be run",
matching the principle that the duplicate-running guard is a safeguard,
not a critical path.
The POD now documents this fallback so future readers know the empty
resultset has two possible meanings: nothing is currently running, or
we could not look. The Koha::Logger->warn ensures the latter is still
observable in the configured log destination.
Test plan:
1) prove t/db_dependent/Koha/Reports.t still passes.
2) Optional: revoke SELECT on information_schema.processlist for the
koha DB user, run a report, observe a warning in the logs and the
report still executes (with no duplicate enforcement).
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
- Fix POD synopsis: invalid syntax and wrong field (userid vs
borrowernumber) replaced with a correct example and parameter list.
- Build the WHERE clause with parameterised bindings via
$schema->storage->dbh, removing string interpolation of
$user_id / $report_id into LIKE patterns and the regex match.
IDs are formatted with %d so non-integer input becomes 0 and cannot
inject regex/LIKE metacharacters.
- Return a Koha::Reports resultset instead of a list of MySQL process
ids. The method name now matches what it returns and callers can use
the standard ->count / iteration interface. The single in-tree caller
in reports/guided_reports.pl is updated; the template error key is
simplified from duplicate_running_report_ids to a boolean
duplicate_running_report (the ids were never displayed).
Test plan:
1) Apply the previous Bug 41918 patches and this follow-up.
2) Set duplicate_running_reports_per_user_limit to 1 in koha-conf.xml.
3) Restart all the things.
4) Open a long-running saved report in two tabs and run it from both.
5) Verify the second tab shows "This report is already running..."
6) Set the limit to 0 and confirm both tabs run normally.
7) prove t/db_dependent/Reports.t still passes.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Alex Carver [Acerock7] <alex@rcls.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Bob Bennhoff <bbennhoff@clicweb.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Test plan (ktd)
0) Update the database (ktd, ignore if sandbox):
$ updatedatabase
1) Enable 'RealTimeHoldsQueue' and 'DisplayAddHoldGroups' system preferences
2) restart_all # This is required or the worker won't pick up the new sys pref values
3) Place at least 2 holds for a patron on different items/biblios, without grouping them:
<staff_url>/cgi-bin/koha/members/moremember.pl?borrowernumber=19
4) Build the holds queue once to get a baseline:
$ perl misc/cronjobs/holds/build_holds_queue.pl --force
5) Visit the holds queue and confirm the holds appear without a group indicator:
<staff_url>/cgi-bin/koha/circ/view_holdsqueue.pl?branchlimit=&itemtypeslimit=&run_report=1
6) Create a hold group for those holds via the patron page
<staff_url>/cgi-bin/koha/circ/circulation.pl?borrowernumber=<patron_borrowernumber>
7) Without running build_holds_queue.pl, refresh the holds queue, confirm the 'part of a hold group' indicator now appears
8) Ungroup the holds, refresh the holds queue, confirm the 'part of a hold group' is no longer showing
9) Run the unit tests:
$ prove t/db_dependent/Reserves/HoldGroup.t
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Replace the empty =head3 stub with a proper synopsis explaining the URL,
the supported filtering / ordering / embed surface, and the permission
requirement. Aligns the controller with the documentation density of
other Koha REST controllers and removes the "Missing POD" QA noise.
Test plan:
1. perl -MKoha::REST::V1::ActionLogs -e '' (still loads)
2. perl -c Koha/REST/V1/ActionLogs.pm (still parses)
3. perldoc Koha/REST/V1/ActionLogs.pm (renders the new POD)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
QA review noted that the new /api/v1/action_logs endpoint exposed only
column data, so the log viewer ended up showing bare borrowernumbers in
the Librarian and Object columns where the old server-side rendering
showed firstname/surname (borrowernumber). Add the DBIC plumbing needed
to support x-koha-embed for those relationships.
Two belongs_to relationships are declared below the auto-generated
marker on Koha::Schema::Result::ActionLog:
* librarian -> borrowers.borrowernumber via action_logs.user
* patron -> borrowers.borrowernumber via action_logs.object
The patron relation is intentionally polymorphic; the action_logs.object
column holds different identifiers per module (biblionumber for
CATALOGUING, subscriptionid for SERIAL, ...) so callers must filter by
module before trusting the joined patron - this is documented in both
schema and Koha::ActionLog comments.
Koha::ActionLog gains explicit librarian() and patron() accessor methods
following the Koha::Hold pattern, returning Koha::Patron objects (or
undef when the relation does not match). The accessors deliberately
avoid `use Koha::Patrons` at compile time: Koha::ActionLog is loaded
very early via C4::Log and Koha::Patron is not yet defined at that
point - the accessors reach for it lazily at call time.
koha_object_class / koha_objects_class overrides are also added to bring
the result class in line with the rest of the codebase.
Test plan:
1. prove t/db_dependent/api/v1/action_logs.t (still green)
2. perl -MKoha::ActionLog -e 'print "ok"' (no circular import)
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Mirrors the bug 42188 main commit for biblios. Replace the custom
_marc_record_to_diffable helper with MARC::Record::MiJ's to_mij_structure
so AUTHORITIES action log diffs use the same MARC-in-JSON shape as
CATALOGUING diffs. Add Koha::Authority::_unblessed_for_log to expose
only the auth_header columns that exist independently of the MARC
record (authtypecode, origincode, linkid), avoiding duplication of
MARC-derived data (heading, authtrees), raw marcxml, and system
timestamps. The explicit whitelist also removes the need for the
post-hoc empty-field stripping added in a previous follow-up.
Test plan:
1. prove t/db_dependent/AuthoritiesMarc.t
2. Enable AuthoritiesLog. Create, edit, and delete an authority via
the staff UI, then inspect action_logs.diff in the DB:
- _marc contains MiJ shape { leader: "...", fields: [ { tag: {...} }, ... ] }
- Changes to heading fields produce diffs on
_marc.fields[N].TAG.subfields[0].a
- The diff no longer carries a redundant top-level heading entry
(it's MARC-derived and now only appears in _marc).
3. prove t/db_dependent/Log.t
4. prove t/db_dependent/Koha/Authority.t
5. prove t/db_dependent/Koha/Biblio.t
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Update the existing _unblessed_for_log() subtest to reflect the new
behaviour: MARC-derived columns (title, author, copyrightdate, etc.)
are now excluded, while frameworkcode is always included even when it
is an empty string (the Default framework).
Add a new subtest 'CataloguingLog MARC-in-JSON diff tests' covering:
- ADD: diff _marc key is MARC-in-JSON (has leader + fields array),
indicators and subfield values are correct, and MARC-derived
biblio columns are absent from the diff.
- MODIFY: diff _marc key present, MARC-derived columns absent.
- DELETE: diff _marc key present with correct MARC-in-JSON structure.
Use from_json() instead of decode_json() when reading diffs back from
the database, as the DBD layer returns decoded Perl Unicode strings
which decode_json (which expects UTF-8 bytes) would reject for any
non-ASCII content.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Replace the custom _marc_record_to_diffable helper with MARC::Record::MiJ's
to_mij_structure method, which produces standard MARC-in-JSON. This fixes
three problems with the old approach:
- Dollar signs used as subfield delimiters in the formatted string were
ambiguous when subfield content itself contained " $x " sequences.
- Grouping fields by tag into a hash lost inter-field ordering, making
lossless reconstruction of the MARC record impossible.
- Non-ASCII characters were subject to Perl string context encoding
issues; JSON serialisation of the MiJ structure handles UTF-8 cleanly.
_unblessed_for_log in Koha::Biblio is updated to return only the three
biblio columns that are genuinely independent of the MARC record
(frameworkcode, opac_suppressed, serial). The remaining columns
(author, title, subtitle, notes, etc.) are denormalised copies of MARC
data already captured in the _marc key, so including them produced
redundant and noisy diffs.
The fmt() helper in viewlog.js is updated to render JavaScript arrays
as ordered lists rather than objects with numeric keys, improving the
display of MARC subfield arrays in the diff table.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch makes sure we inject the __sort form of local-number, rather
than the field directly - this ensurs searching works even if mappings mayu
be different and local-number itself is not numeric
To test:
1 - Edit mappings.yaml and set local-number from 'number' to 'string'
NOTE: There are two copies of local-number - one in auth and one in biblio - the latter is the important one here
2 - perl misc/search_tools/rebuild_elasticsearch.pl -r -v
3 - Confirm in staff interface Admin -> Search Engine Configuration that local-number is now a string
4 - Search and get an error
Error: Unable to perform your search. Please try again.
In the logs:
[illegal_argument_exception] Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [local-number] in order to load field data by uninverting the inverted index. Note that this can use significant memory.
5 - Apply patch, restart all
6 - Search again
7 - Success!
Signed-off-by: Bernard Scaife <bernard.scaife@openfifth.co.uk>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This fixes the spelling of "marc" in two places:
- an error message from running the database audit
- the description for the marc_modification_templates permission
Test plan:
1. The easiest way to test this change is to check the diff for the
patch and note that "marc" is changed to "MARC" in two places.
2. Database audit output (More > About Koha > Database audit):
- Check the diff (otherwise would need to alter the database
structure somehow so that the audit would generate the error
message).
3. Check the description for the marc_modification_templates permission:
- Go to any patron record
- Select More > Set permissions
- In the filter, enter: marc modification
- Note that the description is:
Manage marc modification templates (marc_modification_templates)
4. Apply the patch.
5. Repeat step 3 and note that the description is now:
Manage MARC modification templates (marc_modification_templates)
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Fixes regression in which default expiry date is ignored due to regression introduced by fix for bug 34483. If default expiry date is completed in patron import tool, the value is ignored and expiry date unaltered. This patch fixes this error.
Test plan:
1. User should exist and have expiry date in future (say 2099-12-31)
2. Save above as file
3. Tools > Import users
4. Choose file saved in 2
5. Field to use for record matching: cardnumber
6. In default values set expiration date to a date in past (say 2025-10-19) using date picker
7. Select overwrite the existing one with this
8. Leave renew existing patrons as "use the expiration date in the file if present"
9. Run
10. Observe user expiry date is still 2099-12-31
11. Apply patch / restart_all
12. Repeat 4 - 9 above
13. Observe user expiry date is now 2025-10-19
Signed-off-by: Eric Phetteplace <ephetteplace@cca.edu>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Rename /libraries/{id}/holidays to /libraries/{id}/closed_dates
for consistency with existing sub-resource naming (desks,
cash_registers).
Efficiency: instead of calling $calendar->is_holiday() per day
(which re-checks all three data sources each time), access the
pre-loaded data structures directly: weekly_closed_days array,
day_month_closed_days hash, and _holidays hash. The logic properly
handles exceptions (dates marked open despite a weekly/annual
closure).
Additional fixes:
- Add 365-day max range guard
- Add missing 401/403 responses in swagger spec
- Add request_id_header parameter
- Test exception handling (open on normally-closed day)
- Remove fragile 'empty defaults' assertion
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
- Add list_holidays controller to Koha::REST::V1::Libraries
- Returns closed days for a library within a configurable date range
- Uses Koha::Calendar to determine holidays
- Defaults to today + 3 months if no range specified
- Add OpenAPI spec for the new endpoint
- Register route in swagger.yaml
- Add unit tests in t/db_dependent/api/v1/libraries.t
To test:
1. Apply patch
2. prove t/db_dependent/api/v1/libraries.t
3. Verify all tests pass, including the new list_holidays subtest
which covers:
- Holidays returned within a specified date range
- Empty list when no holidays exist in range
- 400 response when 'to' date precedes 'from' date
- 404 response for non-existent library
- Default range (no params) returns successfully
Sponsored-by: Büchereizentrale Schleswig-Holstein <https://www.bz-sh.de/>
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Update suggestion action logging to populate the diff column using the
correct before/after semantics:
- CREATE: empty before-state ({}), new suggestion data as after-state
- MODIFY: pre-change unblessed data as before, post-store object as after
- DELETE: log before the SQL DELETE so the object can still be fetched;
pass the suggestion as both $infos (info column) and $original (diff
before-state), logaction sets after-state to {}
Also add tests verifying the diff column is populated for all three actions.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The OPAC REST self-checkout path (guarded by OpacTrustedCheckout) is
functionally SCO-over-API, so it should honour the same hold policy
as the SCO web interface. Previously _check_availability unconditionally
promoted RESERVED, RESERVE_WAITING, TRANSFERRED and PROCESSING from
the confirmation hash to the blockers hash for any public caller. This
meant the OPAC REST endpoint always refused held items regardless of
the librarian's AllowItemsOnHoldCheckoutSCO configuration.
With this patch the promotion is gated on AllowItemsOnHoldCheckoutSCO:
* 0 - Promote all four hold confirmations to blockers (previous
behaviour for pending holds)
* 1 - Promote only waiting/transit/processing; silently allow
checkout of items with a pending hold
* 2 - Promote none; silently allow checkout of items with any hold
Remaining hold confirmations are stripped from the response so the
caller does not have to resolve a confirmation token for a policy the
admin has already granted.
Test plan:
1. prove t/db_dependent/api/v1/checkouts.t
2. Enable OpacTrustedCheckout. POST an item with a pending hold for
another patron to /api/v1/public/patrons/X/checkouts:
- AllowItemsOnHoldCheckoutSCO=0: 403 Checkout not authorized
- AllowItemsOnHoldCheckoutSCO=1: 201 Created
3. Place a waiting hold for another patron on an item. Repeat:
- AllowItemsOnHoldCheckoutSCO=1: 403
- AllowItemsOnHoldCheckoutSCO=2: 201
Signed-off-by: Kristi Krueger <kkrueger@cuyahogalibrary.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Extended attributes weren't being identified as changes and so need to be checked to see if any of them have changed
Sponsored-by: Westminster City Council <https://www.westminster.gov.uk/>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Christopher Brannon <cbrannon@cdalibrary.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Christopher Brannon <cbrannon@cdalibrary.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Adds categories to the db insertion mechanics and allows values to be passed to the patron insertion method
Signed-off-by: Christopher Brannon <cbrannon@cdalibrary.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This is part of a sponsored piece of development from one of our customers. There was a specification of requirements but I was conscious of the ideas already discussed on this bug and wanted to try and develop a solution that would balance both sets of requirements. The concept works as follows:
- Self-renewal can be enabled and controlled at a category level
- The window within which it is available is customisable (both pre and post-expiry)
- Defaults to the NotifyBorrowerDeparture syspref
- The amount in fines that block self-renewal is customisable
- Defaults to the category level charges and then the noissescharge preference
- The message to display if renewal fails is customisable
- Libraries can also add information to display to the patron at the start of the process.
- The self-renewal process will also respect the OPACPatronDetails and AutoApprovePatronProfileSettings sysprefs. If OPACPatronDetails is enabled, the renewal will redirect the patron to the OPAC details form and ask them to confirm their details are still correct. Any changes will be logged as patron modifications and approved in line with the auto approval syspref
- Renewal happens regardless of whether OPACPatronDetails is enabled, only the details changes are logged as modification requests
- Notifications are sent to the patron in line with their messaging preferences
Test plan:
1. Apply all patches and reset_all
2. In the intranet, navigate to patron categories and edit the ‘Patron’ category
3. Scroll to the bottom, there should be a section called ‘Account expiry and self-renewal’
4. Set ‘Allow self-renewal’ to Yes
5. Set Self-renewal availability period to be 10 days
6. Set Self-renewal availability post-expiry’ to also be 10 days and save
7. In system preferences, set the OPACPatronDetails preference to ‘Don’t allow’
8. Choose a patron that has the ‘Patron’ type category
9. Edit that patron so that you can log into the OPAC and set their expiry date to be yesterday’s date
10. Log into the OPAC as that patron (it may be easiest to do this in incognito as we will be swapping between the intranet and OPAC throughout the test plan and it will remove the need to keep logging out and in)
11. You should see a message that says “You are eligible for self-renewal. Please click here to renew your account.”
12. Click that link - you will be presented with a modal that says “Are you sure you want to renew your account?”
13. Click “Close” - we don’t want to complete renewal yet as there is more functionality available to test
14. Navigate to the patron category again, set ‘Self-renewal availability post-expiry’ to be 0
15. Refresh the OPAC, the self-renewal link should not be there as we are now outside the patron self-renewal window
16. Undo that change in the patron category, the link should reappear when refreshing the OPAC
17. Now set the patron’s expiry date to be tomorrow
18. In the category, set ‘Self-renewal availability period’ to be blank
19. Refresh the OPAC, it should still show the link as it is defaulting to NotifyBorrowerDeparture
20. Set NotifyBorrowerDeparture to 0
21. Refresh the OPAC, the link should vanish
22. In the category, restore ‘Self-renewal availability period’ to 10 days
23. Refresh the OPAC, the link should reappear
24. In the intranet, set the patron to be debarred using a manual restriction
25. In the OPAC refresh the page - self-renewal should no longer be available
26. Remove the restriction - the renewal link will be visible again
27. Set a fine on the patron record of 6 USD
28. Refresh the OPAC, the self-renewal link will be gone because we are over the global ‘noissuescharge’ preference value of 5
29. In the category, set ‘Checkout charge limit’ to 7
30. Refresh the OPAC, the link will be visible because we are now under the category charge
31. Add an additional fine of 2 to the patron account
32. Refresh the OPAC, the self-renewal link is gone
33. In the category, set ‘Self-renewal charge limit’ to be 10
34. Refresh the OPAC, the link will be visible again
35. In the category, set ‘Self-renewal failure message’ to be whatever you want
36. In the OPAC, refresh the page and then click into the self-renewal process
37. This time, click “No” - you should see the failure message you have just set
38. Click “Close” and refresh the page to reset this message
39. In the category, set a message in the “Self-renewal information message”
40. In the OPAC, refresh the page and then click the renewal link again
41. You should be met with the modal, this time displaying the information message we just set up in the patron attributes
42. Click “Yes” - you should see the “Are you sure you want to renew your account?” message again
43. This time when prompted with the “Are you sure” message, click “Yes”
44. The page should refresh and display a confirmation with the new expiry date.
45. In the intranet, view the patron’s record - the date should have been updated
46. In the Modification log tab, there should be a log entry for “Membership renewed”
47. Change the patron’s expiry date again, this time to be yesterday’s date
48. In the OPAC, click on “Personal detail”, the new expiry date should show there
49. Now set the OPACPatronDetails syspref to be “Allow”
50. Click on “Summary” again, the self-renewal link should be there again
51. Click into the self-renewal process and click “Yes” on your verification step
52. This time you should be met with a message that says “Confirm your account details”
53. Click “Continue” - you will be redirected to the OPAC details form
54. There should be a message at the top that says “Please verify your details to proceed with your self-renewal.”
55. Update a field somewhere on the patron form
56. Scroll to the bottom - the button should say “Submit renewal request”
57. Click this button
58. You will be shown the renewal confirmation message
59. In the intranet, on the homepage you should now see a modification request
60. Check the request, it should match the fields you changed - don’t approve or deny for now
61. Set ‘AutoApprovePatronProfileSettings’ in sysprefs to “Enable”
62. Reset your patron’s expiry date to today’s date
63. In the OPAC, click into Personal details and then back into Summary again (we need to do this to remove the success parameter from the URL)
64. The renewal link will be visible again - go through the process again
65. On the form, you should see an alert that you have already sent a modification request
66. This time, change a different field to the previous attempt
67. Submit the form
68. In the intranet there should be no modification this time and the change should be visible in the patron record as it has been automatically approved and handled as per the syspref
69. prove t/db_dependent/Koha/Patron.t
70. prove t/db_dependent/api/v1/patrons_self_renewal.t
71. yarn cypress run --spec t/cypress/integration/Islands/SelfRenewal.ts
72. Sign off!
Signed-off-by: Christopher Brannon <cbrannon@cdalibrary.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Christopher Brannon <cbrannon@cdalibrary.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
POST /api/v1/authorities currently offers only the heading-based
C4::AuthoritiesMarc::FindDuplicateAuthority check, and the only
escape hatch is the `x-koha-override` header.
Koha already ships configurable record matching rules (see
Administration > Record matching rules) and C4::Matcher already
supports `record_type = authority`. This patch lets an API caller
opt into one of those rules when creating an authority by sending
its numeric id in a new `x-koha-matchrule` header.
Behaviour:
* If no `x-koha-matchrule` header is sent the existing heading-based
duplicate check is used. Existing clients are unaffected.
* If `x-koha-matchrule: <id>` is sent:
* non-existent id -> 400 `matching_rule_not_found`
* matcher not record_type=authority
-> 400 `invalid_matching_rule_type`
* matcher returns matches -> 409 `duplicate`; the response
body additionally contains a
`matches: [{authid, score}, ...]`
array listing every hit returned
by the matcher
* matcher returns nothing -> record is created (201)
* `x-koha-override: any|duplicate` continues to bypass all duplicate
checks, including the matcher.
A non-integer header value is rejected by the OpenAPI schema
validator before the controller runs.
Test plan:
1) Apply the patch series.
2) Run `prove t/db_dependent/api/v1/authorities.t`
=> all subtests pass.
3) Log in to the staff interface as a user with the
'manage_matching_rules' subpermission and go to
Administration > Record matching rules > New record matching
rule:
a) Set Matching rule code e.g. `AUTH_HEADING`
b) Set Description e.g. `Authority heading match`
c) Set Record type to `authority`
d) Set Threshold to e.g. `1000`
e) Add a match point:
* Search index: `Heading-Main` (or your authorities index)
* Score: `1000`
* Tag: `110` (or the heading tag used by your
authority type)
* Subfields: `a`
f) Save. Note the numeric matching rule id in the edit URL
(`admin/matching-rules.pl?op=edit_matching_rule&matcher_id=N`).
4) Export an existing authority record as MARCXML (for example
from the authority details page). Save it as `duplicate.xml`.
5) From the command line, POST the record back to Koha using the
new header (replace <N>, credentials and host):
curl -u <user>:<pass> -X POST \
-H 'Content-Type: application/marcxml+xml' \
-H 'x-authority-type: CORPO_NAME' \
-H 'x-koha-matchrule: <N>' \
--data-binary @duplicate.xml \
http://<your-koha>/api/v1/authorities
=> Expect HTTP 409. The JSON response body contains
`error_code: "duplicate"` and a `matches` array listing the
authid(s) that were found along with their score.
6) Edit `duplicate.xml` so the main heading is something that is
not already in the catalogue. POST it again with the same
header.
=> Expect HTTP 201 and a `Location` header pointing to the new
authority.
7) Re-run the POST from step 5 but additionally send
`-H 'x-koha-override: duplicate'`.
=> Expect HTTP 201 (override bypasses the matcher).
8) Send a POST with `x-koha-matchrule: 999999`.
=> Expect HTTP 400 with `error_code: "matching_rule_not_found"`.
9) Create a second matching rule with Record type `biblio` and
note its id. POST with `x-koha-matchrule` set to that id.
=> Expect HTTP 400 with
`error_code: "invalid_matching_rule_type"`.
10) POST with `x-koha-matchrule: abc`.
=> Expect HTTP 400 from the OpenAPI schema validator.
11) POST without `x-koha-matchrule`.
=> Expect the existing heading-based duplicate check to run
exactly as before (409 for a known duplicate, 201 otherwise).
12) Sign off.
Signed-off-by: Brian J. Barr <brian.barr@churchofjesuschrist.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
When autoMemberNum is enabled, Koha picks the next cardnumber as
MAX(numeric cardnumber in borrowers) + 1. If patron A's auto-generated
cardnumber is later changed (e.g. they get a physical card) or the
patron is deleted, the original number becomes "free" and the next
self-registrant is assigned it -- inheriting anything third-party
services (OverDrive, Hoopla, unauthenticated self-check) have keyed on
the cardnumber.
This patch makes Koha::Patron::fixup_cardnumber use a monotonic counter
(the autoMemberNumValue syspref) so that once a number has been
auto-generated for any patron, it is never auto-generated again.
The syspref row is locked with SELECT ... FOR UPDATE inside the
existing txn_do, which serialises concurrent callers via InnoDB
row-level locking and addresses the race condition raised against
earlier revisions of this patch.
The formula is: next = MAX(counter, db_max) + 1. When the counter is 0
(pre-upgrade or missing syspref), this degrades gracefully to
db_max + 1 -- the legacy behaviour -- so no feature toggle is needed.
The counter is only bumped when Koha actually auto-generates a
cardnumber; plugin- or caller-supplied values do not touch it.
Test plan:
1. Enable autoMemberNum.
2. Self-register patron A via OPAC, note cardnumber N.
3. Edit patron A in the staff interface, change cardnumber to
something non-numeric like "MANUAL-XYZ".
4. Self-register patron B.
5. Confirm B gets N+1, NOT N.
6. Delete patron A.
7. Self-register patron C.
8. Confirm C gets N+2, NOT A's original number.
9. Install a plugin implementing the patron_barcode_transform hook;
confirm it still runs and its value is honoured (counter is not
bumped when the plugin supplies a cardnumber).
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The safe_do wrapper adds indirection without value:
- In Patron::Modification::approve, every operation inside safe_do
goes through Object::store or Object::delete, which already call
translate_exception internally. The wrapper was translating
already-translated exceptions.
- In Item::add_to_bundle, the outer try/catch just called rethrow,
making the safe_do + try/catch nesting a no-op.
This follow-up removes safe_do and aligns both callers with the
pattern already used by Object::store and Object::delete: call
translate_exception directly in the catch block when raw DBIC
operations are involved.
Changes:
- Remove safe_do from Koha::Schema (and its Try::Tiny import)
- Flatten add_to_bundle to use translate_exception in its catch
- Flatten Patron::Modification::approve to rely on Object::store
- Remove safe_do test, adjust test count
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The QA tools flag all methods without POD in any touched file.
This patch adds the missing POD for pre-existing methods that were
flagged:
- Koha::Item::_add_statistic
- Koha::Object::_date_or_datetime_column_type
- Koha::Object::_datetime_column_type
- Koha::Object::_numeric_column_type
- Koha::Object::_decimal_column_type
- Koha::Object::_recursive_fixup
- Koha::Object::_get_object_class
- Koha::Object::DESTROY
- Koha::Patron::Modification::_type (also fixes heading mismatch)
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Replace the implicit exception translation call with an explicit safe_do
pattern that clearly shows exceptions are rethrown.
Changes:
- Use $schema->safe_do() wrapper for automatic exception translation
- Add explicit $_->rethrow() in catch block for clarity
- Follow the same pattern used in Koha::Patron::Modification::approve()
- Make control flow more obvious to future maintainers
This change maintains identical behavior while improving code clarity
by making it explicit that the catch block never returns normally.
Sponsored-by: OpenFifth <https://openfifth.co.uk>
Signed-off-by: Jackie Usher <jackie.usher@westsussex.gov.uk>
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Complete the migration of enum data truncation exception handling from
Koha::Object to the centralized ExceptionTranslator utility. This eliminates
the last remaining manual DBIx::Class exception handling in Object code.
Changes:
- Add optional object parameter to ExceptionTranslator->translate_exception()
- Update Schema wrapper methods to accept and pass object parameter
- Enhance enum truncation handling to extract actual property values when object is available
- Remove manual enum exception handling from Koha::Object::store()
- Add comprehensive test coverage for enum handling with and without object context
- Maintain contextual warning behavior for store() operations
The centralized system now handles ALL DBIx::Class exception scenarios:
- FK constraint failures (add/update and delete operations)
- Duplicate key violations
- Invalid data type values
- Enum data truncation with actual property values
This completes the centralization effort while providing better error reporting
than the previous manual approach.
Sponsored-by: OpenFifth <https://openfifth.co.uk>
Signed-off-by: Jackie Usher <jackie.usher@westsussex.gov.uk>
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Replace manual DBIx::Class::Exception handling in the approve() method with
the centralized exception translation system while preserving domain-specific
exception behavior.
Changes:
- Use $schema->safe_do() for database operations with automatic exception translation
- Wrap safe_do in try/catch to convert any exception to domain-specific Patron::Modification exception
- Simplify exception handling logic while maintaining the same API contract
- Remove manual DBIx::Class::Exception type checking
This maintains backward compatibility by always throwing
Koha::Exceptions::Patron::Modification exceptions as before, but leverages
the centralized system for consistent database exception handling.
Sponsored-by: OpenFifth <https://openfifth.co.uk>
Signed-off-by: Jackie Usher <jackie.usher@westsussex.gov.uk>
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Pedro Amorim
Martin Renvoize
Lucas Gass
Kyle M Hall
Nick Clemens
Simon
Bernard Scaife
Tomás Cohen Arazi
Paul Derscheid
Matt Blenkinsop
David Cook