The daemon runs as a long-lived process and never clears its L1 cache.
System preference changes (e.g. IncludeSeeFromInSearches) are not
picked up until the daemon is manually restarted.
Flush L1 caches at the start of each commit() call so that preference
changes take effect on the next indexing batch without requiring a
daemon restart.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
There was a mismatch between the documentation and the shortcut used
in the code. This patch fixes the documentation so it matches the code:
'pf|preserve-field=s' => \@preserve_fields,
=item B<-k|--preserve-field>
Prevent specified patron fields for existing patrons from being overwritten
So pf is the right shortcut.
Signed-off-by: Aleisha Amohia <aleisha@catalyst.net.nz>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
The patch added cronlogaction() calls but did not import the function.
This caused a runtime error: Undefined subroutine &main::cronlogaction.
Test plan:
1. Apply patch
2. Run:
$ ktd --shell
k$ perl misc/cronjobs/cleanup_database.pl --illrequests-days 30 \
--illrequests-status COMP -v
=> SUCCESS: Dry-run works, reports count of requests that would be removed
3. Run with --confirm to actually purge
4. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This adds two new parameters:
--illrequests-status
--illrequests-days (optional, defaults to 30 days)
These can be repeated to include specific statuses for cleanup. At least one status must be specified.
To test:
0 - Apply patch
1 - Create several ILL requests with different statuses (e.g., CANCELLED, REJECTED, COMPLETED, PENDING)
2 - Set some requests to be older than 30 days (update their updated timestamp)
3 - Run cleanup_database.pl with --illrequests-status CANCELLED
4 - CANCELLED requests older than 30 days are removed, others remain
5 - Run cleanup_database.pl with --illrequests-status CANCELLED --illrequests-status REJECTED
6 - Both CANCELLED and REJECTED requests older than 30 days are removed, other statuses remain
7 - Run cleanup_database.pl with --illrequests-days 5 --illrequests-status PENDING
8 - PENDING requests older than 5 days are removed
9 - Verify dry-run mode (no --confirm) correctly counts records that would be deleted without deleting them
10 - Run cleanup_database.pl with --illrequests-days without any --illrequests-status
11 - Error message displayed requiring at least one status to be specified
Sponsored-by: Koha-Suomi Oy
Signed-off-by: John Doe <tomascohen@gmail.com>
edit: tidied the file (tcohen)
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
When running the script bulkmarcimport.pl, the number of MARC records reported at the end of the execution does not match the actual number of records processed from the input file.
This patch fixes 3 issues
- broken (invalid) records from batch get now counted (record_number)
- when using the number param (n) to limit the number of records to be processed valid and invalid records get counted
- when using th offset param (o) the scipt errored when broken records were part of the offset.
To test:
1. Download the MARC file with the broken record
2. Run the following import commands and compare the counted import numbers:
a) all records
perl misc/migration_tools/bulkmarcimport.pl -b -v -m MARCXML --file 2_records_1st_broken.xml
1 MARC records done in 0.0421221256256104 seconds
b) limit to 1 record in total
perl misc/migration_tools/bulkmarcimport.pl -b -v -n 1 -m MARCXML --file 2_records_1st_broken.xml
1 MARC records done in 0.0420489311218262 seconds
c) limit to 1 record offset 1 record
perl misc/migration_tools/bulkmarcimport.pl -b -v -n 1 -o 1 -m MARCXML --file 2_records_1st_broken.xml
:2: parser error : Opening and ending tag mismatch: leader2 line 2 and leader
<leader2>00216nam a22001097a 4500</leader>
3. Apply the patch
4. Repeat steps in 2 and compare the results
a) 2 MARC records done in 0.0405838489532471 seconds
b) 1 MARC records done in 0.00948596000671387 seconds *
c) 1 MARC records done in 0.0294840335845947 seconds
*though this one was skipped
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
When running the script bulkmarcimport.pl, the number of MARC records reported at the end of the execution does not match the actual number of records processed from the input file.
To test:
1. Download the joined MARC file containing 5 records.
2. Run the import command:
./misc/migration_tools/bulkmarcimport.pl -b --file=marc_file.mrc
3. Check the final script report
==> 11 MARC records done in 0.566349983215332 seconds
Expected output should show 5 records
4. Apply the patch
5. Repeat step 2, 3
==> The correct number is shown
Signed-off-by: Janusz Kaczmarek <januszop@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Add Koha::Script -cron to restrict_patrons_with_failed_notices.pl
and sftp_file.pl which used cronlogaction directly without importing
Koha::Script. Remove their now-redundant manual calls.
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Remove start/end cronlogaction calls and unused C4::Log imports
from scripts that use Koha::Script, as these are now handled
automatically by the INIT/END blocks.
Scripts with mid-execution informational logging (plugins_nightly,
harvest_oai, fines, process_message_queue) retain their C4::Log
import and non-start/end cronlogaction calls.
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Add INIT and END blocks to Koha::Script that automatically call
cronlogaction on script start (with command line options) and
completion. This removes the need for scripts to manually handle
cron log entries.
This will mean that any script importing Koha::Script will be logged,
but I think the addition is beneficial.
To test:
1 - Ensure CronjobLog is enabled
2 - perl misc/cronjobs/fines.pl --test
3 - Confirm start and end are logged in action logs:
http://localhost:8081/cgi-bin/koha/tools/viewlog.pl?src=&user=&modules=CRONJOBS&actions=&object=&info=&interfaces=&from=&to=&output=screen&basename=Export&report_name=&do_it=1
4 - Apply patch
5 - Repeat 2
6 - Reload the logs, confirm the new logs look the same
7 - Sign off
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Always include the plural version, not the singular one (the plural
imports the singular)
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Laura_Escamilla <laura.escamilla@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch replaces all calls to C4::Stats::UpdateStats with the wrapped
function from the Koha namespace. As this is the last function in C4::Stats, the
module is also deleted.
Several tests depended on transitive includes to Koha::Statistics through
C4::Stats. I've identified uses of that module by searching, and added the
necessary use in each case. I've run `prove -r t xt` with this patch applied and
didn't hit any relevant errors.
Test plan:
1. Inspect the patch, and verify each changed call maintains the same parameters.
2. Apply patch.
3. Prove the following files:
- t/db_dependent/Accounts.t
- t/db_dependent/Circulation.t
- t/db_dependent/Koha/Account.t
- t/db_dependent/Koha/ILL/Request.t
- t/db_dependent/Koha/Item.t
- t/db_dependent/Koha/Pseudonymization.t
- t/db_dependent/Koha/Recalls.t
Sponsored-by: Open Fifth
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Laura_Escamilla <laura.escamilla@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch adds two new parameters to control the deletion of background jobs that did not complete.
Some libraries may want to keep a failed job longer than a completed one for debugging reasons.
To test:
0 - Apply patch
1 - Make some old failed jobs:
INSERT INTO background_jobs (status,type,enqueued_on) VALUES
('started','update_elastic_index','2020-01-01'),
('failed','update_elastic_index','2026-01-01'),
('started','other_type','2021-01-01'),
('failed','other_type','2026-01-01');
2 - Confirm defaults of 365 and update_elastic_index for new params:
perl misc/cronjobs/cleanup_database.pl -v --jobs-unfinished-days
3 - Should indicate 1 job older than 365 of type update_elastic_index would have been deleted
4 - Confirm days param works
perl misc/cronjobs/cleanup_database.pl -v --jobs-unfinished-days 30
5 - Shoudl find 2 elastic jobs
6 - Confirm type param
perl misc/cronjobs/cleanup_database.pl -v --jobs-unfinished-days --jobs-unfinished-type=other_type
7 - Should find 1
8 - Confirm type and days
perl misc/cronjobs/cleanup_database.pl -v --jobs-unfinished-days 30 --jobs-unfinished-type=other_type
9 - Should find 2
10 - Confirm repeated type
perl misc/cronjobs/cleanup_database.pl -v --jobs-unfinished-days=30 --jobs-unfinished-type=other_type --jobs-unfinished-type=update_elastic_index
11 - Finds 4
12 - Confirm deletion
perl misc/cronjobs/cleanup_database.pl -v --jobs-unfinished-days=30 --jobs-unfinished-type=other_type --jobs-unfinished-type=update_elastic_index --confirm
13 - All your made up jobs are removed
Signed-off-by: Trevor Diamond <trevor.diamond@mainlib.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Two follow-up fixes from QA review of the previous commit:
- Detect Search::Elasticsearch::Error::NoNodes via ref($_) rather than
regex-matching the stringified exception, matching the existing
pattern already used in about.pl.
- Only reset jobs to 'new' for retry when NoNodes is the sole problem.
If a genuine indexing error also occurred in the same batch, record
the batch as 'failed' instead of silently discarding that error via
the NoNodes retry path.
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
When update_index() throws, the daemon now:
- On NoNodes: aborts the batch, recreates the indexer object to reset
the dead-node connection pool, and resets jobs to 'new' for retry.
- On other errors: marks jobs as 'failed' instead of 'finished'.
This prevents silent data loss where records disappear from search
results because their indexing jobs were incorrectly marked as done.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Note: this requires a command line access to the test machine
TEST PLAN:
1 - Anywhere on the computer running koha, run following command:
mkdir tmp
echo '<!DOCTYPE html><html><head><meta charset="UTF-8"></head><body><p>Тетрис — это классика!</p></body></html>' > tmp/tmp.html
echo '* { color: red };' > tmp.css
./misc/cronjobs/printoverdues.sh tmp tmp.css
2 - Open the pdf in tmp/tmp.pdf, notice it is broken
3 - Apply patch
4 - Rerun ./misc/cronjobs/printoverdues.sh tmp tmp.css
5 - Open the pdf in tmp/tmp.pdf, notice it is now displaying properly, notice the css property still applies
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Bug 40866 introduced JSON logging for HOLDS CREATE but stored null for
'confirmations' and 'forced' when no overrides were present. This patch
mirrors the earlier change to CIRCULATION ISSUE by defaulting them to
empty arrays, keeping the two log streams structurally identical and
simplifying reporting queries.
Changes:
C4/Reserves.pm:
- Default confirmations/forced to [] in the AddReserve log payload.
t/db_dependent/Reserves.t:
- Update the Bug 40866 subtest to assert the empty-array defaults on
normal holds, and fix a 'my $log_data' redeclaration.
tools/viewlog.pl:
- Extend the CSV export JSON-decode block to also cover HOLDS logs,
guarding the item lookup on itemnumber being present (HOLDS payloads
may lack one) and filling the CSV item columns from the decoded
payload. The legacy bare-itemnumber fallback is restricted to
CIRCULATION so a legacy HOLDS hold_id is not mistaken for an
itemnumber.
koha-tmpl/intranet-tmpl/prog/js/viewlog.js:
- Extend the info column override rendering (renderCircInfo) to HOLDS
entries so forced/confirmations arrays display translated, exactly as
they do for CIRCULATION. The log viewer moved to an ajax datatable in
bug 29016, so this replaces the earlier json_found change to
viewlog.pl/viewlog.tt.
misc/maintenance/migrate_action_logs_to_json.pl:
- Extended to handle legacy pre-Bug 40866 HOLDS CREATE logs where info
is a bare hold_id, recovering branchcode / biblionumber / itemnumber
from old_reserves / reserves.
- Uses the same keyset-pagination pattern as the circulation loop so
rows are not skipped under --commit.
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch adds an out-of-band migration script to convert legacy
CIRCULATION ISSUE action logs from the old itemnumber-only format
to the new consistent JSON format.
The script:
- Finds all CIRCULATION ISSUE logs where info is just an itemnumber
- Converts them to JSON format with the standard structure
- Attempts to recover issue_id and branchcode from old_issues/issues
- Runs in dry-run mode by default (--commit flag to actually update)
- Processes in batches to handle large databases safely
- Includes verbose mode for detailed output
Usage:
perl misc/maintenance/migrate_action_logs_to_json.pl --help
perl misc/maintenance/migrate_action_logs_to_json.pl --verbose
perl misc/maintenance/migrate_action_logs_to_json.pl --commit
This is designed as an out-of-band maintenance script rather than
an atomic update because:
1. May be expensive on large databases with many log entries
2. Migration is optional - display code handles both formats
3. Can be run during off-peak hours at admin's discretion
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Like Bug 32012 for runreport.pl the script search_for_data_inconsistencies.pl should use binmode UTF-8 to correctly display datas.
Test plan:
1) Edit an authorised value to add some diacritics in code, in "LOC" for example
2) Edit an item to use this code
3) Delete this authorised value
4) Run misc/maintenance/search_for_data_inconsistencies.pl
=> Check you have a correct message with diacritics
Signed-off-by: Chris Mathevet <chris.mathevet@inLibro.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
To test:
Test the cronjob purge_suggestions.pl
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
The stockrotation cronjob was only sending email reports when there were
actionable items (items that needed to be moved between branches). This
meant that libraries would not receive any notification when all items
in their rotation were stationary, even when the --send-all flag was
passed to the script.
This patch modifies the report logic to also send reports when there are
only stationary items, ensuring libraries have visibility into the complete
status of their stock rotation items.
Test plan:
1. Set up a stock rotation rota with items that are all stationary
(items that should stay at their current branch)
2. Run the stockrotation cronjob: perl misc/cronjobs/stockrotation.pl
3. Without this patch: No email report is generated
4. Apply this patch
5. Run the cronjob again
6. Verify that an email report is now generated showing the stationary items
7. Test with mixed scenarios (some actionable, some stationary items)
8. Confirm reports are sent in all cases where items exist in the rotation
Signed-off-by: Laura_Escamilla <laura.escamilla@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Correct genuine spelling errors identified by the codespell 2.4.x
full scan after the KTD container OS upgrade. Changes are in
comments, POD, strings, and template text across 55 files.
"SAN Ouest Provence" (the French library co-operative and Koha
sponsor) is preserved as a proper noun. Perl copyright lines
receive an inline # codespell:ignore annotation; about.tt files
(contributor attribution pages) are added to the Koha::Devel::Files
codespell exception list so that future uses of "Provence" as a
typo for "Province" are still caught in all other files.
Notable user-visible fixes:
- "submiting" -> "submitting" in authority and bibliographic merge screens
- "deleteable" -> "deletable" in patron exception messages
- "Acquistions" -> "Acquisitions" in reports dictionary
Test plan:
1. Apply patches
2. In the staff interface, go to Cataloguing > Merge records and
confirm the submit button reads "Submitting" (not "Submiting")
3. In the staff interface, go to Authorities > Merge and confirm
the same correction
4. Attempt to delete a patron with outstanding issues and confirm
the error message reads "deletable" (not "deleteable")
5. prove xt/author/codespell.t
=> All tests pass
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This is part of a sponsored piece of development from one of our customers. There was a specification of requirements but I was conscious of the ideas already discussed on this bug and wanted to try and develop a solution that would balance both sets of requirements. The concept works as follows:
- Self-renewal can be enabled and controlled at a category level
- The window within which it is available is customisable (both pre and post-expiry)
- Defaults to the NotifyBorrowerDeparture syspref
- The amount in fines that block self-renewal is customisable
- Defaults to the category level charges and then the noissescharge preference
- The message to display if renewal fails is customisable
- Libraries can also add information to display to the patron at the start of the process.
- The self-renewal process will also respect the OPACPatronDetails and AutoApprovePatronProfileSettings sysprefs. If OPACPatronDetails is enabled, the renewal will redirect the patron to the OPAC details form and ask them to confirm their details are still correct. Any changes will be logged as patron modifications and approved in line with the auto approval syspref
- Renewal happens regardless of whether OPACPatronDetails is enabled, only the details changes are logged as modification requests
- Notifications are sent to the patron in line with their messaging preferences
Test plan:
1. Apply all patches and reset_all
2. In the intranet, navigate to patron categories and edit the ‘Patron’ category
3. Scroll to the bottom, there should be a section called ‘Account expiry and self-renewal’
4. Set ‘Allow self-renewal’ to Yes
5. Set Self-renewal availability period to be 10 days
6. Set Self-renewal availability post-expiry’ to also be 10 days and save
7. In system preferences, set the OPACPatronDetails preference to ‘Don’t allow’
8. Choose a patron that has the ‘Patron’ type category
9. Edit that patron so that you can log into the OPAC and set their expiry date to be yesterday’s date
10. Log into the OPAC as that patron (it may be easiest to do this in incognito as we will be swapping between the intranet and OPAC throughout the test plan and it will remove the need to keep logging out and in)
11. You should see a message that says “You are eligible for self-renewal. Please click here to renew your account.”
12. Click that link - you will be presented with a modal that says “Are you sure you want to renew your account?”
13. Click “Close” - we don’t want to complete renewal yet as there is more functionality available to test
14. Navigate to the patron category again, set ‘Self-renewal availability post-expiry’ to be 0
15. Refresh the OPAC, the self-renewal link should not be there as we are now outside the patron self-renewal window
16. Undo that change in the patron category, the link should reappear when refreshing the OPAC
17. Now set the patron’s expiry date to be tomorrow
18. In the category, set ‘Self-renewal availability period’ to be blank
19. Refresh the OPAC, it should still show the link as it is defaulting to NotifyBorrowerDeparture
20. Set NotifyBorrowerDeparture to 0
21. Refresh the OPAC, the link should vanish
22. In the category, restore ‘Self-renewal availability period’ to 10 days
23. Refresh the OPAC, the link should reappear
24. In the intranet, set the patron to be debarred using a manual restriction
25. In the OPAC refresh the page - self-renewal should no longer be available
26. Remove the restriction - the renewal link will be visible again
27. Set a fine on the patron record of 6 USD
28. Refresh the OPAC, the self-renewal link will be gone because we are over the global ‘noissuescharge’ preference value of 5
29. In the category, set ‘Checkout charge limit’ to 7
30. Refresh the OPAC, the link will be visible because we are now under the category charge
31. Add an additional fine of 2 to the patron account
32. Refresh the OPAC, the self-renewal link is gone
33. In the category, set ‘Self-renewal charge limit’ to be 10
34. Refresh the OPAC, the link will be visible again
35. In the category, set ‘Self-renewal failure message’ to be whatever you want
36. In the OPAC, refresh the page and then click into the self-renewal process
37. This time, click “No” - you should see the failure message you have just set
38. Click “Close” and refresh the page to reset this message
39. In the category, set a message in the “Self-renewal information message”
40. In the OPAC, refresh the page and then click the renewal link again
41. You should be met with the modal, this time displaying the information message we just set up in the patron attributes
42. Click “Yes” - you should see the “Are you sure you want to renew your account?” message again
43. This time when prompted with the “Are you sure” message, click “Yes”
44. The page should refresh and display a confirmation with the new expiry date.
45. In the intranet, view the patron’s record - the date should have been updated
46. In the Modification log tab, there should be a log entry for “Membership renewed”
47. Change the patron’s expiry date again, this time to be yesterday’s date
48. In the OPAC, click on “Personal detail”, the new expiry date should show there
49. Now set the OPACPatronDetails syspref to be “Allow”
50. Click on “Summary” again, the self-renewal link should be there again
51. Click into the self-renewal process and click “Yes” on your verification step
52. This time you should be met with a message that says “Confirm your account details”
53. Click “Continue” - you will be redirected to the OPAC details form
54. There should be a message at the top that says “Please verify your details to proceed with your self-renewal.”
55. Update a field somewhere on the patron form
56. Scroll to the bottom - the button should say “Submit renewal request”
57. Click this button
58. You will be shown the renewal confirmation message
59. In the intranet, on the homepage you should now see a modification request
60. Check the request, it should match the fields you changed - don’t approve or deny for now
61. Set ‘AutoApprovePatronProfileSettings’ in sysprefs to “Enable”
62. Reset your patron’s expiry date to today’s date
63. In the OPAC, click into Personal details and then back into Summary again (we need to do this to remove the success parameter from the URL)
64. The renewal link will be visible again - go through the process again
65. On the form, you should see an alert that you have already sent a modification request
66. This time, change a different field to the previous attempt
67. Submit the form
68. In the intranet there should be no modification this time and the change should be visible in the patron record as it has been automatically approved and handled as per the syspref
69. prove t/db_dependent/Koha/Patron.t
70. prove t/db_dependent/api/v1/patrons_self_renewal.t
71. yarn cypress run --spec t/cypress/integration/Islands/SelfRenewal.ts
72. Sign off!
Signed-off-by: Christopher Brannon <cbrannon@cdalibrary.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This looks trivial at first, but it's not, because of forking and the
comment in C4::Context
I don't think the comment makes sense, we need to retrieve a new dbh,
and so passing the 'new' parameters to C4::Context->dbh should do the
same.
Test plan:
Try rebuild_zebra.pl and confirm there is no error
% perl misc/migration_tools/rebuild_zebra.pl --processes 2 -b
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch adds a new option to define the providers in cronjob erm_run_harvester.pl
that are used to run the SUSHI harvesting for COUNTER Reports in the ERM module.
Use parameter --provider-id or -p
The parameter provider_id is repeatable.
If the parameter for provider_id is not used all active providers will be harvested (as before).
The script will check if these providers are active (as before).
Script to run on ktd: perl misc/cronjobs/erm_run_harvester.pl --begin-date 2025-09-01 --end-date 2025-09-30
To test:
- Run the script and make sure that all active providers are harvested.
- Apply patch.
- Run the script with only one provider-id.
- Run the script with two provider-ids by using -p 1 -p 2
- Run the script without the parameter.
- Make sure that in all cases only active providers are harvested.
- Sign off :-).
Sponsored-by: Karlsruhe Institute of Technology (KIT)
Signed-off-by: Jan Kissig <bibliothek@th-wildau.de>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
- Add missing PrintSlipCSS accordion panel to the notice editor
- Pass AllNoticeCSS and PrintNoticeCSS to gather_print_notices.pl HTML output
- Include inline <style> blocks in print-notices.tt template
Signed-off-by: Marion Durand <marion.durand@biblibre.com>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Test plan (for --cards arguments) :
1. Initial Setup:
a) Go to Tools > Patron card creator > New > Card batch
b) Create two different batches (Batch 1 and Batch 2) using the 'Add by borrowernumber(s):' option.
If using **KTD** you can use the following borrowernumbers for both batches:
15
29
47
else you will need to create your own patrons and enter their borrowernumber.
c) In your terminal, set the timestamps manually:
--OPTION A (KTD users):
ktd --shell
koha-mysql kohadev
--OPTION B:
Access your database
Then:
UPDATE creator_batches SET timestamp = DATE_SUB(NOW(), INTERVAL 30 DAY) WHERE batch_id = '1';
UPDATE creator_batches SET timestamp = DATE_SUB(NOW(), INTERVAL 5 DAY) WHERE batch_id = '2';
Note: your batch_id may differ.
2. Before Applying the Patch:
a) Run the following cleanup script:
./misc/cronjobs/cleanup_database.pl --cards 15 --confirm --verbose
b) Error in console output
c) Nothing is deleted even though one batch was older than 15 days
SELECT * FROM creator_batches;
3. Apply The Patch
4. After Applying The Patch:
a) Redo step 1 (Initial Setup).
b) Run again the following cleanup script:
./misc/cronjobs/cleanup_database.pl --cards 15 --confirm --verbose
c) Check that the console output says: "Purging ... more than 15 days ago"
d) Batch 1 is deleted, but Batch 2 (5 days old) is STILL PRESENT. Success!
SELECT * FROM creator_batches;
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Test plan :
1. Initial Setup:
a) Go to Cataloging > Label creator > New > Label batch
b) Create two different batches (Batch 1 and Batch 2) using the 'Add by barcodes or itemnumbers' option.
If using **KTD** you can use those barcodes for the two batches:
39999000010114
39999000010138
39999000010152
else you will need to create your own records with items and enter their barcodes.
c) In your terminal (in a ktd --shell), set the timestamps manually:
--OPTION A (KTD users):
ktd --shell
koha-mysql kohadev
--OPTION B:
Access your database
Then:
UPDATE creator_batches SET timestamp = DATE_SUB(NOW(), INTERVAL 30 DAY) WHERE batch_id = '1';
UPDATE creator_batches SET timestamp = DATE_SUB(NOW(), INTERVAL 5 DAY) WHERE batch_id = '2';
Note: your batch_id may differ.
2. Before Applying the Patch:
a) Run the following cleanup script:
./misc/cronjobs/cleanup_database.pl --labels 15 --confirm --verbose
b) Check that the console output says: "Purging ... more than 1 days ago"
c) Check database and notice that it is now empty, even though one set was less than 15 days old.
SELECT * FROM creator_batches;
3. Apply The Patch
4. After Applying The Patch:
a) Redo step 1 (Initial Setup).
b) Run again the following cleanup script:
./misc/cronjobs/cleanup_database.pl --labels 15 --confirm --verbose
c) Check that the console output says: "Purging ... more than 15 days ago"
d) Batch 1 is deleted, but Batch 2 (5 days old) is STILL PRESENT. Success!
SELECT * FROM creator_batches;
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
HTML1 is gone. Almost...
The previous rule said that we could not have TT tags within HTML tags
because it caused problems with the translation.
It was incorrect.
Bug 38720 removed that rule, and stated that the only problematic use
was a IF with ">".
It was also incorrect.
This one bug states that we cannot have TT tags to construct HTML tags
IF one of the attribute contains a translatable string.
The list of translatable attributes are listed in tmpl_process3.pl.
This patch restores xt/tt_valid.t (and move it to xt/author) and adjust
slightly the test to take into account this last statement.
Test plan:
0. Apply only this first patch
1. Run xt/author/tt_valid.t
=> It fails on mana-subscription-search-result.inc
2. Update po for your prefered language
=> The string "This resource has been reported more than" is not in the
po files
3. Apply the second patch, repeat 1 and 2
=> It still fails: adding a space between the TT tag and the attribute is not enough here
=> The string is picked now! but... install the template and open the
translated version =>
4. Apply the third patch, repeat 1 and 2
=> The tests pass!
=> The string is still picked, install => The template looks good!
Now if you are familiar with Mana you can try to see the translated
string on the UI
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
xgettext.pl tries to produce output to STDOUT by default, but fails
because of an error in the code. It should go to the actual STDOUT
instead of a file named '&STDOUT'.
Test plan:
1. Run xgettext.pl:
echo koha-tmpl/opac-tmpl/bootstrap/en/includes/serial-status.inc > transfiles
./misc/translator/xgettext.pl -vaf transfiles
2. Observe no standard output, and the existence of a file
called `&STDOUT`. Output to the screen is:
echo koha-tmpl/opac-tmpl/bootstrap/en/includes/serial-status.inc > transfiles
kohadev-koha@kohadevbox:koha(main)$ ./misc/translator/xgettext.pl -vaf transfiles
./misc/translator/xgettext.pl: Outputting to STDOUT...
./misc/translator/xgettext.pl: Opening input file list "transfiles"
./misc/translator/xgettext.pl: Processing file "./koha-tmpl/opac-tmpl/bootstrap/en/includes/serial-status.inc"
3. Apply the patch.
4. Run again.
5. Observe that information is output to the screen (see example of
outpunt in the bug comment).
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Standardizes error code spelling across the codebase:
- Changes 'too_much_oweing' to 'too_much_owing'
- Changes 'auto_too_much_oweing' to 'auto_too_much_owing'
The word "owing" is the correct spelling in English.
Files updated:
- C4/Circulation.pm
- C4/SIP/Sip/MsgType.pm
- circ/renew.pl
- koha-tmpl/intranet-tmpl/prog/en/modules/circ/renew.tt
- koha-tmpl/intranet-tmpl/prog/js/checkouts.js
- koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-reserve.tt
- koha-tmpl/opac-tmpl/bootstrap/en/modules/opac-user.tt
- misc/cronjobs/automatic_renewals.pl
- opac/opac-reserve.pl
- opac/opac-user.pl
- t/db_dependent/Circulation.t
All tests should pass successfully.
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
If PatronSelfRegistration we should ignore (i.e. do nothing) if
--del-exp-selfreg is passed to cleanup_database.pl
It prevents the script to warn that the pref is not set.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
The add_csp_nonces.pl file should use the exceptions in
Koha::Devel::Files.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
tidy.pl will still tidy whitespace around script elements that have
attributes like "nonce"
Signed-off-by: Lari Taskula <lari.taskula@hypernova.fi>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Lari Taskula <lari.taskula@hypernova.fi>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Add misc/devel/add_csp_nonces.pl which automatically adds CSP nonce
attributes to inline <script> tags in Koha templates.
The script:
- Finds all .tt and .inc files in koha-tmpl/
- Adds nonce="[% Koha.CSPNonce | $raw %]" to inline scripts
- Skips external scripts (those with src= attribute)
- Skips scripts that already have nonce= attribute
- Skips third-party library files in lib/ and vendor/
Usage:
perl misc/devel/add_csp_nonces.pl # Dry run
perl misc/devel/add_csp_nonces.pl --apply # Apply changes
This script enables easy backporting of CSP nonce support to older
Koha versions by running it against their template directories.
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Lari Taskula <lari.taskula@hypernova.fi>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Currently the patron homelibrary option for OverdueNoticeFrom will select overdues for a branches triggers
based on the issuing branch
The other two options select the trigger/overdues based on the selection
This patch alters the selection of overdues to match only for the patrons branch
To test:
1 - Apply 41315 and follow the plan there
2 - Apply this bug and run the overdues script, assuming CPL is branch A:
perl misc/cronjobs/overdue_notices.pl --triggered --library CPL --test --nomail -v -v
3 - No overdues should be triggered
4 - Run the script for one of the patrons home libraries, in my case MPL
perl misc/cronjobs/overdue_notices.pl --triggered --library MPL --test --nomail -v -v
5 - Note only overdues for patrons from that branch are triggered
Signed-off-by: Ben Daeuber <bdaeuber@fargolibrary.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Koha is built to prevent sending duplicate notices. Notices with static language ( e.g. "You have one or more holds ready to pick up" ) will easily trigger this filter.
RestrictPatronsWithFailedNotices should ignore these notices because they do not indicate any problem with contacting the patron.
Test Plan:
1) Enable RestrictPatronsWithFailedNotices
2) Enqueue to identical notices for a patron
3) Run process message queue to send one and fail the other as a
duplicate ( alternately, just modify them in the database )
4) Run restrict_patrons_with_failed_notices.pl
5) Note the restriction
6) Remove the restriction
7) Apply this patch
8) Run restrict_patrons_with_failed_notices.pl
9) Note that no restriction was created!
Signed-off-by: Marion Durand <marion.durand@biblibre.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Peter Moore <peter@mrcpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
To test:
1 - configure StoreLastBorrower but not AnonymousPatron
2 - perl misc/cronjobs/anonymize_last_borrowers.pl -c
3 - the cron tells 'Preference 'AnonymousPatron' must be enabled to anonymize item's last borrower'
4 - put a value in AnonymousPatron
5 - repeat 2
6 - cron runs good
Signed-off-by: Peter Moore <peter@mrcpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
To test the confirm flag:
1. Run perl misc/cronjobs/batch_anonymise.pl --days 1
2. Notice the warning about needing a confirm flag
3. Run perl misc/cronjobs/batch_anonymise.pl --days 1 --confirm
4. The script runs
Signed-off-by: Peter Moore <peter@mrcpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch adds the feature to anonymize patrons from items_last_borrower table. This code is run from batch_anonymize script, which is triggered from a new cron job anonymize_last_borrowers.pl
To test:
1) apply this patch and the previous ones
2) perl installer/data/mysql/updatedatabase.pl
3) set StoreLastBorrower preference to Allow.
4) set AnonymousPatron preference to a valid patron id
5) Create a Check out followed by a Check in.
CHECK => a row should appear in items_last_borrower table with the borrower and the item number.
6) In mysql, update created_on of items_last_borrower and returndate of old_issues to two days earlier.
7) perl misc/cronjobs/anonymize_last_borrowers.pl
CHECK => borrower number in items_last_borrower is not anonymized yet.
8) Repeat step 5 and 6 but instead of two days earlier, set the new entry in items_last_borrower to three days earlier.
9) set AnonymizeLastBorrower to 'Anonymize' and AnonymizeLastBorrowerDays to 2 days.
10) perl misc/cronjobs/anonymize_last_borrowers.pl
SUCCESS => borrower number in items_last_borrower changed to AnonymousPatron id
11) repeat step 8
12) set AnonymousPatron preference to 0
13) perl misc/cronjobs/anonymize_last_borrowers.pl
SUCCESS => borrower number in items_last_borrower changed to null
14) prove t/db_dependent/Koha/Patrons.t
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Peter Moore <peter@mrcpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch ensures that the accountlines.branchcode field correctly records
which branch's circulation rules were applied when creating OVERDUE, LOST,
and PROCESSING fee debits. Previously, this field was often NULL or set
to the logged-in user's branch, making it difficult to track which branch's
policies governed the fee calculation.
Changes made:
1. New helper method: Koha::Checkout->branch_for_fee_context()
- Centralizes branch resolution logic based on system preferences
- For LOST/PROCESSING: honors LostChargesControl preference
- For OVERDUE: honors CircControl preference
- Respects HomeOrHoldingBranch when choosing item's branch
- Supports both Koha objects and plain hashrefs
- Returns branchcode or undef if context unavailable
2. C4::Accounts::chargelostitem()
- Now accepts optional $opts hashref parameter
- Accepts interface, library_id, and issue_id in options
- Determines appropriate branch using branch_for_fee_context
- Passes resolved branch to add_debit as library_id
- Issue_id lookup handles bundled items correctly
3. C4::Circulation::LostItem()
- Calls branch_for_fee_context before chargelostitem
- Passes interface context ('cron' for cronjob, otherwise current)
- Passes resolved branch and issue_id to chargelostitem
4. C4::Overdues::UpdateFine()
- Calls branch_for_fee_context to determine OVERDUE branch
- Passes result as library_id to add_debit
- Resolves long-standing FIXME about branch selection
5. misc/cronjobs/longoverdue.pl
- Determines branch context before marking items lost
- Passes library_id to LostItem when charging fees
The branch resolution logic respects these system preferences:
- CircControl: PatronLibrary | PickupLibrary | ItemHomeLibrary
- LostChargesControl: PatronLibrary | PickupLibrary | ItemHomeLibrary
- HomeOrHoldingBranch: homebranch | holdingbranch
Test plan:
1. Create three test libraries (e.g., FPL, CPL, PVL)
2. Create a test patron with home library PVL
3. Create an item with:
- homebranch = FPL
- holdingbranch = CPL
4. Set system preferences:
- FinesMode: Calculate and charge
- Create circulation rule that charges overdue fines
5. Test OVERDUE fees with CircControl:
a. Set CircControl = PatronLibrary, HomeOrHoldingBranch = homebranch
b. Check out item to patron at CPL, backdate to be overdue
c. Run: perl /usr/share/koha/bin/cronjobs/fines.pl
d. Check accountlines: SELECT accountlines_id, branchcode, debit_type_code
FROM accountlines ORDER BY accountlines_id DESC LIMIT 5;
e. Verify branchcode = PVL (patron's home library)
f. Delete the fine, repeat with CircControl = PickupLibrary
g. Verify branchcode = CPL (checkout library)
h. Delete the fine, repeat with CircControl = ItemHomeLibrary
i. Verify branchcode = FPL (item's home library)
j. Change HomeOrHoldingBranch = holdingbranch, repeat
k. Verify branchcode = CPL (item's holding library)
6. Test LOST fees with LostChargesControl:
a. Set WhenLostChargeReplacementFee = Charge
b. Set item replacement price > 0
c. Ensure item is checked out to patron
d. Set LostChargesControl = PatronLibrary
e. Mark item as lost in staff interface
f. Check accountlines for LOST debit
g. Verify branchcode = PVL (patron's home library)
h. Return item, re-checkout, repeat with LostChargesControl = PickupLibrary
i. Verify branchcode = CPL (checkout library)
j. Repeat with LostChargesControl = ItemHomeLibrary + homebranch
k. Verify branchcode = FPL (item's home)
l. Repeat with LostChargesControl = ItemHomeLibrary + holdingbranch
m. Verify branchcode = CPL (item's holding)
7. Test longoverdue cronjob:
a. Re-checkout item with due date triggering longoverdue
b. Run: perl /usr/share/koha/bin/cronjobs/longoverdue.pl --lost 30=1 --charge 1 --confirm --verbose
c. Verify LOST debit created with correct branchcode per LostChargesControl
8. Test edge cases:
- WhenLostChargeReplacementFee = Don't charge → no LOST debit
- Replacement price = 0 → no LOST debit
- FinesMode = Don't calculate → no OVERDUE debits
Signed-off-by: Trevor Diamond <trevor.diamond@mainlib.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Interestingly this script already had a $surname variable, just never accepted a parameter for it.
This enhancement enables the user to supply a surname parameter when creating a superlibrarian user via the commandline create_superlibrarian.pl script. Koha patrons MUST have a name, so if no surname is provided, the userid will be used for the surname instead.
To test:
1. Apply the patch and restart services
2. Attempt to create a superlibrarian with no surname provided.
In KTD: misc/devel/create_superlibrarian.pl --userid test --cardnumber test --password XXXX --branchcode CPL --categorycode S
3. Search for your patron in Koha. Confirm the surname was automatically set to the userid 'test'
4. Attempt to create a superlibrarian, this time provide a surname that is different from the userid so easy to tell apart.
In KTD: misc/devel/create_superlibrarian.pl --userid test1 --cardnumber test1 --password XXXX --branchcode CPL --categorycode S --surname testsurname
5. Search for your patron in Koha. Confirm the surname was set correctly using the given surname.
Sponsored-by: Catalyst IT
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch additionally fetches the reply to address for advance notices
To test:
1 - Have some issues that will be due soon for a patron(s)
Checkout with the due date set to tomorrow
2 - Verify the patron(s) have an email defined, verify the issuing branch has a reply to set
3 - Verify the patron(s) have messaging preference set to receive Advance notices by email 1 day in advance
4 - perl misc/cronjobs/advance_notices.pl -v -c
5 - Check the message queue
SELECT * FROM message_queue WHERE borrowernumber={##}
6 - Note no rpely to is set
7 - Apply patch
8 - perl misc/cronjobs/advance_notices.pl -v -c
9 - Check the message queue
10 - Confirm newer message has the reply to populated
11 - Ensure the issued item has a different homebranch with a reply to address se
12 - perl misc/cronjobs/advance_notices.pl -v -c --frombranch item-homebranch
13 - Confirm message has correct reply to
14 - Set patron to want digest in messaging preferences
15 - Generate notices
16 - Confirm digets has the right reply to
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Tidied inline
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Tomás Cohen Arazi
Katrin Fischer
John Doe
Lari Strand
Jan Kissig
Hammat Wele
Nick Clemens
Jonathan Druart
Olivia Reynolds
Martin Renvoize
Baptiste Wojtkowski
Fridolin Somers
Lucas Gass
Matt Blenkinsop
Raphael Straub
Emily Lamancusa
Ayoub Glizi-Vicioso
Victor Grousset/tuxayo
Tadeusz „tadzik” Sośnierz
David Cook
Kyle M Hall
Andrew Fuerste Henry
Laura_Escamilla
Aleisha Amohia
Marcel de Rooy