From https://css-tricks.com/snippets/css/make-pre-text-wrap/
/* Browser specific (not valid) styles to make preformatted text wrap */
pre {
white-space: pre-wrap; /* css-3 */
white-space: -moz-pre-wrap; /* Mozilla, since 1999 */
white-space: -pre-wrap; /* Opera 4-6 */
white-space: -o-pre-wrap; /* Opera 7 */
word-wrap: break-word; /* Internet Explorer 5.5+ */
}
... I think it's now safe to remove the backward compatibility versions and simply keep pre-wrap.
Test plan (from bug 921)0:
* place a hold on a title with a very very long title or author
* mark it as "waiting for pickup" by doing a checkin
* go to command line interface, and run
* misc/cronjobs/gather_print_notices.pl /tmp
* misc/cronjobs/printoverdues.sh /tmp
* if your title/author is long enough, it's splitted on 2 lines
with this patch (it is not before this patch)
It will also remove the warnings from t/db_dependent/Patron/Borrower_Discharge.t
WARNING: Ignored `white-space: -moz-pre-wrap` at 6:14, invalid value.
WARNING: Ignored `white-space: -o-pre-wrap` at 7:14, invalid value.
WARNING: Ignored `word-wrap: break-work` at 8:14, invalid value.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 03cf21fd92)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
With this patch we adjust what's needed to make misc/devel/tidy.pl run
identically as before
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit e7e42c5e88)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
This patch introduces a new Koha::Devel namespace. It clearly shows
that its purpose is for development and will not contain any business
logic.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 1d4071298c)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 8f90ac9d4e)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
For creating this patch I looked at all parameters defined in the
handler mapping, removed duplicated ones and tracked down which ones
really were parameters in `GetOptions`. This included tracking down
variables that were later assigned other values and things like that.
The mapping is used to display the right parameter name if missing a
mandatory one.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 46cc88acc3)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Bug 31143 created a script in cronjob and then moved it to maintenance. However, it is still referred as in cronjob in help strings. This patch updates the hint.
No test needed I guess. Only changing a string
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 34886a9b5b)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
In order to remove false positives with the QA script we should replace some tabs with spaces.
There are more but I wanted to make sure no regression would be
introduced
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 492e6814a1)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
This patch corrects a few spelling mistakes and missing grammer and adds
a little more detail to the '--date', '--email' and '--frombranch'
options.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit f6fd9f2e90)
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Although we're reverting 32216, we still want to be able to force print notices as per the original intent.
This patch adds that into queue_notices, rather than in the cronjob
Co-authored-by: Jake Deery <jake.deery@openfifth.co.uk>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This reverts commit 3040f46d5e.
Bug 30300 adds a patron messaging preference for the expiration notice and abstracts this logic into queue_notice
Bug 32216 removes this call to enable print notice functionality but this would be handled already by queue_notice based on the patron's messaging preferences
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
The patches on this bug add the ability for staff members to set the patron expiry notice as mandatory at the category level. When set, the cronjob will automatically send an expiry notice regardless of existing messaging preferences. If it is turned off, the old messaging preferences will still be in place and actioned
Test plan:
1) Apply patch and run updatedatabase
2) In system preferences set MembershipExpiryDaysNotice to 5
3) Choose a patron and edit their expiry date to today's date, ensuring they have a primary email set
4) Create a MEMBERSHIP_EXPIRY notice (content of the notice is irrelevant, just needs to exist)
5) Run perl misc/cronjobs/membership_expiry.pl -c -v -before 10 -after 10
6) Check the patron's messages, there should be no expiry notice queued
7) Set the patron's category 'enforce expiry notice' field to Yes
8) Repeat step 5, there should be a notice queued
9) Set that patron's primary contact method to primary email and run step 5 again
10) There should be a notice queued for an email
11) Set the category level enforce expiry notice back to No
12) Choose a second patron and set an email address, expiry date to today and set the patron expiry messaging preference for that patron to 'email'
13) Repeat step 5 - this time only one notice should be queued on the second patron, no new notice should queue for the first
14) Re-set the category level option to Yes
15) Repeat step 5 - this time both patron's should have notices queued
16) Unset the category level option again, the behaviour should be the same as step 13
17) Set the category level option back to Yes and check the patron messaign preferences menus in both the OPAC and Staff client - the patron expiry option should not be visible while the cateogry level enforcement is set to Yes
Signed-off-by: Hanna Leiker <tech.hanna@salinapublic.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
->connect does not raise an exception, so we didn't reached the catch
block.
Adjusting the code but I am not really happy with it, but more changes
will derail the original bug
Now we log:
Cannot connect to broker (Access refused for user 'guest') at /kohadevbox/koha/Koha/BackgroundJob.pm line 96.
Cannot connect to the message broker, the jobs will be processed anyway at /kohadevbox/koha/misc/workers/background_jobs_worker.pl line 100.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This change adds error handling for ERROR frames to background_jobs_worker.pl.
A previous patch added it to es_indexer_daemon.pl which is a different
background jobs worker.
To test ERROR handling for background_jobs_worker.pl:
1. Change your RabbitMQ connection details in koha-conf.xml or the hard-coded ones in Koha/BackgroundJob.pm
to include an invalid password. For example, if you don't have a message_broker already defined, use the following:
<message_broker>
<password>bad</password>
</message_broker>
2. sudo koha-worker --restart --quiet kohadev
3. View the errors in the logs:
tail -f /var/log/koha/kohadev/worker-output.log
tail -f /var/log/koha/kohadev/worker-error.log
To test MESSAGE handling for background_jobs_worker.pl:
1. Ensure RabbitMQ connection details are correct
2. Go to Batch item modification and add the following barcodes:
39999000001310
39999000004571
3. Add a z - Public Note of the following and click Save:
THEBACKGROUNDWORKERWORKED
4. Search the catalogue using THEBACKGROUNDWORKERWORKED in the top
catalogue search
5. Note that you get 2 results in your search
6. Look in the logs just to make sure there are no new messages there
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This change causes es_indexer_daemon.pl to die on ERROR frames instead
of spinning in an infinite loop. It also adds a content-type STOMP frame header
to help consumers know what data type is in the MESSAGE frame body.
To test ERROR handling for es_indexer_daemon.pl:
1. Switch to using Elasticsearch
2. Change your RabbitMQ connection details in koha-conf.xml or the hard-coded ones in Koha/BackgroundJob.pm
to include an invalid password. For example, if you don't have a message_broker already defined, use the following:
<message_broker>
<password>bad</password>
</message_broker>
3. sudo koha-es-indexer --restart kohadev
4. View the errors in the logs:
tail -f /var/log/koha/kohadev/es-indexer-output.log
tail -f /var/log/koha/kohadev/es-indexer-error.log
To test MESSAGE handling for es_indexer_daemon.pl:
1. Switch to using Elasticsearch (and ensure RabbitMQ connection details are correct)
2. Change the title of a bib record to something new and unique
3. Try to search for the bib using the new title
4. Note you can find it
5. Look in the logs just to make sure there are no messages there
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Bug 27486 renamed system preference "delimiter" into "CSVDelimiter"
This patch just replaces the commented code with a TODO text
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
The cronjob debar_patrons_with_fines.pl is meant to add a restriction based on a fines threshold.
The problem is that the script adds a MANUAL restriction, which makes no sense. MANUAL restrictions are meant to be manually created, never automatic as this script does.
In addition, they are repeatable, not unique. Every time this script runs, it will add an additional restriction.
There is already a restriction type meant for this type of use case: SUSPENSION
We should change this script to create SUSPENSION restrictions instead of MANUAL restrictions.
In summary, this script should use the unique restriction type SUSPENSION to prevent multiple additional restrictions from being created on each run.
Test Plan:
1) Set up a patron with fines
2) Run misc/cronjobs/debar_patrons_with_fines.pl -c -m "Test message"
3) Note the restriction is created
4) Run misc/cronjobs/debar_patrons_with_fines.pl -c -m "Test message" again
5) Note another restriction is created
6) Delete the restrictions
7) Apply this patch
8) Run misc/cronjobs/debar_patrons_with_fines.pl -c -m "Test message"
9) Note the restriction is created
10) Run misc/cronjobs/debar_patrons_with_fines.pl -c -m "Test message" again
11) Note another restriction is *not* created!
Signed-off-by: Magnus Enger <magnus@libriotech.no>
Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds a new test to the data inconsistencies script to report where
MARC records contain item fields.
These may not cause issues in Koha, however, in discovery harvests they have eben found to create errors
To test:
1 - Follow test plan on bug 39412 to generate records with item tags
2 - perl misc/maintenance/search_for_data_inconsistencies.pl
3 - Note new section and hint on how to fix
4 - perl misc/maintenance/touch_all_biblios.pl
5 - repeat 2
6 - Errors cleared
Signed-off-by: Magnus Enger <magnus@libriotech.no>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch makes the indexer poll the DB in configurable batches. Pretty
much like Rabbit does.
The only change is that it will now pass a LIMIT <batch_size> to the
query.
To test:
0. Have KTD started with ES (skip if you have an instance running):
$ ktd --es7 up -d
1. Stop the indexer and rabbit:
$ ktd --shell
k$ sudo koha-es-indexer --stop kohadev
k$ sudo service rabbitmq-server stop
2. On a separate terminal, watch the queue:
k$ watch -n 0.5 'echo "SELECT COUNT(*) FROM background_jobs WHERE status=\"new\"" | koha-mysql kohadev'
=> SUCCESS: It should be empty
3. Add a bunch of things to index (this works with default KTD data):
k$ for i in $(seq 1 438; seq 1 438) ; do echo "INSERT INTO background_jobs (status,queue,data) VALUES ('new','elastic_index','{\"record_server\":\"biblioserver\",\"record_ids\":[$i]}')" | koha-mysql kohadev ; done
4. Start the ES indexer:
k$ sudo koha-es-indexer --start kohadev
5. Look at the queue size
=> FAIL: It shrinks too fast [1]
6. Apply this patch
7. Restart the indexer
k$ perl misc/workers/es_indexer_daemon.pl &
8. Repeat 3
9. Look at the queue size:
=> SUCCESS: It shrinks 'slowly'
10. Sign off :-D
[1] This is a problem in certain scenarios, not in all of them. This is
the best I come up with in order to test the behavior change. Anyone can
set a new size of their taste.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
If using ES, the script will collect the newly created records and biblionumbers in atwo arrays (@search_engine_record, @search_engine_record_ids) and pass that to the indexer when $commitnum records are read. The arrays are cleaned afterwards.
BUT: The records and biblionumbers were always stored, even if we're not using Elasticsearch. And as the cleaning of those arrays only happens if we are in fact using ES, all the data is added but never cleaned.
This patch fixes this by only storing record and biblionumber if we have an $indexer (i.e. if we're using Elasticsearch).
I have no idea though how/if the records are indexed when using Zebra.
After applying this patch and importing a large enough file without using Elasticsearch, the RAM usage also stays more or less constant.
@Jan, please verify!
Signed-off-by: Jan Kissig <bibliothek@th-wildau.de>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
So, I reviewed the code, which has a bunch of issues:
* there's a call to `next RECORD`, but no `RECORD` loop label.
* $record_number is incremented twice
* and the calls to txn_begin is moved from the outside the loop into each record, starting one transaction per record. Not sure how mysql handles this, but postgres would not like it.
This patch fixes those issues:
After I fixed the first two issues, I ran the patched script on a file containing 75 authorities (from GND), and not one was added to the DB, presuambly because nested transactions don't work in mysql?
So I also moved the BEGIN transaction outside of the main loop and started a new one after each commit, and added a final commit after to loop.
Signed-off-by: Jan Kissig <bibliothek@th-wildau.de>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Include checking for invalid marc data in main loop to avoid
loading all marc data into memory at once.
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
connexion is a product, added to ignore file
tablee -> tables
a few other small corrections
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Add the option "--without_restriction_type" so that delete_patrons.pl
does not delete patrons with given restriction types.
To test:
1) Run
misc/cronjobs/delete_patrons.pl --not_borrowed_since 2010-01-01
and choose a borrowernumber that would be deleted.
2) In your browser go to
/cgi-bin/koha/members/moremember.pl?borrowernumber=<borrowernumber>
and add a manual restriction to this patron.
3) Apply the patch.
4) Run
misc/cronjobs/delete_patrons.pl --not_borrowed_since 2010-01-01 \
--without_restriction_type MANUAL
5) Check that the patron with the manual restriction would not be
deleted.
Sponsored-by: Karlsruhe Institute of Technology (KIT)
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch changes the script to use 'Koha::PseudonymizedTransaction->new_from_statistic' directly
rather than calling 'Koha::Statistic->pseudonymize()' which generates a background job.
Additionally, some of the checks from 'pseudonymnize' and the background job are moved into the search
so we don't fetch any to pseudonymize that should not be.
To test:
1 - Generate some entries in statistics (checkin, checkout, renew, etc)
2 - Enable pseudonymization (see bug 28911 if you need bcrypt settings)
3 - perl misc/maintenance/pseudonymize_statistics.pl -v -c
4 - Statistics pseudonymized
5 - Check the background jobs table, note jobs were generated
6 - Apply patch
7 - perl misc/maintenance/pseudonymize_statistics.pl -v -c
8 - You get same count as before, and are warned of existing pseudonymized transactions
9 - enter 'Y' to continue
10 - Check the pseudonymized_transacations table
11 - Confirm same rows were generated twice
12 - Check the background jobs
13 - Confirm there were no new jobs generated (for pseudonymization)
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds a new option switch `--fix` (or `-f`) to make the script
try to fix the problems it found.
Bonus: `--help` option switch added.
To test:
1. Apply this patch
2. Run:
$ ktd --shell
k$ perl -MKoha::SearchEngine -MKoha::SearchEngine::Indexer -e 'my $i = Koha::SearchEngine::Indexer->new({index => $Koha::SearchEngine::BIBLIOS_INDEX}); $i->delete_index([1]);'
=> SUCCESS: Bibliographic record #deleted from ES index
3. Run:
k$ koha-mysql kohadev
> DELETE FROM biblio WHERE biblionumber=2 OR biblionumber=3;
=> SUCCESS: Bibliographic records 2 and 3 deleted directly from the
database, no reindex triggered by Koha.
4. Run:
k$ perl misc/maintenance/compare_es_to_db.pl
=> SUCCESS: The results from (2) and (3) are explained. [1]
5. Run:
k$ perl misc/maintenance/compare_es_to_db.pl --fix
=> SUCCESS: It says it is gonna fix things
6. Repeat 4
=> SUCCESS: The problems we generated are no longer reported! They've
been fixed!
7. Sign off :-D
[1] Note KTD ships some broken records on purpose so devs need to deal
with a not-perfect DB everyday, and catch misses in the code.
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
In ktd context:
if host's node_modules ktd has been generated prior to bug 38644, the template-toolkit's
prettier plugin is not there and prettier will empty the file.
It seems to be a prettier bug, and the following behaviour is confusing:
From the host:
% yarn run prettier --write koha-tmpl/intranet-tmpl/prog/en/modules/test.tt
yarn run v1.22.22
$ prettier koha-tmpl/intranet-tmpl/prog/en/modules/test.tt
[error] Cannot find module '@koha-community/prettier-plugin-template-toolkit'
[error] Require stack:
[error] - /home/jonathan/workspace/koha/node_modules/prettier/index.js
[error] - /home/jonathan/workspace/koha/node_modules/prettier/cli.js
[error] - /home/jonathan/workspace/koha/node_modules/prettier/bin-prettier.js
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
=> The file is not modified
Within ktd:
$ yarn run prettier --write koha-tmpl/intranet-tmpl/prog/en/modules/test.tt
yarn run v1.22.22
$ prettier --write koha-tmpl/intranet-tmpl/prog/en/modules/test.tt
koha-tmpl/intranet-tmpl/prog/en/modules/test.tt 6ms
Done in 0.39s.
=> No error, and the file is emptied!
It seems that the plugin is found by prettier but not used (?)
Test plan:
1. Follow this from step 1 to 14
https://gitlab.com/koha-community/koha-testing-docker/-/issues/483
2. Apply patch
3. misc/devel/tidy.pl --no-write koha-tmpl/intranet-tmpl/prog/en/modules/intranet-main.tt
It should print the new error message
4. Make a change to a .tt file, commit
5. There will be a message about and issue with prettier. But it will
still commit. It's expect.
6. Inspect and see that the commit has the changes
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds the same logging as other cronjobs have
To test:
1 - Enable system preference CronjobLog
2 - perl misc/cronjobs/writeoff_debts.pl -v -c --added-after 2025-01-01
3 - Check the logs - no entries
4 - Apply patch
5 - Repeat 2
6 - Check the logs, two entries - one for the start and one for the end
Signed-off-by: Magnus Enger <magnus@libriotech.no>
Tested on top of bug 39236. Works as advertised.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
From bug 22435 - we need to use 'CREATE' as the account_offset type
From bug 31095 - we need to 'use Koha::Patrons;'
To test:
1 - Add a fine to a patron account
2 - perl misc/cronjobs/writeoff_debts.pl -v -c --added-after 2025-01-01
3 - Cronjob dies
4 - Apply patch
5 - Rerun
6 - Job succeeds!
7 - Confirm patron's fine is written off
8 - Sign off
Signed-off-by: Magnus Enger <magnus@libriotech.no>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
The first part of the Zebra rebuild is the exporting. This part is
made faster. The second part with zebraidx is not changed.
A new commandline parameter -forks is added to the rebuild_zebra.pl
script. A subroutine export_marc_records is added between index_records
and export_marc_records_from_sth. The last routine has a new parameter,
the sequence number of the export file.
NOTE: This report does not touch koha-rebuild-zebra yet! This will be
done on a follow-up.
Test plan:
Note that the number of forks/records below can be adjusted
according to your server and database setup.
[1] Reindex a subset of 100 records without forks:
su [YOUR_KOHA_USER]
misc/migration_tools/rebuild_zebra.pl -a -b -r -d /tmp/rebuild01 -k --length 100
Check if /tmp/rebuild01/biblio contains one export file for auth/bib.
Verify that max. 100 auth and bib were indexed (check Auth search, Cataloguing)
[2] Reindex an additional subset of 100 recs with forks (remove -r, add -forks):
su [YOUR_KOHA_USER]
misc/migration_tools/rebuild_zebra.pl -a -b -d /tmp/rebuild02 -k --length 100 --offset 100 -forks 3
Check if /tmp/rebuild02/biblio contains 3 export files for auth/bib.
Verify that max. 200 auth and bib were indexed (check Auth search, Cataloguing)
[3] Run a full reindex with forks:
su [YOUR_KOHA_USER]
misc/migration_tools/rebuild_zebra.pl -a -b -d /tmp/rebuild03 -k -forks 3
Check both searches again.
[4] Bonus: To get a feeling of improved speed, reindex a larger production db with and
without using -forks. (Use something like above.) You may add -I to skip indexing
in order to better compare both exports.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Reindexed a prod db in 96 mins instead of 150 mins (3 forks, 4 cores). Main gain in
biblio export; complete export took 35 mins, zebraidx 61 mins.
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
It will be a commented line in the debian template and added also
in the crontab.example file.
As mentioned on BZ, the comment can easily be removed automatically
during Docker builds or startups etc.
Test plan:
Just read the patch. Nothing changes here actually.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds print notice functionality to the membership_expiry.pl script,
by adding both conditional and and forced modes. When passing only --confirm,
the script will check the patron's record for an email address. If one is
found, an email notice is generated. If one is not, a print notice is used
instead. By adding a -p flag, print notices will be generated, even if an
email address is present.
To test:
a) set the MembershipExpiryDaysNotice syspref to 7 days
b) choose or set up half a dozen patrons with expiry dates
1) take heed: these expiry dates will need to be T+7 days
c) add fake email addresses to two of the patrons
1) take heed: the other patrons should have no primary or secondary
email
d) set up the MEMBERSHIP_EXPIRY notice
a) set content & subject in the email notice to contain the word
email
b) set content & subject in the print notice to contain the word
print
e) run ./misc/cronjobs/membership_expiry.pl -n -c
1) notice how the printed notices contain the word email
f) run ./misc/cronjobs/membership_expiry.pl -c
g) check the notices for the patrons from step b
1) notice how only email notices are generated
h) APPLY PATCH
i) repeat step e
1) notice how a mix of email and print notices are now generated
j) run ./misc/cronjobs/membership_expiry.pl -n -p -c
1) notice how the output is the same as in step i - this is by
design
k) repeat steps f-g
1) notice how some patrons will only get an email notice, whilst
others will only get a print notice
2) this should occur based on which patrons do, or do not have a
valid primary or secondary email
l) run ./misc/cronjobs/membership_expiry.pl -p -c
m) repeat step g
1) notice that all patrons now get a print notice, whilst some get
both a print notice, and an email notice. Again, this should
occur based on whether the patron has a valid primary or
secondary email
n) SIGN OFF
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Cleanup_database can delete all statistics entries more than X days
old. If one is using pseudonymization to create pseudonymized_transactions
data, then one may wish to use that cleanup_database function to delete
statistics entries that have been duplicated in pseudonymized_transactions.
However, not all types of transactions in statistics are duplicated in
pseudonymized transactions.
Pseudonymized_transactions currently only includes checkouts, returns,
and renewals.
This patch adds two additional parameters to cleanup_database.pl:
1. --statistics-type
Defines the types of statistics to purge. Will purge all types if
parameter is omitted. Repeatable.
2. --statistics-type-pseudo
Grabs values from @Koha::Statistic::pseudonymization_types. At the
time of writing this patch, they are:
renew, issue, return and onsite_checkout
To test:
1. prove t/db_dependent/Koha/Statistic.t
2. Before applying this patch:
3. Create some statistics entry by checking out, renewing and checking in
items.
4. perl misc/cronjobs/cleanup_database.pl --statistics 1 --verbose
Observe:
Purging statistics older than 1 days
n statistics would have been removed
Where n is the amount of statistics rows matching your test environment
5. Apply patch
6. Repeat step 4, observe same result
7. perl misc/cronjobs/cleanup_database.pl --statistics 1 --statistics-type-pseudo --verbose
Observe:
Purging statistics older than 1 days with types "onsite_checkout,renew,return,issue".
n statistics would have been removed
Where n is the amount of statistics rows matching your test environment
(order of listed types does not matter)
8. perl misc/cronjobs/cleanup_database.pl --statistics 1 --statistics-type test1 --statistics-type test2 --verbose
Observe:
Purging statistics older than 1 days with types "test1,test2".
n statistics would have been removed
Where n is the amount of statistics rows matching your test environment
(order of listed types does not matter)
9. perl misc/cronjobs/cleanup_database.pl --statistics 1 --statistics-type-pseudo --statistics-type test1 --verbose
Observe:
Purging statistics older than 1 days with types "test1,onsite_checkout,renew,return,issue".
n statistics would have been removed
Where n is the amount of statistics rows matching your test environment
(order of listed types does not matter)
10. Try previous cleanup_database.pl commands with --confirm flag and make sure correct rows are purged
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
If no patron categories require overdue notices, then there will be no overdue rules/status triggers. This isn't necessarily a bad thing and shouldn't be noisy.
To test:
1. Go to Administration -> Patron categories
2. Set all your category types to have overdue notice required: No
3. Go to Tools -> Overdue notices/status triggers
4. Notice you're unable to set any overdue rules
5. Run the cronjob misc/cronjobs/overdue_notices.pl and notice the error about no overdue rules is printed
6. Apply the patch and restart services
7. Run the cronjob misc/cronjobs/overdue_notices.pl and notice the error is not printed
8. Run the cronjob misc/cronjobs/overdue_notices.pl --verbose and notice the error is printed
Sponsored-by: Catalyst IT
Signed-off-by: Imani Thomas <imani.thomas@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Since we have misc/export_records.pl and we want to remove
auth_header.marc, the ancient script misc/exportauth.pl can be removed
as well (it is relied upon auth_header.marc).
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch makes the script provide links to the staff interface instead
of the OPAC. This way, the links can just be picked to access the
problematic records for fixing them.
To test:
1. Run:
$ ktd --shell
k$ perl misc/maintenance/compare_es_to_db.pl
=> FAIL: Generated links point to the OPAC
2. Apply this patch
3. Repeat 1
=> SUCCESS: Generated links are useful as they point to the staff
interface
4. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
It is never used
% git grep string_list
should not return anything
Signed-off-by: Thibault Keromnes <thibault.keromnes@univ-paris8.fr>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Paul Derscheid
Jonathan Druart
Tomas Cohen Arazi
Baptiste Wojtkowski
Martin Renvoize
Katrin Fischer
Matt Blenkinsop
Matt Blenkinsop
David Cook
Fridolin Somers
Kyle M Hall
Nick Clemens
Thomas Klausner
David Gustafsson
Raphael Straub
Emily Lamancusa
Marcel de Rooy
PerplexedTheta
Lari Taskula
Aleisha Amohia
Janusz Kaczmarek