This patch updates the EDI code to use the Koha::Logger wrapper in
preference to the bare Log::Log4perl module.
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Improve messages clarity.
Show success message if no schema diff was found from audit.
Code refactor to ensure the title+message shown in UI is the same as the one shown in CLI and are all translatable.
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch updates the module as suggested by David to handle just the
diff code and leaves the display logic up to the templates and script.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: JesseM <jesse@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Test plan #1 ensure same script functionality (same as bug 34064):
0. Apply patch
1. vi ./installer/data/mysql/kohastructure.sql
2. Comment out some columns, change NULL status, or whatever you like
3. perl misc/maintenance/audit_database.pl \
--filename /kohadevbox/koha/installer/data/mysql/kohastructure.sql
4. Note that the output includes SQL commands to change the database
to match the new kohastructure.sql
5a. Try using koha-foreach and note that the database name appears above
the database comparison
5b. koha-foreach "perl misc/maintenance/audit_database.pl \
--filename /kohadevbox/koha/installer/data/mysql/kohastructure.sql"
Test plan #2:
1. After doing test plan #1, access the new 'database audit' tab, visit:
<staff_url>/cgi-bin/koha/about.pl?tab=database
2. Ensure the output is the same as test plan #1
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: JesseM <jesse@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch simply generates a new user agent when reauthenticating when the CSRF token
for the session has expired.
To test:
1 - Edit Koha/Token.pm, line 275
from: { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS * 3600 ) },
to: { MaxAge => $params->{MaxAge} // ( CSRF_EXPIRY_HOURS ) },
2 - This will make your CSRF tokens expire after 8 seconds
3 - vim /etc/koha/sites/kohadev/connexion.cnf
4 - Set content:
host:
port: 8888
koha:http://localhost:8081
log:/var/log/koha/kohadev/connexion.log
match:ISBN
user:kohauser
password:kohapass
overlay_action:replace
nomatch_action:create_new
item_action:always_add
import_mode:direct
framework:BKS
overlay_framework:
debug:1
5 - Save the sample file from this bug into your kohaclone (or copy and paste into a file your koha test site can reach)
6 - On the command line:
perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf
7 - In another terminal:
cat bug_33418.test | nc -v localhost 8888
8 - Successful request
9 - Wait 10 seconds (more than 8 anyways)
10 - Repeat 7
11 - Unsuccessful request - Invalid CSRF token
12 - Ctrl+C to stop the connexion daemon
13 - Apply patch
14 - repeat 6
15 - repeat 7, success
16 - Wait more than 8 seconds
17 - repeat 7, success!
18 - repeat again until you are satisfied
19 - Sign off!
WNC amended and tidied
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Default to KohaAdminEmailAddress if ReplytoDefault is not set
Test plan:
1. Enter a valid email address into ReplytoDefault and
KohaAdminEmailAddress sysprefs
2. Apply patch and restart services
3. Generate a file on your Koha server. Place it in /tmp/test.mrc
4. Manually connect to the SFTP remote host and accept the hostkey
5. Enter Koha shell
6. Run the sftp_file.pl command-line script
./sftp_file.pl --host <your remote hostname> --user <your user> --pass <your password> --upload_dir <your upload directory> --file /tmp/test.mrc --email <your email>
7. Confirm the SFTP was successful
8. Confirm that the email that you received has a from address matching the
ReplytoDefault syspref
9. Empty out the ReplytoDefault syspref and repeat steps 5 and 6
10. Confirm that the new email you received has a from address matching
the KohaAdminEmailAddress syspref
Sponsored-by: Catalyst IT
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch skips record indexing and real time holds queue updates when updating the localuse field
from statistics. A note is added to the script that the user should reindex if the localuse field is mapped.
Additionally a verbose option is added to the script, and doubled use of GetOptions is removed.
Lastly, a check is added to confirm the items value is being changed before the value is stored.
To test:
1 - Enable the real time holds queue
2 - Enable Elasticsearch
3 - perl misc/maintenance/update_localuse_from_statistics.pl --confirm
4 - Note all items are touched and reported
5 - Check the background jobs table - there are many jobs generated
6 - Apply patch
7 - perl misc/maintenance/update_localuse_from_statistics.pl --confirm
8 - Note 0 items are reported updated, no new background jobs
9 - perl misc/maintenance/update_localuse_from_statistics.pl --confirm --verbose
10 - No items reported
11 - Update some items in the DB
UPDATE items SET localuse = 99 WHERE itemnumber LIKE '%9';
12 - perl misc/maintenance/update_localuse_from_statistics.pl --confirm
13 - Only the number of items changed above reported
14 - UPDATE items SET localuse = 99 WHERE itemnumber LIKE '%9';
15 - perl misc/maintenance/update_localuse_from_statistics.pl --confirm --verbose
16 - Each item changed reported, and the amounts, and the total items updated.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
The system is vulnerable to Formula Injection attacks as the data
stored within the database and exported as CSV/Excel is not being
sanitized or validated against implanted formula payloads
This patch modifies all uses of Text::CSV and derived classes to pass
the "formula" parameter with value of "empty" which replaces formulas
by empty string.
Test Plan:
1) Apply this patch
2) For guided_reports.pl, attempt to export CSV where you've set a column to a formula somehow
( such as "=1+3" )
3) Export that CSV file
4) Note the formula has not been exported
5) Repeat this plan for the remaining scripts that export CSV files
where users can define the outputted data
Signed-off-by: Magnus Enger <magnus@libriotech.no>
Fixed two conflicts. I have tested that this works as advertised on:
- Reports (Download > Comma separated text (.csv)) [Text::CSV::Encoded]
- Circulation > Overdues > Download file of all overdues [Text::CSV_XS]
- misc/export_borrowers.pl [Text::CSV]
This covers all modules used, and both GUI and command line.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Change none to empty in the commit message ! None is the default,
doing nothing. Empty clears the formulas.
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This follow up enhances the sip_cli_emulator.pl to use start-item and/or end-item as params. With these new params the original test plan can be extended:
Test plan:
a) create several manual invoices for patron 23529000035676 : http://localhost:8081/cgi-bin/koha/members/maninvoice.pl?borrowernumber=19
b) run
perl misc/sip_cli_emulator.pl -a 127.0.0.1 -p 6001 -su term1 -sp term1 -l CPL --patron 23529000035676 -m patron_information -s " Y " --start-item=1 --end-item=2
to get fine 1 and 2 or
perl misc/sip_cli_emulator.pl -a 127.0.0.1 -p 6001 -su term1 -sp term1 -l CPL --patron 23529000035676 -m patron_information -s " Y " --start-item=3 --end-item=3
to retrieve fine 3
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds a 'delete' flag to the cronjob to select whether a file should be deleted or archived once processed
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Andrew Fuerste Henry <andrewfh@dubcolib.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds a cronjob to take a marc file from the Koha file directory system and import it before creating order lines
Test plan:
This test plan involves running a cronjob, not sure how easy this is in the sandboxes
1) In system preferences, click Search and then select the Acquisitions option from the left hand menu
2) Enable MarcOrderingAutomation
3) Paste the following into MarcFieldsToOrder
price: 975$p
quantity: 975$q
budget_code: 975$h
4) Paste the following into MarcItemFieldsToOrder
homebranch: 949$a
holdingbranch: 949$b
itype: 949$y
nonpublic_note: 949$x
public_note: 949$z
loc: 949$c
ccode: 949$8
notforloan: 949$7
uri: 949$u
copyno: 949$t
price: 949$g
replacementprice: 949$v
itemcallnumber: 949$o
quantity: 949$k
budget_code: 949$l
Now save the sysprefs
5) In the administration homepage, under Acquisition parameters there should now be a link called "MARC order accounts"
6) Click on this and click the New account button
7) Choose a vendor and a budget
8) Enter a description
9) Enter the following in download directory - "/var/lib/koha/kohadev/tmp/koha_kohadev_upload"
10) Select any record matching settings you want (not important for the test plan, this uses the same code as the marc file staging that already exists in Koha)
11) Check Yes under Check for embedded item record data
12) Click submit
13) We now need to upload a MARC file into the directory that we put in the account. The easiest way to do this is through the marc staging tool
14) Navigate to Cataloging > Stage MARC records for import
15) Upload the file attached to this bug using the Choose file button
16) Click Upload file.
17) Leave the page without completing the new form that appears, we don't want to stage this file. The file we uploaded is now in the directory ready for the cronjob to process as if it had been transferred via SFTP.
18) In the CLI run, perl misc/cronjobs/marc_ordering_process.pl -v -c
19) The logging should show that the file has been processed for that account
20) Navigate to Acquisitions
21) Search for the Vendor that you put in the MARC ordering account
22) There should be a basket created on the vendor with the name of the file you uploaded (there will also be a reference at the start e.g. ea2323966faef2c59a6056dbd1f7d658_filename.marcxml. This is because we used the marc staging tool to upload the file and that adds a reference to filenames. This will not appear when using files from an SFTP location).
23) Click in the basket
24) There should be three orderlines, complete with accounting details that have been mapped from the file.
One line should have a fund of Fund 1_2, quantity of 4 and a replacement price of 6.75, one should have Secondary fund, 2, 6.40 and one should have Main fund,2, 6.30
Signed-off-by: Andrew Fuerste Henry <andrewfh@dubcolib.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Option name changed, POD updated.
The reason not to do this by default is that it is an extra record conversion
to check the validity, so may slow down the import.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds a conversion from MARC -> XML -> MARC to catch any parsing errors.
If errors are found, we then lint the record to catch any problems, output the
warnings, and skip the record.
To test:
1 - Download the sample records from this bug report
2 - perl misc/migration_tools/bulkmarcimport.pl -b --file=520_nosubfield.mrc -v
fails!
3 - perl misc/migration_tools/bulkmarcimport.pl -b --file=003_subfielda.mrc -v
fails!
4 - Apply patch
5 - Repeat 2 & 3 - no change
6 - Add -st switch to the commands:
perl misc/migration_tools/bulkmarcimport.pl -b --file=520_nosubfield.mrc -v -st
perl misc/migration_tools/bulkmarcimport.pl -b --file=003_subfielda.mrc -v -st
7 - The records are now skipped, and the script completes
8 - Confirm the warnings generated are useful
9 - Sign off!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
To test:
1 - Grab the sample file on this bug report
2 - perl misc/migration_tools/bulkmarcimport.pl -b --file=spaceditems.pl -v
3 - An exception for 'Duplicate ID' is thrown and script dies
4 - Apply patch
5 - run script again
6 - The script finishes, no items are added, but record is
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
We can now call metadata_record directly on the Koha::Biblio object.
This aptch updates all modules and controllers to use
Koha::Biblio->metadata_record directly where appropriate. The
exceptions are where we don't require any filtering or the filtering
makes sense to do after the initial fetch.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
It would be nice to be able to filter the plugins run by plugins
nightly for development and testing, and for added flexibility
as to when to run cronjob_nightly for different plugins if need be.
Test Plan:
1) Apply this patch
2) Check the metadata for an installed plugin, copy the name or another
metadata value.
3) Run plugins_nightly.pl with a filter that does not match
e.g. plugins_nightly.pl -m name="No plugin has this name"
4) Note no plugins cronjob methods are run
5) Run plugins_nightly.pl with a filter that *does* match
e.g. plugins_nightly.pl -m name="Example Kitchen-Sink Plugin"
6) Not only the matching plugin runs!
7) Run plugins_nightly with no filter
e.g. plugins_nightly.pl
8) Note all plugins with nightly cronjobs are run!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Most messages in Koha are handled by process_message_queue.pl. This script calls before_send_messages which calls the plugin hook before_send_messages. This works for nearly all messages, but some messages like WELCOME notices are triggered immediately and do not way to for process_message_queue.pl to run. We should move the triggering of before_send_messages into SendQueuedMessages so it is always handled.
Test Plan:
1) Install version 2.5.1 of the kitchen sink plugin
2) Run process_messsage_queue.pl and also trigger a WELCOME notice
3) Note the "Plugin hook before_send_message called with the params" message in the logs for the former but not the latter
4) Apply this patch
5) Repeat step 2, note there the message also shows in the logs for the WELCOME notice!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
In an earlier patch, the `id` column of the `oai_servers`
table was renamed to `oai_server_id`. However, the code
in misc/cronjobs/harvest_oai.pl still uses the old column
name, which results in the following error:
$ misc/cronjobs/harvest_oai.pl --list
DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st execute failed: Unknown column 'id' in 'order clause' at /kohadevbox/koha/Koha/Objects.pm line 403
This patch fixes that.
Test plan:
1) $ misc/cronjobs/harvest_oai.pl --list
[...you get the DBI Exception...]
2) Apply the patch.
3) $ misc/cronjobs/harvest_oai.pl --list
[...list of OAI repositories...]
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Rename Oaipmh to OAI
Rename oaiservers to oai_servers
Rename OaiServer to OAIServer
Rename oai_servers.id to oai_servers.oai_server_id
Rename ServerSearch to server_search
Use a callback function for logs
Use context for string "Set"
Split translation string
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Test plan:
- Run updatedatabase.pl
- Apply the DBIC update patch
- Setup an OAI repository in Administration -> OAI repositories (admin/oaiservers.pl)
If the repository dataformat is in oai_dc, you can use the sample XSLT files:
For Unimarc:
OAIDCtoUNIMARCXML.xsl for bibliographic records
AuthOAIDCtoUNIMARCXML.xsl for authorities
For Marc21:
OAIDCtoMARC21XML.xsl for bibliographic records
AuthOAIDCtoMARC21XML.xsl for authorities
If the repository dataformat is in marc-xml, you can use
OAIMarcxml2KohaMarcxml.xsl for both Unimarc and Marc21.
- Some repositories to test with:
Bibliographic records, oai_dc:
Endpoint: http://staroai.theses.fr/OAIHandler
Set: ddc:260
XSLT: <path_to_src>/koha-tmpl/intranet-tmpl/prog/en/xslt/OAIDCtoUNIMARCXML.xsl
or
XSLT: <path_to_src>/koha-tmpl/intranet-tmpl/prog/en/xslt/OAIDCtoMARC21XML.xsl
Authorities, oai_dc:
Endpoint: https://www.idref.fr/OAI/oai.jsp
Set: a
XSLT: <path_to_src>/koha-tmpl/intranet-tmpl/prog/en/xslt/AuthOAIDCtoUNIMARCXML.xsl
or
XSLT: <path_to_src>/koha-tmpl/intranet-tmpl/prog/en/xslt/AuthOAIDCtoMARC21XML.xsl
Authorities, marc-xml, Unimarc:
Endpoint: https://www.idref.fr/OAI/oai.jsp
Set: a
XSLT: <path_to_src>/koha-tmpl/intranet-tmpl/prog/en/xslt/OAIMarcxml2KohaMarcxml.xsl
- Note: marc-xml harvesting will not work with HTTP::OAI::Harvester version 3.xx
Version 4.xx is needed.
See Bug 17704 for updating Koha default version to 4.xx
- If you want to receive a report email, fill the OAI-PMH:HarvestEmailReport
syspref. The model used (OAI_HARVEST_REPORT) can be edited in tools -> Notices and slips.
- start a harvest with the following command:
misc/cronjobs/harvest_oai.pl
This script starts an OAI Harvest
This script has the following parameters :
-h --help: this message
-v --verbose (enable output to stdout)
-r --repository: id of the OAI repository
-d --days: number of days to harvest from (optional)
-l --list: list the OAI repositories
-f --force: force harvesting (ignore records datestamps) (optional)
Example: misc/cronjobs/harvest_oai.pl -r 1 -d 10 -v -f
- Check the logs in the Log Viewer (tools/viewlog.pl)
(CronjobLog syspref must be activated)
Sponsored-by: KohaLa
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Sponsored-by: Association KohaLa - https://koha-fr.org/
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Bug 29948 added a "multiple_sortable" type for sysprefs, to have the
options sortable. But the options were not picked for translation.
This patch fixes it.
Test plan:
Run `cd misc/translator && perl translate update LANG` before and after
this patch. Notice that with this patch you see the different options of
OPACAuthorIdentifiersAndInformation in the LANG-pref.po file (eg.
"Associated group (373$a$s$t$u$v$0)")
Translate the options
Install the translated version of the templates and confirm that the
strings appear translated on the interface.
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
The holds queue builder can take a very long time to run on systems with many holds. For example, a partner with 124,784 unfilled ( not found ) holds, is taking about 64 minutes to run. If we run that same number of holds in 5 parallel chunks ( splitting the number of records as evenly as possible, but *not* taking into account the holds per bib ), it takes 21.5 minutes. If we use 10 loops, it takes less then 14 minutes.
Test Plan:
0) Install the Perl library Parallel::ForkManager
1) Generate a huge number of holds ( a few thousand at the minimum )
2) Run the holds queue builder, use the `time` utility to track how much
time it took to run
3) Set HoldsQueueParallelLoopsCount to 10
4) Repeat step 2, note the improvement in speed
5) Experiment with other values for HoldsQueueParallelLoopsCount
6) prove t/db_dependent/HoldsQueue.t
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
When the 'RestrictPatronsWithFailedNotices' syspref is enabled then
patrons with email and sms notices which failed sending (have a
message_queue.status field of 'failed') have a restriction (debarment)
applied to them.
Test plan:
1. In the Koha staff client > Tools > Overdue notice/status triggers and
create the 'First' rule for all patron categories as:
Delay: 1
Letter: Overdue Notice
SMS: ticked
Ensure you have an SMS notice for the ODUE letter.
2. In the system preferences make sure you enter dummy data into the
SMSSendUsername, SMSSendPassword and EmailSMSSendDriverFromAddress
sysprefs
2. Find two non-debarred patrons and make sure they have invalid SMS numbers set. The SMS numbers must be INCORRECT, for example "123" as an SMS number. Leaving this field empty will result in the message_transport_type defaulting to 'print' instead of 'sms'.
3. Check one item out to each patron in step 2
4. Jump into the database and run the query:
UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower1>;
UPDATE issues SET date_due=<2 days before current date> WHERE
borrowernumber=<borrower2>;
5. Go to misc/cronjobs directory and enter the Koha shell:
sudo koha-shell <instancename>
6. Run: ./overdue_notices.pl
7. Exit the shell and jump back into the database and run the query:
SELECT message_transport_type, status FROM message_queue WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;
8. Confirm both new notice records have the message_transport_type is 'sms' and the status of 'pending'
9. Exit the database and re-enter the Koha shell and run the command:
./process_message_queue.pl
10. Jump back into the database re-run the query from step 7 and confirm
the status is 'failed' for both
11. Also run the query:
SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;
Notice there is no added debarment to these two patrons
12. Apply patch, restart memcached and plack. In the
installer/data/mysql directory enter the Koha shell and run the command:
./update_database.pl
13. In the Administration > Global System Preferences interface of the
staff client notice there is a new system (set to "Don't" by default)
named 'RestrictPatronsWithFailedNotices'.
Enable it (i.e. select 'Do')
14. Create a new file in the /etc/cron.daily directory named koha-custom and
add the following line to it:
koha-foreach --chdir --enabled
/usr/share/koha/bin/cronjobs/restrict_patrons_with_failed_notices.pl
15. In the misc/cronjobs directory enter the Koha shell and run the command:
./restrict_patrons_with_failed_notices.pl
16. The script should output text saying: There are borrowers with
failed SMS or email notices
However because you haven't given the script the argument -c it won't
apply debarments (restrictions) to any of the patrons with the failed
SMS or email notices.
16. Query the borrower_debarments table:
SELECT * FROM borrower_debarments WHERE borrowernumber=<borrower1> OR
borrowernumber=<borrower2>;
Notice they still have no restriction
17. Now in the Koha shell run the command:
./restrict_patrons_with_failed_notices.pl -c
18. Notice the script outputs the text:
There are borrowers with failed SMS or email notices
Applying restriction to patron <borrowernumber>: <borrower firstname>
<borrower surname>;
19. Repeat step 16 and notice both patrons now have 1 restriction each
with the borrower_debarments.type=SUSPENSION and comment=SMSnumber
invalid and expiration=NULL
20. Query the borrowers table:
SELECT debarred, debarredcomment FROM borrowers WHERE
borrowernumber=<borrower1> OR borrowernumber=<borrower2>;
21. Notice the values are:
debarred= 9999-12-31
debarredcomment= SMS number invalid
22. Repeat step 17 and notice the script outputs:
There are borrowers with failed SMS or email notices
Patron <borrowernumber>: <borrower firstname> <borrower surname> is currently restricted due to having an invalid SMS number. No new restriction applied"
23. Repeat step 16 and notice no new debarment has been added to those
borrowers as they have already been restricted from having a failed SMS
notice.
24. In the Koha home directory run the command:
prove t/db_dependent/Koha/Notices.t
This unit test contains the tests for the new subroutines added to
Koha/Notice/Message.pm which are restrict_patron_when_notice_fails() and
get_failed_notices()
25. All tests should pass
26. Sign off
Sponsored-by: Catalyst IT
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Libraries may want to use this for more varied purposes and
adjust the language of the email based on the sftp upload
that succeeded or failed.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Tidy and add exec to atomic update
Fix POD
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Sponsored-by: Horowhenua Libraries, Toi Ohomai Institute of Technology, Plant and Food Research Limited, Waitaki District Council, South Taranaki District Council New Zealand
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
SFTP_FAILURE and SFTP_SUCCESSFUL notices can be configured from Notices
and slip tool.
Sponsored-by: Horowhenua Libraries, Toi Ohomai Institute of Technology, Plant and Food Research Limited, Waitaki District Council, South Taranaki District Council New Zealand
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
To test you need to have a remote server you are authorised to SFTP files to.
Test plan:
1. Apply patch and restart services
2. Generate a file on your Koha server. Place it in /tmp/test.mrc
3. Manually connect to the SFTP remote host and accept the hostkey
4. Enter Koha shell
5. Run the sftp_file.pl command-line script
./sftp_file.pl --host <your remote hostname> --user <your user> --pass <your password> --upload_dir <your upload directory> --file /tmp/test.mrc
6. Confirm the SFTP was successful
Sponsored-by: Horowhenua Libraries, Toi Ohomai Institute of Technology, Plant and Food Research Limited, Waitaki District Council, South Taranaki District Council New Zealand
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch identifies date fields in the database (timestamp, datetime, date) that are 0000-00-00 and sets them to NULL
Test plan -
Examples of setting up some 0000-00-00 date fields:
e.g. Borrowers table
update borrowers set dateexpiry="0000-00-00" where borrowernumber=49;
e.g. Items table
update items set datelastseen="0000-00-00" where itemnumber=12;
To identify the cases:
1) Set up some 0000-00-00 date fields
2) Run the misc/maintenance/search_for_data_inconsistencies.pl script
3) Run fix_invalid_dates.pl -v
To fix the cases:
1) Run fix_invalid_dates.pl -c -v
2) Notice the value has changed from 0000-00-00 to NULL
e.g. Borrowers table
select dateexpiry from borrowers where borrowernumber=49;
e.g. Items table
select datelastseen from items where itemnumber=12;
Signed-off-by: Shi Yao Wang <shi-yao.wang@inlibro.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
After the introduction of misc/workers/es_indexer_daemon.pl,
misc/workers/background_jobs_worker.pl should not handle the
elastic_index queue. But the latter script still mentions this
queue in the POD. This patch removes that mention.
To test:
- Run perldoc misc/workers/background_jobs_worker.pl
- Verify "elastic_index" is mentioned under --queue
- Apply this patch
- Run perldoc again and verify the mention of "elastic_index"
is gone
Signed-off-by: Sukhmandeep Benipal <sukhmandeep.benipal@inLibro.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds POD to the script and rpelaces the custom help script with pod2usage
TO test:
perl misc/cronjobs/erm_run_harvester.pl
Confirm you get helpful usage data
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch simply adds logging to the cronjob
To test:
1 - Enable CronjobLog
http://localhost:8081/cgi-bin/koha/admin/preferences.pl?op=search&searchfield=CronjobLog
2 - Enable ERMModule
http://localhost:8081/cgi-bin/koha/admin/preferences.pl?op=search&searchfield=ERMModule
3 - Home -> E-resource Management -> Data providers -> New data provider
Fill from registry or create your own, but change the URL and put in any required numbers
We don't need a real test, just to run the ob
4 - perl misc/cronjobs/erm_run_harvester.pl --begin-date 2024-01-01 --dry-run --debug
5 - View the logs, no log of the script
6 - Apply patches
7 - Run the job again
8 - Check logs
9 - Success!
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Test plan:
1) Create an authority with a unique value in a searchable field, for instance a PERSO_NAME authority with 'BZ18273' in the 100$a field.
2) Create a second authority with the same value in the same field.
3) Export this second authority as iso2709 (Save -> MARC (Unicode/UTF-8))
4) Run bulkmarcimport.pl with the following parameters:
perl bulkmarcimport.pl -a --file=/kohadevbox/koha/auth-xxxx.utf8 --match='Heading,100$a' -l /kohadevbox/koha/bulk.log
5) Check in the log file and/or through search that a third authority has been added (despite using the --match option)
6) Apply the patch
7) Delete the 3 authorities and repeat step 1 to 4.
8) Check that a third authority has not been added (--match worked)
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch remove the subroutine C4::Members::DeleteExpiredOpacRegistrations.
The code is moved and adjusted to Koha::Patrons.
We now have 2 new methods:
Koha::Patrons->filter_by_expired_opac_registrations
Koha::Patrons->filter_by_safe_to_delete
Test plan:
Run the misc/cronjobs/cleanup_database.pl script with the
--del-exp-selfreg (as well as --verbose and --confirm) and confirm that
it behaves as expected
Note that it improve the output of the verbose mode and now tell if the something
is wrong with the config (syspref not set)
Signed-off-by: Sukhmandeep Benipal <sukhmandeep.benipal@inLibro.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch removes the subroutine
C4::Members::DeleteUnverifiedOpacRegistrations and use the
Koha::Objects->filter_by_last_update method that is used for the other
flag in this script.
Test plan:
Confirm that --del-unv-selfreg works the same before and after this
patch applied
Signed-off-by: Sukhmandeep Benipal <sukhmandeep.benipal@inLibro.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Add documentation for the new -si, --skip_indexing parameter
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Before the previous patch we were indexing with every AddAuthority/ModAuthority call and I assumed
we were also indexing during the commity, however, it appears we were not. This patch ensures we push
the record and ids into the arrays for indexing during commit.
Additionally I add a skip to next record on error, to match biblio behaviour.
I also correct a log referring to biblios during authority importing
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds a new optoin to skip indexing to bulkmarcimport.
It also fixes a bug where authorities were being indexed multiple times
during import.
To test:
1 - Apply patch
2 - Download the sample files on this bug
3 - Perform asearch engine search that should retrieve all records( 'a' in Zebra, '*' in ES)
4 - Note the total (435 in KTD)
5 - perl misc/migration_tools/bulkmarcimport.pl -b -v --file=bug_35466_b.mrc
6 - Search again, note increaed by 100
7 - perl misc/migration_tools/bulkmarcimport.pl -b -v --file=bug_35466_b.mrc --skip_indexing
8 - Search again, no increase
9 - perl misc/search_tools/rebuild_elasticsearch.pl -v
10 - Search again, increase, records were added but not initially indexed
11 - Browse to authorities and search as for biblios
12 - perl misc/migration_tools/bulkmarcimport.pl -a -v --file=bug_35466_a.mrc
13 - Search again, note increase
14 - perl misc/migration_tools/bulkmarcimport.pl -a -v --file=bug_35466_a.mrc --skip_indexing
15 - Search again, no increase
16 - perl misc/search_tools/rebuild_elasticsearch.pl -v
17 - Search again, increase, records were added but not initially indexed
18 - Sign off! Hi5!
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Thomas Klausner <domm@plix.at>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch was envisioned by Julian Maurice.
Test plan:
0. Apply the 1st patch (the revert)
1. Go to `misc/translator/po`
2. es-ES-messages-js.po
3. Mark "Edit" string as fuzzy (around line 48). It should look like:
```
msgid "Edit"
msgstr "Editar"
```
4. Edit ./intranet-main.tt and add the following lines at the bottom,
inside the `$(document).ready` block:
```javascript
console.log(_("Edit"));
console.log(__("Edit"));
```
5. Install the templates
k$ koha-translate --install es-ES --dev kohadev && restart_all
6. Enable *es-ES* by searching for `language` in the sysprefs and switch
to it for the staff interface.
7. Go to the Koha home page, open the browser console
=> FAIL: Notice that the second log in the console is displaying the
fuzzy string (i.e. is being translated when it shouldn't)
8. Apply this patch
9. Re-install the translated templates:
k$ koha-translate --update es-ES --dev kohadev && restart_all
10. Repeat 7
=> SUCCESS: With this patch applied both logs show the English version of the
string.
11. Remove the fuzzy flag on `es-ES-messages-js.po`
12. Repeat 9 and 10
=> SUCCESS: The results are similar to step 7, but in this case they are
expected as the string is not marked fuzzy.
13. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Aleisha Amohia <aleishaamohia@hotmail.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This reverts commit 4eb9816354.
Signed-off-by: Aleisha Amohia <aleishaamohia@hotmail.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
When the automatic renewal cron job is using parallel processing, it
aims to process all of the renewals for any given patron together in one
chunk to avoid data conflicts. To accomplish this, it starts a new data
chunk each time it encounters a new patron. However, if a patron's
renewing checkouts aren't all consecutive in the database, that patron's
data ends up split across multiple chunks.
We need to sort the issues by borrowernumber before attempting to chunk
them in order to make sure they are chunked and processed correctly.
To test (using KTD default test data):
Setup:
1. Edit the default circulation rule:
- Set Automatic renewal to "Yes"
- Set No automatic renewal before to 3
2. Open the following patron accounts in separate tabs:
- Floyd Delgado
- Joyce Gaines
- Edna Acosta
- Mary Burton
3. Perform the following patron account edits for each of the above
patrons (and keep the tabs open):
- Enable automatic renewal notices, and set them to digests only
- Add a value to the email field
4. Enter the kshell (ktd --shell)
5. Edit /etc/koha/sites/kohadev/koha-conf.xml, and add the following
lines near the end, just above the </config> and </yazgfs> closing
tags:
<auto_renew_cronjob>
<parallel_loops_count>2</parallel_loops_count>
</auto_renew_cronjob>
6. restart_all
Reproducing the issue:
7. Apply the test patch only
8. Run perl generate_checkouts.pl to generate test data
9. perl misc/cronjobs/automatic_renewals.pl -v -c
--> The test patch added output that will show how the renewals were
chunked into "chunk 0" and "chunk 1" for the two parallel loops.
Note that the issues for each borrower are not processed nicely in
one chunk, but are separated across multiple chunks and alternated
with other borrowers.
10. Check the checkouts for each of the four patrons from above
--> All checkouts should have renewed
11. Check the notices tab for each of the four patrons
--> Notice errors in the automatic renewal digest notices. A patron's
renewals may be split across multiple digests, a digest may be
missing renewals, or a patron may not have received a digest at all
Testing the patch:
12. Apply the second patch
13. Reset the due dates on all checkouts so that they will all be
eligible for automatic renewal again:
- koha-mysql kohadev
- UPDATE issues SET date_due=<two days from today>;
14. perl misc/cronjobs/automatic_renewals.pl -v -c
--> Note that the renewals are now correctly chunked by patron
15. Check the checkouts and notices tab for each of the four patrons
--> All checkouts should have renewed, and all patrons should have a
single new Auto Renewals Digest notice that correctly lists all of
their renewed items
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Edit: tidied the code block inline (tcohen)
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Olivier V <olivier.vezina@inLibro.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
1 - perl misc/migration_tools/bulkmarcimport.pl -b --file=notafile.mrc -v
2 - Output:
Characteristic MARC flavour: MARC21
0 MARC records done in 0.00804400444030762 seconds
3 - Apply patch
4 - perl misc/migration_tools/bulkmarcimport.pl -b --file=notafile.mrc -v
5 - Output:
Characteristic MARC flavour: MARC21
Could not open notafile.mrc: No such file or directory at misc/migration_tools/bulkmarcimport.pl line 262.
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Script migration_tools/build_oai_sets.pl is missing ORDER BY
biblionumber.
This is a problem when using OFFSET and LIMIT, the results may be
differenly orderder between two calls of this script.
Test plan :
1) Create a OAI SET with all records
2) Run migration_tools/build_oai_sets.pl with the first half of records
3) Run migration_tools/build_oai_sets.pl with the second half of records
4) Check all records are in the OAI set
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch removes setting AutoCommit to 0 and commiting only every X records.
Instead we commit as we go and report progress using a parameter.
Bug 36474 reduced the numebr of changes that are being committed, so this should be a reasonable change. The
use of commits without transactions was causing problems if the library was active while the script ran.
To test:
1 - perl misc/cronjobs/update_totalissues.pl -c
2 - Script runs, but with unknown parameter
3 - perl misc/cronjobs/update_totalissues.pl -p 10
4 - Script runs and reports every 10 records
5 - per; misc/cronjobs/update_totalissues.pl
6 - Script runs and reports every 100 records by default
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds a new check in UpdateTotalIssues to check that we are changing the number
of total issues before calling ModBiblio
To test:
0 - Enable CataloguingLog
1 - Checkout an item
2 - Run : misc/cronjobs/update_totalissues.pl --use-stats --commit=1000 -v
3 - In report, note all biblios were updated
4 - Check action_logs - note a new entry for every biblio
5 - Apply patch
6 - Repeat
7 - Note no biblios reported updated
8 - Note no new cataloguing log entries
9 - Checkout the item again
10 - Run again
11 - Note biblionumber has updated count in verbose output
12 - Note report only rpeort 1 biblio modified, the rest only processed
13 - Only one line added to action_logs
14 - Run it again
15 - Confirm no updates
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This change lazy-loads modules for setters in the
koha-preferences tool, so that getters are free to run super fast.
Between BZ 37657 and BZ 37682 we effectively eliminate the
overhead of running "get" or "dump" commands via the koha-preferences
tool.
Test plan:
1. time misc/admin/koha-preferences get SearchEngine
2. Note time is about .35 seconds
3. time misc/admin/koha-preferences dump
4. Note time is about .35 seconds
5. Create sysprefs.yml
---
marcflavour: MARC21
viewMARC: 1
6. time misc/admin/koha-preferences load -i sysprefs.yml
7. Note time is about .35 seconds
8. time misc/admin/koha-preferences set SearchEngine Elasticsearch
9. Note time is about 1 seconds
10. Apply patch
11. Repeat the koha-preferences commands above
12. Note that the "dump" and "get" commands run in about .09-.1
seconds. The "load" and "set" commands still take the same amount
of time as their behaviours haven't changed
13. misc/admin/koha-preferences set SearchEngine Elasticsearch1
14. koha-mysql kohadev
15. select * from action_logs where module = 'SYSTEMPREFERENCE'
order by action_id desc limit 5;
16. Note that the action log showing Elasticsearch1 update says
"interface"
of "commandline" and "script" of "koha-preferences"
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
By using Koha::Database->dbh() to use a minimal database handle
which doesn't preload the whole DBIx::Class schema, we're able to
run the same command 2-3 times faster.
This is beneficial when running the tool in a loop which runs the
command serially one by one.
Test plan:
1. time misc/admin/koha-preferences get SearchEngine
2. Note time is about 1 second
3. time misc/admin/koha-preferences dump
4. Note time is about 1 second
5. Create sysprefs.yml
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
When libraries have a lot of checkouts, or an AMH, checkins can happen while the cron is running.
This patch simply adds an eval around the auto renewal attempt in case of early check in or other errors.
You can verify cron completed by enabling cronjob log in system preferences and checking the action logs
To test:
1 - Add 'sleep(10);' to automatic_renewals.pl
2 - Set circulation rules to enable automatic renewals
3 - Issue an item to a patron
4 - perl misc/cronjobs/automatic_renewals.pl -v
5 - Confirm item would not be renewed
6 - perl misc/cronjobs/automatic_renewals.pl -v -c
7 - Quickly check in the item
8 - The cronjob dies
DBIx::Class::Row::update(): Can't update Koha::Schema::Result::Issue=HASH(0x586e1a674fb0): row not found at /kohadevbox/koha/Koha/Object.pm line 172
9 - Apply patch
10 - Checkout the item again
11 - perl misc/cronjobs/automatic_renewals.pl -v -c
12 - Quickly checkin the item
13 - You get a warning, but the cron completes
Signed-off-by: CJ Lynce <cj.lynce@westlakelibrary.org>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Tidy the whole thing
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Since version 24.05, due to the changes mentioned at
https://wiki.koha-community.org/wiki/Koha_/svc/_HTTP_API#Changes_coming_in_Koha_24.05 ,
the `connexion_import_daemon.pl` stopped working. The reason for this is that
it did not use CSRF tokens.
To test:
1. Get a Koha instance on 24.05, before applying the patch.
2. Create a plain text file somewhere on the server containing a raw MARC
record (not XML). Let's call it `marc.txt`.
3. On the server, create a config file like this:
```
host: 0.0.0.0
port: 5500
koha: http://localhost:82 # Where 82 is the port of the Koha staff interface.
user: foo # A Koha staff user.
password: Fooo1234 # The Koha staff user's password.
import_mode: stage
```
4. Run `./connexion_import_daemon.pl --config the-config-file-path`
5. In another terminal on the same server (or from anywhere that can reach the
port opened by the `connexion_import_daemon.pl` script,
run `nc localhost 5500 < marc.txt`
6. Observe in the stderr of the daemon script: `Response: Unsuccessful request`
7. Stop the daemon script.
8. Apply the patch and repeat steps 4 and 5.
9. Observe in the stderr of the daemon script:
`Response: Success. Batch number ... - biblio record number HASH(...) added to Koha`
10. Check at /cgi-bin/koha/tools/manage-marc-import.pl for a batch named
`(webservice)`. It should contain one record now. This is how we know that
authentication between the daemon and Koha worked, which is what this
patch tries to address.
Thanks-to: David Cook <dcook@prosentient.com.au>
Sponsored-by: Reformational Study Centre <www.refstudycentre.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Trivial patch.
Change the --branch and --skip-branch options of the longoverdue cron script
to --library and --skip-library to meet the Terminology Guidelines.
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This script doesn't seem to be included in cron files by default.
This change is to allow script parameters to effect only certain branches.
This allows the script to be added multiple times to a cron file with
different settings for different branches.
Test plan:
1. apply patch
2. identify two books at different branches the same number of days overdue.
3. run the longoverdue.pl script specifying one of the branches in the
--branch command line parameter.
i.e. koha-shell -c 'misc/cronjobs/longoverdue.pl --branch branch_code --lost 60=2 --maxdays=61 --confirm' instance_name
4. observe that the book at the specified branch has been or would be affected
by the script while the other book is not.
Signed-off-by: Tadeusz „tadzik” Sośnierz <tadeusz@sosnierz.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Enable the export_records.pl script use a report output to export biblios or authorities
Test plan:
1. Apply patches and restart services
2. Create a SQL report (id=1)
SELECT biblionumber
FROM biblio
3. Create a SQL report (id=2) and set an item as notforloan = -1
SELECT title, author, biblio.biblionumber
FROM biblio
LEFT JOIN items USING (biblionumber)
WHERE items.notforloan = <<Not for loan|NOT_LOAN>>
4. Create a SQL report (id=3)
SELECT title, author
FROM biblio
5. Create a SQL report (id=4)
SELECT authid
FROM auth_header
6. Run export_records.pl using report id=1 and confirm a koha.mrc file is created in the misc directory:
cd misc
./export_records.pl --report_id=1
7. Delete the koha.mrc file
8. Run export_records.pl using report id=2
./export_records.pl --report_id=2
9. Notice you are prompted to supply a parameter
10. Re-run report id=2 supplying a parameter. Confirm the koha.mrc file is created and contains bib data
./export_records.pl --report_id=2 --report_param=-1
11. Run export_records.pl using report id=3
./export_records.pl --report_id=3
12. Notice you get the message: The --report_id you specified does not fetch a biblionumber
13. Delete the koha.mrc file
14. Run export_records.pl using report id=4
./export_records.pl --report_id=4
15. Notice you get a message 'The --report_id you specified does not fetch a biblionumber'
16. Re-run export_records.pl setting the record-type=auths
./export_records.pl --record-type=auths --report_id=4
17. Notice the koha.mrc file is generated and contains auth data
Sponsored-by: Horowhenua Libraries, Toi Ohomai Institute of Technology, Plant and Food Research Limited, Waitaki District Council, South Taranaki District Council New Zealand
Signed-off-by: Alexandre Noel <alexandre.noel@inlibro.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This change consistently sends the Csrf-Token in the request header.
Previously, one POST sent it in the request body, while the other POST
sent it in the request header. Since we're using an API, it's best
for us to always send it in the request header
Test plan:
0. Apply the patch
1. perl ./misc/migration_tools/koha-svc.pl \
http://localhost:8081/cgi-bin/koha/svc koha koha 29 > bib-29.xml
2. perl ./misc/migration_tools/koha-svc.pl \
http://localhost:8081/cgi-bin/koha/svc koha koha 29 bib-29.xml
3. Note that the following appears in STDOUT and there is no 403 error:
"update 29 from bib-29.xml"
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This change fixes the Koha::SVC to store the CSRF token for
the authenticated session for further POSTing.
Test plan:
0. Apply the patch
1. perl ./misc/migration_tools/koha-svc.pl \
http://localhost:8081/cgi-bin/koha/svc koha koha 29 > bib-29.xml
2. perl ./misc/migration_tools/koha-svc.pl \
http://localhost:8081/cgi-bin/koha/svc koha koha 29 bib-29.xml
3. Note that the following appears in STDOUT and there is no 403 error:
"update 29 from bib-29.xml"
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
When using __() (ie. Gettext.js) we are seeing the translations that are marked as fuzzy.
This is definitely not the expected behaviour.
It happens because (our version of) po2json are old and no longer maintained,
and just embed them.
It seems that the bin we have has been upgraded to a JS version
(different authors).
Test plan:
(replace LANG with your language code)
0. Do not apply this patch
Edit misc/translator/po/LANG-messages-js.po
Mark a string as fuzzy
Edit ./intranet-main.tt and add the following lines inside $(document).ready
console.log(_("Your string"));
console.log(__("Your string"));
Replace "Your string" with the string you are actually testing.
Update the templates: `koha-translate --update LANG --dev kohadev && restart_all`
Go to the Koha home page, open the console.
=> Notice that the second log in the console is displaying the fuzzy string.
1. Apply this patch
Install the new version of po2json using `yarn install`
Repeat the previous steps.
=> With this patch applied both logs show the English version of the
string.
Remove fuzzy, update the templates and try again.
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
In "Bug 19532: Make recalls.status an ENUM" one-letter statuses were
replaced by their verbal equivalents. But in overdue_recalls.pl
remained "R". As a result, recalls never overdue.
No test plan provided--obvious correction.
Sponsored-by: Ignatianum University in Cracow
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
The cron can take a very long time to run on systems with many issues.
For example, a partner with ~250k auto_renew issues is taking about 9 hours to run.
If we run that same number of issues in 5 parallel chunks
( splitting the number of issues as evenly as possible ), it could take under 2 hours.
Test Plan:
1) Generate a number of issues marked for auto_renew
2) Run the automatic_renewals.pl, use the `time` utility to track how much time it took to run
3) Set parallel_loops to 10 in auto_renew_cronjob section of config in koha-conf
4) Repeat step 2, note the improvement in speed
5) Experiment with other values
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds logging of unhandled exceptions that could occur. This
is happening on busy production sites right now. This is also useful for
plugin jobs that might not be 100% following the guidelines and would
benefit from this.
But as the [DO NOT PUSH] patch highlights, this is something we really
want to have on our current codebase, as a database connection drop
might make us reach that `catch` block we are adding logging to on this
patch.
To test:
1. Apply the [DO NOT PUSH] patch
2. Run:
$ ktd --shell
k$ restart_all ; tail -f /var/log/koha/kohadev/worker*.log
3. Pick a valid barcode on the staff UI
4. Use the 'Batch delete items' tool in the cataloguing section
5. Start the job for deleting the item
=> FAIL: The item got deleted, but the job marked as failed and no logs
about the reasons
6. Apply this patch and repeat 2-5
=> SUCCESS: Same scenario but there's a log with the error message
7. Sign off :-D
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
By adding an empty text body, we force Email::Stuffer/Email::MIME
to use multipart/mixed handling for the attachment instead of forcing
a single part (ie direct attachment) email, which is not consistently
handled by different email clients.
An empty text body is language-neutral (ie not imposing English),
and it allows SMTP servers to inject organisational footers into
the email (e.g. confidentiality notices).
Signed-off-by: Magnus Enger <magnus@libriotech.no>
I have tested this solution in production, and it works for me.
Bug 32575: Tidy patch
Signed-off-by: Magnus Enger <magnus@libriotech.no>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
To test:
1 - Perform some transactions
2 - Enable Pseudonymization
3 - perl misc/maintenance/pseudonymize_statistics.pl -v
4 - Confirm test run an nothing changed
5 - perl misc/maintenance/pseudonymize_statistics.pl -v -c
6 - Confirm statistics are pseudonymized
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Amend: tidied the changes [tcohen]
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Like IncludeSeeFromInSearches adds see from heading in search (4xx),
new system preference IncludeSeeAlsoFromInSearches adds see also from heading (5xx).
Test plan:
Test on both Elasticsearch and Zebra:
1)
1.1) Create an authority record with heading, see from heading 4xx and see also from heading 5xx
1.2) Use this authority in a biblio record
2)
2.1) Set IncludeSeeFromInSearches and IncludeSeeAlsoFromInSearches to "Don't include"
2.3) Rebuild search engine
2.4) Perform a search on heading text => you find the biblio record
2.5) Perform a search on see from heading => you do not find the biblio record
2.6) Perform a search on see also from heading => you do not find the biblio record
3)
3.1) Set IncludeSeeFromInSearches and IncludeSeeAlsoFromInSearches to "Include"
3.2) Rebuild search engine
3.3) Perform a search on heading text => you find the biblio record
3.4) Perform a search on see from heading => you find the biblio record
3.5) Perform a search on see also from heading => you find the biblio record
Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
To test:
1. Set up autorenewals bu adjusting circulation rules:
'Automatic renewal' -> 'Yes'
'No automatic renewal before' -> 5
2. Set 'AutoRenewalNotices' to 'according to patron messaging preferences'.
3. Set an AUTO_RENEWALS and AUTO_RENEWALS_DGST notice to include branch info. I am using this to test:
Branchcode: [% branch.branchcode %]
Branch name: [% branch.branchname %]
Branch address: [% branch.branchaddress1 %]
Branch address2: [% IF branch.branchaddress2 %][% branch.branchaddress2 %][% END %]
Branch city: [% branch.branchcity %], [% branch.branchstate %] [% branch.branchzip %]
4. Make sure your branch has the proper infro. filled out in Libraries administration.
5. Find a patron and adjust the messaging preferences so they receive automatic renewal notices. Also make sure the patron has an email.
5. Check out some items and make them due with the next 5 days.
6. Run the automatic_renewal cron job:
perl /kohadevbox/koha/misc/cronjobs/automatic_renewals.pl -c -v
7. Notice no branch information displays.
8. APPLY PATCH
9. Checkout items from multiple issuing branches to a single patron.
10. Make sure the patron's messaging prefs are set to revieve NON-digestable notices.
11. Run the automatic renewal job, each notice should include the branch information from the issuing library.
12. Change the patron's messageing preferences to receieve digestable notices.
13. Run the job without the --digest-per-branch flag. You should get a single notice with the branch info. coming from the patron's home branch.
14. Run the job with the --digest-per-branch flag. You should get seperate digested notices with the branch info. coming from the issueing library branch.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
For reasons unknown, GetOptions is inserting an empty string into the letter_code list. If you are running the script with a letter code filter, the empty string is added to the OR so it functions. If no letter_code is passed, the search requires the letter code to be an empty string, which will of course fail. Even more perplexing is that this does not happen for the type list which is essentially identical code.
Test Plan:
1) Generate some messages in the message queue
2) Run `process_message_queue.pl -v -c`
3) Note nothing happens
4) Apply this patch
5) Repeat step 2
6) Messages are sent!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
We are not using it and it's confusing, let's remove the context stack.
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Test plan:
1. git grep 'reseve'. Notice there are instances of 'reseve'
2. Apply patch
3. Repeat step 1, there should be no instances of 'reseve'
Sponsored-by: Catalyst IT, New Zealand
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
--category-code was not checked in the "at least one filter option"
check but it is clearly a filter option.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
* Change the SYNOPSIS to better describe the different ways to use the
script
* Only show the SYNOPSIS when options used are wrong (unknown option,
no filter options, or neither -c nor -v)
* Show the options details only with --help
* Clarify the fact that -v is required when -c is not supplied in the
description of both options
* Print a specific error message for the following cases:
* no filters options
* neither -c nor -v was supplied
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch adds a hint to the end of the script to notify the end user
that they may need to run the build_holds_queue cronjob if they are
using RealTimeHoldsQueue.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
If RealTimeHoldsQueue is on, touch_all_biblios triggers a update_holds_queue_for_biblios background job for each affected record. This will result in a as many background jobs being queued up as records! It makes far more sense for this script to not do that which gives the administrator the option for running the holds queue builder if the changes would affect holdability, or to not run it at all.
Test Plan:
1) Run touch_all_biblios.pl
3) Note a update_holds_queue_for_biblios background job is queued for each record touched
4) Apply this patch
5) Merge touch_all_biblios.pl again
6) Note that no update_holds_queue_for_biblios jobs were queued
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This bug originaly wants to get rid of "noindex" coming from this meta
tag:
<meta name="robots" content="noindex">
But actually we have other strings from the meta tags that should not be
translated.
Test plan:
0. Do not apply this patch
1. cd misc/translator/po && gulp po:update --lang es-ES (or any other
lang)
2. git commit -a -m"wip"
3. Apply this patch
4. Repeat 1 and git diff to show the diff
Notice that strings that should not be translated are removed from the
po files (actually commented)
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Philip Orr <philip.orr@lmscloud.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Philip Orr <philip.orr@lmscloud.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds the option to specify a framework to be used when overlaying records from webservices/connexion
To test:
1 - vim /etc/koha/sites/kohadev/connexion.cnf
2 - Set content:
host:
port: 8888
koha:http://localhost:8081
log:/var/log/koha/kohadev/connexion.log
match:ISBN
user:kohauser
password:kohapass
overlay_action:replace
nomatch_action:create_new
item_action:always_add
import_mode:direct
framework:BKS
overlay_framework:
debug:1
3 - Save the sample file from this bug into your kohaclone (or copy and paste into a file your koha test site can reach)
4 - On the command line:
perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf
5 - In another terminal:
cat bug_33418.test | nc -v localhost 8888
6 - It should report success and a biblionumber
7 - In Koha: Cataloguing->Manage staged MARC record
8 - View the most recent batch with file name (webservice)
9 - Confirm it was imported, no match
10 - Click 'View' under the 'Record' column
11 - Confirm record loads correctly and 'MARC framework' detail is 'Books, Booklets, Workbooks'
12 - On the terminal repeat the command:
cat bug_33418.test | nc -v localhost 8888
13 - It should succed
14 - View the new batch, confirm the record matched this time
15 - View the record details, confirm framework is now 'default'
16 - On the first terminal hit Ctrl+C to stop the daemon
16 - Edit connexion.cnf and change:
import_mode:stage
framework:ACQ
overlay_framework:IR
17 - Restart daemon:
perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf
18 - Delete the record created above
19 - On the second terminal repeat the command:
cat bug_33418.test | nc -v localhost 8888
20 - Confirm the batch is created, but not imported
21 - In terminal:
perl misc/cronjobs/import_webservice_batch.pl --framework=ACQ --overlay_framework=BKS
22 - Confirm batch imported, and record in ACQ framework
23 - In terminal:
cat bug_33418.test | nc -v localhost 8888
perl misc/cronjobs/import_webservice_batch.pl --framework=ACQ --overlay_framework=BKS
24 - Confirm batch added, record matched, record imported, and record now in Books framework
25 - Stop the deamon, edit connexion.cnf:
import_mode:direct
26 - Start the daemon, and on other terminal repeat:
cat bug_33418.test | nc -v localhost 8888
27 - Confirm record in Binders framework
28 - Set record framework to Books
29 - Stop daemon, edit cnf and remove 'overlay_framework' setting
30 - Start daemon and cat the file again
31 - Confirm the record remains in Books framework
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
misc/cronjobs/staticfines.pl is calling output_pref() but it is missing from module import :
use Koha::DateUtils qw( dt_from_string );
Test but running staticfines.pl
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Update the preference name from 'Borrower' to 'Patron' and correct
spelling of receive.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
To test:
1. APPLY PATCH and restart all
2. Run the update_localuse_from_statistics.pl script without a confirm flag, nothing happens.
3. Run the script and add the --confirm flag, stuff happens.
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
To test:
1. Run the update_totalissues cron, the holds queue is updated.
2. APPLY PATCH, restart services
3. Run update_totalissues cron again, this time the holds queue should always be skipped.
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Koha::Objects depends on Koha::DateUtils, which depends on C4::Context,
which depends on Koha::Config::SysPrefs, which depends on Koha::Objects
Apart from the circular dependency, the dependency on C4::Context alone
is problematic as it loads a bunch of modules that are not needed at all
in Koha::Objects (YAML::XS and ZOOM for instance).
As Koha::Objects is used as a base for a lot of modules, we should take
care to only load the minimum required.
This patch removes uses of Koha::DateUtils from Koha::Objects.
It was only used in Koha::Objects::filter_by_last_update
filter_by_last_update now requires that the 'from' and 'to' parameters
must be DateTime objects. Previously it would also allow date and
datetime strings. This possibility was only used in two places:
* misc/cronjobs/cleanup_database.pl
* tools/cleanborrowers.pl
Now they call dt_from_string first and pass a DateTime object to
filter_by_last_update
Test plan:
1. Run `perl -cw Koha/Objects.pm`. It should only say:
"Koha/Objects.pm syntax OK" without warnings
2. Run `prove t/db_dependent/Koha/Objects.t`
3. Verify that misc/cronjobs/cleanup_database.pl works as before,
especially with the options --pseudo-transactions,
--pseudo-transactions-from and --pseudo-transactions-to
4. Go to Tools » Batch patron deletion and anonymization, check "Verify
you want to anonymize patron checkout history" and enter a date in
the text input below. Then click Next and verify that the correct
count of borrowers is shown. Click on the "Finish" button and verify
that the circulation history has been correctly anonymized
See also bug 36432
Signed-off-by: Tadeusz Sośnierz <tadeusz@sosnierz.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Add an unallocated option to CreateQueue and pass through as needed
Avoid deletion of the tmp_holdsqueue, and only check holds
and items that are not currently matched
A future hold with a higher priority will still fail here - because the
item may already be assigned, but on next change to the biblio it would
be corrected
To test:
1) Apply both patches
2) Enable RealTimeHoldsQueue and set HoldsQueueSkipClosed to "open"
3) Add a holiday to the calendar for all libraries for today, visit:
/cgi-bin/koha/tools/holidays.pl
-- Click today's day on the calendar and pick "Holiday repeated every same day of the week"
-- Click "Copy to all libraries". Hit "Save.
4) Place a biblio-level hold on a biblio record and set the pickup location to a library that has available copies, visit:
-- /cgi-bin/koha/reserve/request.pl?biblionumber=76&borrowernumber=51
-- Click the first "Place hold" button to place the biblio-level hold.
5) Verify that that hold got added to the holds queue, visit:
/cgi-bin/koha/circ/view_holdsqueue.pl?branchlimit=&itemtypeslimit=&ccodeslimit=&locationslimit=&run_report=1
6) Place a biblio-level hold on a biblio record where there are no other holds and copies are available at another location, but not the pickup location, visit:
-- /cgi-bin/koha/reserve/request.pl?biblionumber=437&borrowernumber=51
-- On the "pickup at" dropdown, pick something else other than "Centerville", e.g. "Fairfield".
-- Click the first "Place hold" button to place the biblio-level hold.
7) Check the holds queue again, notice that this 2nd hold was not added to the queue:
/cgi-bin/koha/circ/view_holdsqueue.pl?branchlimit=&itemtypeslimit=&ccodeslimit=&locationslimit=&run_report=1
8) Run the updated cronscript:
perl misc/cronjobs/holds/build_holds_queue.pl --force --unallocated
9) Notice nothing changed in the holds queue, visit:
/cgi-bin/koha/circ/view_holdsqueue.pl?branchlimit=&itemtypeslimit=&ccodeslimit=&locationslimit=&run_report=1
10) Remove the holiday we created previously, visit:
/cgi-bin/koha/tools/holidays.pl
-- Click today's day on the calendar and pick "Delete this holiday"
-- Click "Copy to all libraries". Hit "Save.
11) Run the updated cronscript:
perl misc/cronjobs/holds/build_holds_queue.pl --force --unallocated
12) Confirm the second hold is added to the holds queue, visit:
/cgi-bin/koha/circ/view_holdsqueue.pl?branchlimit=&itemtypeslimit=&ccodeslimit=&locationslimit=&run_report=1
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Sometimes it would be better for process_message_queue.pl to stop if a plugin hook fails rather than continue processing. It would be nice if that was a command line option.
Test Plan:
1) Install any plugin with a before_send_messages hook
2) Modify the plugin, add a 'die;' statement at the start of the
before_send_messages method of the plugin.
3) Run process_message_queue.pl as usual
4) Note the exit code is 0
5) Run it again with the new -e setting
6) Note the exit code is 1
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch adds a clarification to writeoff_debts.pl to make it clear that --category-code can't be used as the only filter when running the script. If this is the case, the script just displays the help menu as it is expecting one of the other filters
Test plan:
1) Run perl misc/cronjobs/writeoff_debts.pl --category-code TEST --confirm
2) Observe that the help menu is displayed instead of running the script
3) Check the help menu for the text added in this patch
WNC amended patch - added same warning in the option section
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>
Signed-off-by: Andrew Fuerste Henry <andrewfh@dubcolib.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This script allow to deduplicate authorities automatically.
Script is in misc/maintenance/
It works this way:
1) authorities are fetched from the database. You can limit fetched
results by authtypecode, or directly by specifying WHERE clause
2) for each authority:
2.1) build a Zebra query using the 'search_form' for the heading
2.2) run the query, retrieve the results
2.3) among duplicates, choose the one we want to keep (use
--choose-method option).
2.5) use C4::Authorities::merge to merge authorities
3) delete the merged authorities
Use --help for more informations on options
To be done:
1 - Move to module and cover with tests
2 - Add option to only merge unused authorities
3 - Expand 'ppn' option to be 'control-number' option and allow specifying field
4 - More?
1 & 2 I will attempt - 3 & 4 may be future enhancements
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Sometimes we need to only re-index a subset of our bibliographic data or authorities. Currently this is only possible by enumerating all id (-bn or -ai), which does not work well when indexing eg 100.000 items of a 2.000.000 DB. Re-indexing everything is also overkill.
This patch adds an `--where` flag to misc/search_tools/rebuild_elasticsearch.pl which can take arbitrary SQL (that of course has to match the respective tables) and adds it as an additional param to the resultset to index
To test, start koha-testing-docker with ElasticSearch enabled, for example via `ktd --es7 up
Before applying the patch, rebuild_elasticsearch will index all data:
Biblios:
$ misc/search_tools/rebuild_elasticsearch.pl -b -v
[12387] Checking state of biblios index
[12387] Indexing biblios
[12387] Committing final records...
[12387] Total 435 records indexed
(there might be a waring regarding a broken biblio, which can be ignored)
Auth:
$ misc/search_tools/rebuild_elasticsearch.pl -a -v
[12546] Checking state of authorities index
[12546] Indexing authorities
[12546] 1000 records processed
[12546] Committing final records...
[12546] Total 1706 records indexed
Now apply the patch
Biblio, limit by range of biblioid:
$ misc/search_tools/rebuild_elasticsearch.pl -b -v --where "biblionumber between 100 and 150"
[12765] Checking state of biblios index
[12765] Indexing biblios
[12765] Committing final records...
[12765] Total 50 records indexed
Note that only 50 records where indexed (instead of the whole set of 435 records)
Auth, limit by authtypecode:
$ misc/search_tools/rebuild_elasticsearch.pl -a -v --where "authtypecode = 'GEOGR_NAME'"
[12848] Checking state of authorities index
[12848] Indexing authorities
[12848] Committing final records...
[12848] Total 142 records indexed
Again, only 142 have been indexed.
Sponsored-by: Steiermärkische Landesbibliothek
Sponsored-by: HKS3 / koha-support.eu
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Koha/Illrequests.pm -> Koha/ILL/Requests.pm
Merged:
t/db_dependent/Koha/Illrequests.t
t/db_dependent/Illrequests.t
Into:
t/db_dependent/Koha/ILL/Requests.t
ILL classes file structure is, for the most part, around 7 years old and doesn't follow a strict logic. It's so confusing that some test files exist redundantly.
This housekeeping should help future work in regards to ISO18626 to add Koha as a supplying agency instead of just requesting agency, as is now.
It should also help future housekeeping of moving backend related logic out of the Illrequest.pm into Illbackend.pm (now ILL/Request.pm and ILL/Backend.pm as of this patchset).
It should also help in structuring the addition of a master generic form (see bug 35570)
This patchset will require existing backends to be updated to match the new class names and structure, if they invoke them.
Test plan, k-t-d, run tests:
prove t/db_dependent/api/v1/ill_*
prove t/db_dependent/Koha/ILL/*
Test plan, k-t-d, manual:
1) Install FreeForm, enable ILL module, run:
bash <(curl -s https://raw.githubusercontent.com/ammopt/koha-ill-dev/master/start-ill-dev.sh)
2) You'll have to switch the FreeForm repo to the one compatible with this work, like:
cd /kohadevbox/koha/Koha/Illbackends/FreeForm
git checkout reorganize_ILL
3) Do some generic ILL testing:
3.1) Create a request
3.2) Add a comment to a request
3.3) Edit a request
3.4) Edit a request's item metadata
3.5) Confirm a request
3.6) List requests
3.7) Filter requests list using left side filters
4) Install a metadata enrichment plugin:
https://github.com/PTFS-Europe/koha-plugin-api-pubmed
4.1) Create an ILL batch and insert a pubmedid like 123
4.2) Add the request and finish batch
5) Verify all of the above works as expected
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
In situations in which you are not familiar with all the Koha settings,
and table structure, the fact this script just fails telling there's a
broken FK is just not practical.
We should capture those exceptions and display a useful message instead.
This script does that. It adds some validations and some exception
handling too. It prints a nice message about the bad value the user
passed, and the valid values too!
To test:
1. Run this on a fresh KTD:
$ ktd --shell
k$ perl misc/devel/create_superlibrarian.pl \
--userid tcohen \
--password tomasito \
--cardnumber 123456789 \
--categorycode POT \
--branchcode ATO
=> FAIL: It explodes with a MySQL exception message!
2. Apply this patch
3. Repeat 1
=> SUCCESS: It tells you which value is wrong and what values you can
pick to make the command work
4. Pick a valid value, and repeat
=> SUCCESS: Now the other value is wrong, a nice message is displayed!
5. Fix with a valid value and repeat
=> SUCCESS: Patron created!
6. Sign off :-D
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Introducing $orders->filter_by_obsolete and $orders->cancel.
Test plan:
Run t/db_dependent/Koha/Acquisition/Orders.t
Create basket with few orders. Remove some biblio records.
Run acq_cancel_obsolete_orders.pl -c
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Test Plan:
1) Test importing a patron with the "from the current membership expiry date" option,
note it does not work
2) Apply this patch
3) Restart all the things!
4) Re-test, note the expiration was renewed from the patron's current
expiration date!
Signed-off-by: David Nind <david@davidnind.com>
Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
When misc/devel/install_plugins.pl does not find any plugins, it prints
the list of pluginsdir, but with a literal \n separating the dirs, and
no newline at the end.
To test:
- Edit koha-conf.xml and add a second entry for <pluginsdir>, so there
are two entries. The second one could just be a copy of the original.
- Run "perl misc/devel/install_plugins.pl"
- Note the output looks something like this:
No plugins found
pluginsdir contains:
/var/lib/koha/kohadev/plugins\n/var/lib/koha/kohadev/pluginsroot@kohadevbox:koha(master)$
- Apply the patch and run the script again. Output should be:
No plugins found
pluginsdir contains:
/var/lib/koha/kohadev/plugins
/var/lib/koha/kohadev/plugins
root@kohadevbox:koha(master)$
- Sign off
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
I lost too much time trying to know what the translation script outputs
sh: 1: Syntax error: end of file unexpected
To recreate:
touch 'koha-tmpl/intranet-tmpl/prog/en/modules/<' (yes, don't ask!)
koha-translate -i de-DE --dev kohadev
or
koha-translate -u de-DE --dev kohadev
And you get the shell error without any ideas what's happening!
With this patch applied you will not get any errors because the quote
will actually copy the file (what is expected).
But without the quote the cp command will fail and the explanation will be
displayed
sh: 1: Syntax error: end of file unexpected
Cannot copy /kohadevbox/koha/koha-tmpl/intranet-tmpl/prog/en/modules/< to /kohadevbox/koha/koha-tmpl/intranet-tmpl/prog/de-DE/modules/< at /kohadevbox/koha/misc/translator/tmpl_process3.pl line 385.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Since it is not up to date, we should remove it.
The folder structure changed and this would need digging into SELinux to update that side as well.
In other words, remove this until someone finds time do it. But dont let it here in its current outdated state.
No test plan here.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
To test:
1. APPLY PATCH
2. Set up circ rules to allow for auto renewals.
3. Set AutoRenewalNotices to 'according to message prefs'
4. Make a branch specific version of the the AUTO_RENEWAL notice.
5. Find a patron belonging to that branch and set the message prefs to allow for a AUTO_RENEWAL. But no digest option!
6. Check some things out and make sure they are eligble for auto renewal.
7. Run the cron.
8. Make sure the patron gets the branch specific notice.
9. Delete the branch specific notice.
10. DO 6-7 again.
11. Make sure they now get the all library notice.
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Tidied one line.
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
In misc/export_records.pl add an option to add see-from headings (from authorities 4xx) into bibliographic records export.
Like it is done during record indexing.
Test plan :
1) Choose a biblio record having a field (for example 650) linked to an authority with a see-form.
2) Export this record without see-from headings :
misc/export_records.pl --starting_biblionumber=X --ending_biblionumber=X --filename /tmp/record_without.xml --format xml
3) Export this record with see-from headings :
misc/export_records.pl --starting_biblionumber=X --ending_biblionumber=X --filename /tmp/record_with.xml --format xml --embed_see_from_headings
4) Compare /tmp/record_without.xml and /tmp/record_with.xml
=> you should see two 650, one with main heading and one with see-from heading
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
To test:
1. APPLY PATCH
2. Create a CSV file with a few lines that contain at least firstname, surname, cardnumber
3. Turn on "CronjobLog".
4. Run the import patron script. I did this: perl /kohadevbox/koha/misc/import_patrons.pl --file 'test1.csv' --matchpoint cardnumber -c
5. Check the action logs for cronjobs.
6. You should see info. about the import patron script.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Three routines in Plugins got the verbose parameter on 35507.
We can refine this a bit further.
The idea here is report when you are installing plugins but not
report when just calling plugins (flooding logs).
[1] GetPlugins: Most callers do not expect (or check) results for
failing plugins. This patch makes GetPlugins only return
errors when passing the *errors* flag (in 2 cases).
[a] The misc/devel script prints warnings now using verbose,
so does not need the errors flag anymore.
[b] plugins/plugins-home is the only case left. Tiny adjustment
to keep current behavior. Fixed colspan in template.
Does not need verbose in favor of 'errors' (passed to
template).
[c] For most calls we do not want verbose. New default is 0.
[2] InstallPlugins
[a] Disabled verbose in plugin-upload. Not really needed.
Added a FIXME; we need to improve individual install.
[b] misc/devel: No warnings anymore when calling InstallPlugins
after GetPlugins.
[3] get_enabled_plugins
[a] Plugins->call does not need verbose.
[b] Plugins->feature_enabled does not need it too.
Test plan:
[1] See previous plan. With TestMR data but without patch, run
misc script and go to plugins-home. Do you see load errors on
commandline or form?
[2] Run plugins/plugins-upload (uploading just some file is good
enough); verify that you do not see TestMR lines in logfile.
[3] Run t/db_dependent/Koha/Plugins/Plugins.t for the additional
test on verbose and errors flag.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: AFHDubCoLib <andrewfh@dubcolib.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This script takes a date parameter in SQL format and pseudonymizes all statistics
found before this date.
Only values that can be found will be added, i.e. no deleted patron or item info
will be present.
Additionally - the values stored will be the current values from patrons and items, so
some info will be approximate, much as it is when joining from the statistics table for reporting.
To test:
1 - Perform some issues/returns/renewals/on-site checkouts
2 - Make sure Pseudonymization is disabled
3 - perl misc/maintenance/pseudonymize_statistics.pl
4 - Script ends and reports that preference is not active
5 - Enable the pref, and choose some borrower and item fields
NOTE: See bug 28911 if you need a bcrypt key for your koha-conf.xml
6 - perl misc/maintenance/pseudonymize_statistics.pl
7 - sudo koha-mysql kohadev
8 - SELECT * FROM pseudonymized_transactions
9 - Confirm data is correctly stored
10 - DELETE FROM pseudonymized_transactions;
11 - UPDATE statistics SET datetime = '2023-01-01 00:00:00';
12 - perl misc/maintenance/pseudonymize_statistics.pl -b "2022-12-31 23:59:59";
13 - SELECT * FROM pseudonymized_transactions;
14 - Confirm no entries were made
15 - Select different options in Pseudonmyization prefs, including borrower attributes
This wil require defining an attribute that can be kept for pseudonymization
16 - Confirm options are correctly pseudonymized
Signed-off-by: AFHDubCoLib <andrewfh@dubcolib.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Items are no longer embedded in the MARCXML and because of this the MARC data
does not need to be saved once more after changing record items data.
The "defer_marc_save" is no longer needed since bulkmarcimport.pl was the only
place this option was utilized in order to resave MARC data after possibly changing
items data. There is also a bug bulkmarcimport.pl where the record data is re-saved
without stripping items if duplicate items are found and the dedup barcodes
option is enabled that is resolved by this change.
This change enables MARC Overlay rules to be enabled for bulkmarkcimport.pl as using
the defer_marc option would previously effectively bypass any defined rules.
To test
1) Apply patch 29440 (which this depends on), but no not yet apply this
patch
2) Remove comments around "bulkmarcimport: _("bulkmarcimport.pl"),"
in koha-tmpl/intranet-tmpl/prog/en/modules/admin/marc-overlay-rules.tt
line 463
3) Enable MARC overlay rules and add a rule (with source => "*" or
'bulkmarcimport') to protect some field.
4) Import a biblio using the bulkmarcimport script. Overlay rules will
only be applied on updates so a match condition matching the
record in Koha when bulkmarcimport.pl is run the next time for the same
record needs to be supplied (for example -match "control-number,001")
5) Edit the imported biblio and change the value of the protected field
6) Run bulkmarcimport with the same parameters as in 4) and verify the
the field that should have been protected has been overwritten
7) Revert changes for line 463 in marc-overlay-rules.tt (or will result in
conflict applying the patch), apply the patch and repeat steps 4-5,
verify that the field now was protected from being overwritten.
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This variable actually contains ModBibio options and
ModifyBiblioMarc accepts a subset of these
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
The current behavior for when importing authorities, when match
option is enabled, is to insert as a new authority fetching from db
fails. This is both a bit confusing and not consistent with the biblio
case where an error is generated. Also clean up error and log messages
so both messages/errors and when messages/errors are issued are more
consistent for the biblio and authority case.
Signed-off-by: Frank Hansen <frank.hansen@ub.lu.se>
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
To test:
1) Import biblios using ./bulkmarcimport.pl -v -b -file biblios.mrc -insert -update -c=MARC21 -match "control-number,001" -n=1 -l "/tmp/import.log"
2) Verify in /tmp/import.log that one biblio has been inserted
3) Import again without the "-n" option, verify that one biblio has been
updated and the rest inserted
4) In the staff interface search the catalog for some string appearing in
the imported biblios to verify records have been indexed
5) View/edit some of the biblios the staff interface to very have been
properly imported
6) Import authorities using ./bulkmarcimport.pl -v -m=MARCXML -a -file authorities.xml -insert -update -c=MARC21 -keepids="024a" -match "heading-main,100a" -l "/tmp/import.log" -yaml="/tmp/ids.yml"
7) Go to the authorities page and find the imported authority by for example search for a
400 or 100 field value. Verify that the 001 field of the incoming record has been moved to 024a.
Also check that the value of 024a appear as a key in /tmp/ids.yml.
8) Edit the authorities.xml file and change the value of field 400a
9) Import authorities again using the same options
10) Verify that the 400a field of the authority in Koha has not been
updated (the import was skippped since revision did not change)
11) Edit authorities.xml, replace the value of field 005 in the file with field 005
of the authority in Koha incremented by one
12) Import again with the same options
13) Verify that the 400a field of the authority in Koha now have been
updated
Signed-off-by: Michał Kula <148193449+mkibp@users.noreply.github.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
See bug 33898 comment6.
Test plan:
Based on comment12: Stage MARC import and manage. Look at the same
time to the results of ps aux|grep Z. Verify that the lines with
[background_jobs] <defunct> disappear within 10 seconds.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Actually, the module is not even needed anymore here.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Add forgotten module
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Log (warn) if the job will be processed later, but add a debug however.
Have a specific log for bad status
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
If a frame cannot be correctly processed (most probably because the body
is not valid JSON) then we are not acking or nacking the frame and the
worker is stuck.
In this specific case we should nack without requeuing the frame.
NOTE that requeue must be 'true' or 'false', not 1 or 0, or the default
'true' will be used.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Here I add 500 ms. In my testing with the 1s sleep from the test
plan, I might see one or two 'not found' lines. Obviously things
depend on the time needed before the txn commits. But it will
reduce a flood of these messages.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch makes the worker reject the incoming frame for putting the
message back in the queue, in the event the job id doesn't exist yet.
Which is the case when some actions are being triggered inside a
transaction which hasn't been commited to the DB yet.
To test you will need 3 KTD shells
(a) mysql:
$ ktd --shell
k$ sudo koha-mysql kohadev
(b) logs:
$ ktd --shell
# for restarting the worker and looking at the logs
k$ sudo koha-worker --restart kohadev ; tail -f /var/log/koha/kohadev/worker-*.log
(c) running the test:
$ ktd --shell
1. Have (a), (b) and (c) terminals ready
2. On (c), run:
$ perl -MKoha::Database -MKoha::BackgroundJob::BatchUpdateBiblioHoldsQueue -e 'Koha::Database->schema->txn_do( sub { Koha::BackgroundJob::BatchUpdateBiblioHoldsQueue->new->enqueue({ biblio_ids => [ 1 ] }); sleep 1; } );'
=> FAIL:
* (b) shows (once) an error about a job not existing
3. On (a) run:
> SELECT * FROM background_jobs;
=> FAIL: Notice the job ID mentioned on 2 stands as 'new'.
4. Apply this patch
5. Ctrl+c on (b), and re-run to launch the worker with the patch applied
6. Repeat 2
=> SUCCESS (partial): The error about the job not existing is displayed
many times
7. Repeat 3
=> SUCCESS: The job ID mentioned on 6 stands as 'finished'.
8. Sign off :-D
Discussion:
* The `requeue` header I added is correct, but it is the default
behavior anyway. I prefered to make it explicit, though.
* To avoid that bunch of retries, we should requeue with some delay. I
didn't manage to make it work (yet) but there's a 'delay' plugin for
rabbit [1]. We already install the 'stomp' plugin in
koha-common.postinst. But this plugin requires downloading it. Which
would require further investigation.
* As Nick and Marcel pointed, we need to revisit the whole architecture,
the need of a MQ (DB polling wouldn't have this problem), etc. But
that's for another place.
[1] https://hevodata.com/learn/rabbitmq-delayed-message/#:~:text=To%20delay%20a%20message%2C%20the,to%20queues%20or%20other%20exchanges.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
GET svc/authentication will return a CSRF token in a response header
POST svc/authentication requires a CSRF token which can be sourced
from the response header of GET svc/authentication or some other
place like the meta element on a HTML page
Note: misc/migration_tools/koha-svc.pl is a simple script which
can be used to practically evaluate svc/authentication and svc/bib
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
To test:
1 - Install the Kitchen Sink plugin
2 - Restart all
3 - Enable 'CronjobLog' system preference
4 - perl misc/cronjobs/plugins_nightly.pl
5 - Note you see on the command line 'Remember to clean the kitchen' - this indicates the plugin cron ran
6 - Tools->log viewer, select 'cronjob' and view
7 - Note you only see 'plugins_nightly.pl' Run and End lines
8 - Apply patches
9 - perl misc/cronjobs/plugins_nightly.pl
10 - View logs agian
11 - Note you now see Run and End lines for 'Koha::Plugin::Com::ByWaterSolutions::KitchenSink'
12 - Confirm they look like the other lines
13 - Edit KitchenSink.pm and add 'die "Kittens";' to the cronjob nightly
14 - perl misc/cronjobs/plugins_nightly.pl
15 - View logs, confirm there is a FAILED error message for the KitchenSink cron
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
When running the plugins_nightly.pl cronjob, we should record the plugins that have a nightly method, logging the start and end of each plugins routine
Test Plan:
1) Enable CronjobLog
2) Install a plugin with a nightly cronjob ( e.g.
https://github.com/bywatersolutions/koha-plugin-book-list-printer )
3) Run plugins_nightly.pl
4) Note new entries in the cronjob viewer for the start and end of the
plugin's nightly cronjob run
5) Edit the plugin, add a line like "die 'this is a test';" to the
plugin's nightly cronjob
6) Run plugins_nightly.pl
7) View the action logs, not the log for the error you added!
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch corrects an error in the documentation of the
writeoff_debts.pl script. The --type parameter accepts debit type codes,
not credit type codes.
To test:
- Run ./misc/cronjobs/writeoff_debts.pl
- Make sure the sentence for --type makes sense, grammar and spelling
are correct
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Removes a FIXME line mentioning bug 8000 that implemented
SendAllEmailsTo. But the note was added to the "From" line
and the "To" address is taken from a command line parameter.
To test:
* Review change carefully
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch updates the parameter name to more clearly define what it's
used for and also adds missing documentation.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Because some notices are mandatory, some libraries would like to allow patrons to opt-in to phone notices. The request is to be able to decide if a patron should get a notice based on the value of a given field ( usually sort1 or sort2 ).
Test Plan:
1) Apply this patch
2) Enable TalkingTechItivaPhoneNotification
3) Set up some overdue items for a patron
4) Enable phone notices for overdues in the notice status triggers
5) Run the outbound script for OVERDUE with -t OVERDUE
6) Note the output the output contains a line for that patron
7) Set the sort1 field for that patron to TEST
8) Run the script again, adding in the option "-i sort1:TEST"
9) Note the ouput no longer contains that patron!
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Enable librarians in the staff client to control when unaccepted list
invites will be removed from the database.
Test plan:
1. Apply patches
2. Update database
sudo koha-shell <instance>
cd installer/data/mysql
./updatedatabase.pl
3. Confirm the PurgeListShareInvitesOlderThan syspref = 14
4. Run cleanup_database.pl with --list-invites parameter defined:
sudo koha-shell <instance>
cd misc/cronjobs
./cleanup_database.pl --list-invites 10 -v
5. Confirm the output contains:
"Purging unaccepted list share invites older than 10 days."
6. Set PurgeListShareInvitesOlderThan syspref = 15
7. Run cleanup_database.pl without the --list-invites parameter:
sudo koha-shell <instance>
cd misc/cronjobs
./cleanup_database.pl -v
8. Confirm the output contains:
"Purging unaccepted list share invites older than 15 days."
9. Empty PurgeListShareInvitesOlderThan syspref
10. Run cleanup_database without the --list-invites parameter:
sudo koha-shell <instance.
cd misc/cronjobs
./cleanup_database.pl
11. Observe you get the following output (along with documentation on all
parameters of the script):
"You did not specify any cleanup work for the script to do."
12. Run the cleanup_database with no days defined for the --list-invites
parameter:
sudo koha-shell <instance>
cd misc/cronjobs
./cleanup_database.pl --list-invites -v
13. Confirm the output contains:
"Purging unaccepted list share invites older than 14 days."
Sponsored-by: Catalyst IT, New Zealand
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
The es background indexer is designed to combine background jobs when started based on the 'batch_size' option.
While this is helpful for combining individual updates, it can be problematic when there are several large batch modifications, or when worker has stopped and is restarted.
This patch uses the same logic as in the indexer to split the chunks that are sent directly for indexing.
To test:
1 - Follow test plan on previous patch
2 - Confirm items are correctly indexed and jobs marked
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This parameter is actually --log-module.
Test plan :
Run misc/cronjobs/cleanup_database.pl -h
Confirm you see --log-module
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Renamed some vars. Removed unused $success.
Extended die statement.
Added Modern::Perl.
Added verbose parameter.
Added some logic around the confirm parameter. Making it possible
to run the script in 'dry' mode.
Print totals at the end.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Variable my $amount was missing. Also option
amount has to be declared as integer and
option confirm is no longer string.
Sponsored by: Koha-Suomi Oy
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
[EDIT] Run perltidy to resolve one line.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
If your library wants to debar all Borrower who haven't paid
their fines by the end of the year, this script will do that
trick :)
You can give the message from a file if the cronjob runner
doesn't deal with quotes, or as a command-line parameter.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Bug 15157: Update script
This patch updates debarrBorrowersWithFines.pl script to match
changes made in bug 15156.
To test:
1. Have patron(s) with unpaid fines
2. Run e.g debarrBorrowersWithFines.pl --confirm -m "This is a description of you bad deeds"
(test other options too)
3. Confirm patron(s) with fines has been debarred with the explanation
Sponsored-by: Koha-Suomi Oy
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Bug 15157: Modernise and Update for bug 15156
This patch updates the script to use filter_by_amount_owed, renames
it to debar_patrons_with_fines.pl and moves it to the cronjobs
directory whilst also adding a copyright notice and POD.
We could add a series of options to the script to allow more fine
grained control.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
[EDIT] Run perltidy to resolve three lines.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Add to script misc/cronjobs/runreport.pl quote char arg (only for CSV).
See perl doc https://metacpan.org/pod/Text::CSV#new
This patch also adds missing '--separator' in POD.
Test plan :
1 - Write a SQL report
2 - perl misc/cronjobs/runreport.pl --format csv 1 (or correct report number)
3 - Note you get double quotes
4 - Apply patch
5 - Repeat #2 - no change
6 - perl misc/cronjobs/runreport.pl --format csv --quote "'" 1
7 - Now it is single quote delimited
8 - perl misc/cronjobs/runreport.pl --format tsv --separator "'" 1
9 - Error is reported, you cannot set quote unless csv
10 - Try empty string as quote
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This makes sure that we don't change the line breaking for either
HTML nor plain text print notices.
For plain text, the <pre> is used to keep the line breaks as defined
in the notices template.
For HTML we require on the line breaks added by block elements in the
HTML notice, not adding any additional ones like before.
This gives the full control of the formatting to the notice editor.
To test:
* Make sure your patron doesn't have an email address
* Make sure to check the hold filled notice in messaging preferences
* Place a old on any item
* Checkin the item
* Confirm hold
* Verify notices tab shows the Hold (print) notice
* ./misc/cronjobs/gather_print_notices.pl ./ --html
* Verify the generated file has the HTML output wrapped in pre.
* Reformat the HOLD print notice to be HTML by setting the checkbox.
Mmake sure to add some empty lines for (they should not print later)
and some <br>, <p>, <div> - They should be reflected in the notice later.
* Repeat test.
* All should be well :)
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch makes the trivial change of making the linker script rely on
LinkerRelink instead of CatalogModuleRelink, which is confusing and
incorrect (according to both sysprefs descriptions).
To test:
1. Verify that relinking is now tied to the right syspref
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Phil Ringnalda <phil@chetcolibrary.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
These scripts are outdated and no longer needed in
the codebase.
To test:
* Verify both perlmodule_* scripts have been removed from misc/
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
So that pickup delay can have a different value per patron category,
item type or branch.
To test:
1) Update database, restart services
2) Set ReservesMaxPickUpDelay syspref (if not already set)
3) Edit your circulation rules and set a value under 'Holds pickup
period (day) that is DIFFERENT from ReservesMaxPickUpDelay. Set a few
different numbers for different branches as well.
4) Place a hold on a biblio from the staff client.
5) Check in an item from that biblio and confirm the hold as waiting
6) Confirm the expiration date is calculated using the 'Holds pickup
period' value instead of the ReservesMaxPickUpDelay syspref
7) Revert the waiting status and delete the hold
8) Re-place the hold on the biblio on the OPAC. Notice that when you
change the pick up location, the number of days in the pickup message
below the dropdown changes based on the circ rules.
9) Create a holiday with a date that will overlap with the 'Holds pickup
period'
10) Check in an item from that biblio and confirm the hold as waiting
11) Confirm the expiration date is calculated using the 'Holds pickup
period' value AND considers the special holiday
12) Confirm tests pass t/db_dependent/Holds/WaitingReserves.t
13) Test Talking Tech:
13a) Enable TalkingTechItivaPhoneNotification
13b) Go to Tools -> Notices & slips. Add content to the HOLD phone
(itiva) notice.
13c) In your terminal, run perl
/path/to/koha/misc/cronjobs/thirdparty/TalkingTech_itiva_outbound.pl -o
~/itiva.tmp -w 0 --type=RESERVE
Sponsored-by: Catalyst IT
Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kelly <kelly@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch adds a new 'send_empty' option to runreport.pl
To test:
1 - Create a report in Koha that will not return any results:
SELECT barcode FROM items WHERE 1=2
2 - perl misc/cronjobs/runreport.pl 1
3 - Output is: NO OUTPUT: 0 results from execute_query
4 - perl misc/cronjobs/runreport.pl 1 --send_empty
5 - Output is: no results were returned for the report
6 - perl misc/cronjobs/runreport.pl 1 --send_empty --email
7 - It will die on an email error unless you have SMTP configured - this si good, it means we sent an email
8 - Bonus points: Test on a system that can correctly send emails, confirm it works :-)
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch adds support for passing category_code as a limiter for the
writeoff_debts script.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
The cronjob always logs a dry run even on a live harvest and the end date parameter was not working correctly. This patch addresses both issues
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This commit is a squash of the following:
SUSHI harvesting process in the data providers class:
* Builds the URL query and requests the SUSHI service endpoint
* Parses the JSON response and builds the csv COUNTER file and adds it to counter_files table
Usage statistics data processing:
* When a counter_files entry is stored, CounterFile.pm will:
* Parse the csv COUNTER file and
* Add a usage_titles entry for each unique title in the COUNTER file
* Add the title's respective erm_usage_mus (monthly usage) entries, repeating for each metric_type
* Add the title's respective erm_usage_yus (yearly usage) entries, repeating for each metric_type
Harvesting cronjob;
'Run now':
* API endpoint to start the harvesting process of a data provider
* Button in the data providers list to run the harvesting process for each data provider upon clicked
ERM SUSHI: Background job
Job progress is updated to total amount of usage titles after retrieving
the response from SUSHI;
Job warning and success messages are added accordingly
Redundant duplicate titles will not be added
Redundant duplicate monthly and yearly usage statistics will not be added
Data provider harvest background job harvests once per report_type
Enqueue one background job for each report_type in the usage data provider
Update the way we measure progress in the background job.
It now uses the COUNTER report body rows instead of SUSHI response results.
We're now incrementing and showing the number of skipped mus, skipped
yus, added mus and added yus
There's a bug in the way we calculate yus
Updates to background job progress bar - Depends on 34468
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
automatic_renewals.pl does some data compilation for digest notices that
depends on the error value returned from attempt_auto_renew. However, on
successful renewal $error is undefined. Add additional checks on value
of $success and definedness of $error to compile data accurately and
avoid warns for undefined variable
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
There is a desire for auto_renewals to treat the final renewal differently. We would like to notify the patron of the final renewal - but not again when the next renewal fails. This patch adds the new return value and tests.
To test:
prove -v t/db_dependent/Circulation.t
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch moves the actual renewal out of the auto_renewals cronjob script and into the object and adds tests. The logic for notices is still handled in the script.
To test:
1 - prove -v t/db_dependent/Koha/Checkouts.t
2 - Add a circ rule with auto_renew checked
3 - Checkout an item to a patron and set due date in the past
4 - Checkout an item to a patron and set due date in the future
5 - perl misc/cronjobs/automatic_renewals.pl -v
6 - Confirm one would be renewed and the other is too_soon
7 - perl misc/cronjobs/automatic_renewals.pl -v --confirm
8 - Confirm the expected issue is successfully renewed
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
According to the timestamp option for bibs record type I added the
timestamp option for authority records.
Timestamp is already present in database on field "modification_time"
Test Plan :
1 - Be sure to have authority record type for easiest test create one
2 - Execute script export_records.pl in your koha/misc directory and
choose a date (example yesterday if you just created an authority
right now).(see export_records.pl -h for help)
3 - Timestamp option has no effect on authority record type
4 - Execute script again but choose the date of tomorrow for example
5 - Same result
6 - Apply this patch
7 - Play again steps 2 and 4
8 - On step 2 you will see only your authorities created today (because
script show you authority has changes since the date you choose in
option) and for step 4 you must see an empty file.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This change explicitly ignores SIGPIPE signals in the background jobs
worker.
Daemons like Starman ignore SIGPIPE so it makes sense to explicitly set this.
Differences in the inner workings of MySQL vs MariaDB client libraries have yielded
different behaviours in automatic reconnections and potentially SIGPIPE handling,
so this helps to make the overall behaviour more consistent.
Test plan:
0. Apply patch and run "restart_all"
1. Go to http://localhost:8081/cgi-bin/koha/catalogue/detail.pl?biblionumber=29
2. Click "Save" > "MARCXML"
3. Go to http://localhost:8081/cgi-bin/koha/tools/stage-marc-import.pl
4. Click "Choose file", choose the MARCXML file, click "Upload file"
5. Click "Stage for import"
6. Note the job is marked as "100% Finished"
7. In a separate window run "docker restart koha-db-1"
8. Repeat steps 3-5 for uploading file and running stage for import
9. Note that the job is marked as "100% Finished" as you'd expect
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Added comment on the SIG PIPE line.
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
We have our own "quote_po" sub that is stating that Locale::PO::quote
is buggy because it does not deal with quoting new lines correctly.
However it seems that it is fixed now.
Ideally we could use Locale::PO::quote everywhere, but it does not
escape tab characters:
$string =~ s/\\(?!t)/\\\\/g; # \t is a tab
This means the following:
msgid "Tabulation (\\t)"
msgstr "Tabulation (\\t)"
become:
-msgid "Tabulation (\t)"
-msgstr "Tabulation (\t)"
And we are seeing the following on Weblate:
https://snipboard.io/BjQmDC.jpg
Note that Locale::PO has not been updated since 2014...
The real problem behind this is that we have 2 methods to quote strings.
At first glance it seems that Locale::PO::quote was not used before, but
with the introduction of the koha-i18n project we will have scripts that
will use Locale::PO->save_file_fromarray, which uses Locale::PO->quote
=> Those scripts will be used on the translation server for post
processing (security reason, marking potential XSS strings as fuzzy).
Test plan:
0. Do not apply the patch
1. gulp po:update --lang LANG # Replace LANG with your favorite language
code
2. git commit -a -m"init PO files"
3. Apply this patch
4. Repeate 1.
5. git diff
=> The change is about the "Tabulation" and "New line" strings from
tools/csv-profiles.tt
6. Translate them (replace the \t and \n with %s) and remove the fuzzy
flag
7. install the template: cd misc/translator && perl translate install
LANG
8. Enable the language, use it and go to the "Nouveau profil CSV" view
=> Notice that the \t and \n are correctly displayed.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
When fetching the record we need to eval in case of exception
To test:
1 - Apply first patch
2 - perl misc/link_bibs_to_authorities.pl
3 - Note it dies on parser error at record 369
4 - Apply this patch
5 - perl misc/link_bibs_to_authorities.pl
6 - The process completes - with warning thrown on record 369
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
This patch simplay adds an eval and a warning to link_bibs_to_authorities to catch any errors while searching and continue linking.
To test:
1 - edit /etc/kohia/sites/kohadev/koha-conf.xml and add to elasticsearch stanza
<request_timeout>0</request_timeout>
2 - perl misc/link_bibs_to_authorities.pl
3 - It dies immediately
4 - Apply patch
5 - perl misc/link_bibs_to_authorities.pl
6 - Now it tries every record, throwing warnings along the way, but completes
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
The before_send_messages plugin hook currently sends all the non-smtp related options with the exception of the --where option. This should be added.
Test Plan:
1) Install the latest version of the kitchen sink plugin (
https://github.com/bywatersolutions/dev-koha-plugin-kitchen-sink/releases
)
2) Apply this patch
3) Run process_message_queue.pl with a --where option
4) Run the command: ./misc/cronjobs/process_message_queue.pl --where "letter_code='test'"
5) Note the output looks like:
Plugin hook before_send_message called with the params: $VAR1 = {
'verbose' => 0,
'where' => 'letter_code=\'test\'',
'letter_code' => [],
'type' => [],
'limit' => undef
};
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Test plan:
Toggle pref TrackLastPatronActivityTriggers (clear it).
And try variations of --active, --inactive to see the warnings.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch does so by default. If you do not want them sorted,
export CANONICAL-0 on the command line.
Test plan:
Copy a staff PO file from misc/translator to test.po
Now run: time misc/translator/po2json test.po > json1
And do: time misc/translator/po2json test.po > json2
Run: diff json1 json2; #They should be the same.
Now: export CANONICAL=0
And run: time misc/translator/po2json test.po > json3
And again: time misc/translator/po2json test.po > json4
And run: diff json3 json4; # Lots of changes
Remove the created cruft. And signoff :)
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
[1] Comment in Patron->is_active sharpened.
[2] Patron->update_lastseen
return if $cached && $cached eq $now->ymd;
Should be
return $self if $cached && $cached eq $now->ymd;
Consistent return value.
[3] Typo if there are not TrackLastPatronActivityTriggers enabled
[4] Typo TrackLastPatronActivityTriggers must be contain
[5] Typo patron is does .. option => performs an action
[6] Tidy on dbrev to fix
The file is less tidy than before (bad/messy lines before: 0, now: 4)
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch removes the older PatronLastActivity preference opting to
replace it by the new PatronLastActivityTriggers preference. The feature
can now be disabled by simply not selecting any triggers
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch adds clarifications to the last_seen option in delete_patrons
to clarify that TrackLastPatronActivity and
TrackLastPatronActivityTriggers system preferences act as a couplet.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
To test:
1. In KTD use grep -rn --exclude=*.po "datexpiry" *
2. Notice that the four locations below are showing the typo:
1. git add koha-tmpl/intranet-tmpl/prog/en/modules/tools/import_borrowers.tt
2. git add misc/release_notes/release_notes_18_05_00.html
3. git add misc/release_notes/release_notes_18_05_00.md
4. git add t/db_dependent/Koha/Patrons/Import.t
3. Apply patch. Repeat step 1.
4. Notice that no results show. All instances of the typo have been fixed to dateexpiry.
5. Sign off and have a great day :)
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patchset made it possible for 'too_soon' to be returned instead of 'auto_too_soon' - we just need to take that into account in the error listing
To test:
1 - Set circ rule to no renewal before 2, no auto renewal before 3
2 - Checkout 3 items with auto renew, due tomorrow, in 2 days, in 3 days
3 - perl misc/cronjobs/automatic_renewals.pl -v
4 - Confirm 1 can be renewed, 1 is too soon, 1 is auto too soon
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This is a kind of copy-and-paste variant of background worker.
We probably could do better than having two scripts here ;)
Test plan:
See former test plan. Apply it to ES indexing.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Removed queue from query at MQ side. Discussed on IRC.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
MQ mode: The worker has seen that MQ is running and consumes messages
instead of polling db only.
Test plan:
Stop worker for long tasks.
Make sure that Rabbit MQ runs.
Stage a file. (This adds a long task.)
Goto staff view of jobs and cancel this job.
Check if job is still in MQ with rabbitmqctl list_queues.
Now start worker for long tasks.
Check if job is gone in MQ with rabbitmqctl list_queues.
And check logfile for the adjusted warning like:
[WARN] Job 5 not found, or has wrong status/queue main:: /usr/share/koha/misc/workers/background_jobs_worker.pl (134)
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Removed queue from query at MQ side. Discussed on IRC.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
The PO files are not sorted when we update them which leads to
unnecessary changes that are commited: hard to see differences and
make git index grow superfluously.
Test plan:
0. Do not apply this patch
1. gulp po:update --lang es-ES
2. git commit -a -m"First PO update"
3. Run again the gulp update command
4. git diff
=> You have a lot of changes generated here, the po:update is not
idempotent.
5. Apply this patch
6. Run the gulp update command
7 git commit -a -m"PO update after 34959"
8. Run the gulp update command
9. git diff
=> No changes are generated
Note that this patch will all the entries by files, and per line
numbers.
It fixes a bug in some condition, where we add information/context about
the string. For instance search for "For the first occurrence" in the
file. Prior to this patch this was not correct, we didn't add info about
the first occurrence (but whichever in the list).
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Nothing special here, this sub is not used and we can remove it.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch add 'patron home library' option to the OverdueNoticeFrom preference. When it's selected notification messages for overdues will be bundled and sent from the borrowing user's home library
To Test
1. Create 3 libraries B1, B2 and B3
1.1. Go to Koha administration, click on Libraries
1.2. Click New library
1.3. Fill the Library Code (ex. B1), Name(ex. B1) and email(ex. b1@test.com) fields.
2. Create a user belonging to B1
2.1. Go to patron module, click on New patron and patron
2.2. Fill the Surname,Card number,Primary email,Card number field and select B1 as Library.
3. Create 2 notices : the first belongs to B2 and the second belong to B3
3.1. Go to Cataloging module and click on new
3.2. field the required field (000 subfield 0, 003 subfield 0, 005 subfield 0, 008 subfield 0, 040 subfield c,245 subfield a, and 942 subfield c )
3.3. Click on save
3.4. In the item form, Select Home library = B2 and Current library = B2 and type a Barcode.
3.5. Repeat 3.1 3.2 3.3 3.4 to create a second notice belong to B3
4. Have (or create) a default notice for all branches in Notices and Slips Module
5. Make sure you connected to branch B2 or Switch to branch B2
6. checkout item I2 (belonging to B2) to the user created in step 2 (specifying as due date the day before (1 day overdue))
7. switch to branch B3
8. checkout item I3 (belonging to B3) to the user created in step 2 (specifying as due date the day before (1 day overdue))
9. make sure you have defined 'Overdue notice/status triggers' for that patron category, ie with Delay=1 so that 'Overdue Notice' is selected and 'Email' is checked.
Note: we have a user belong to B1, two item: I2 belong to B2, I3 belong to B3 and the user has borrowed I2 from B2 and I3 from B3.
10. Test
10.1. Search for OverdueNoticeFrom in System preferences and select item home library
10.2. from cli run 'perl ./misc/cronjobs/overdue_notices.pl'
=> Two new entries will be generated in message_queue DB table, coming from different branches that have the SAME list of overdue items.
=> The sender address is the item home library address for each entry.
11. Apply the patch
12. Search for OverdueNoticeFrom in System preferences
=> There is now 'patron home library' in the options list
13. Select patron home library
14. from cli run 'perl ./misc/cronjobs/overdue_notices.pl'
=> You will see that only one message had been created. Its content shows both B2 and B3 overdue items
=> The sender address matches B1 address (the patron home library).
15. Search for OverdueNoticeFrom in System preferences and select cron
16. from cli run './misc/cronjobs/overdue_notices.pl --frombranch patron-homebranch'
=> you will get the same result at step 14
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Hinemoea Viault <hinemoea.viault@inlibro.com>
Was failing the qa script, so I perltidied the changed lines.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Test plan:
1) Select a patron A
2) Change all the message prefs away from defaults.
3) Run misc/maintenance/borrowers-force-messaging-defaults.pl --doit --message-name Item_due
Verify that patron A prefs changed only for 'Item due'
Signed-off-by: Sam Lau <samalau@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Perltidied one line in cleanup_database.pl.
Bug 33837: Perl tidy block in Objects.t
WARN t/db_dependent/Koha/Objects.t
WARN tidiness
The file is less tidy than before (bad/messy lines before: 319, now: 320)
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Like filter on category Bug 19454, it would be helpful to filter on a specified library for misc/maintenance/borrowers-force-messaging-defaults.pl
Test plan:
1) Select two patrons A and B in say libraries L1 and L2.
2) Change the msg prefs for A and B away from defaults.
3) Run misc/maintenance/borrowers-force-messaging-defaults.pl -doit -library L1
Verify that patron A changed and patron B did not.
4) Run with both category and library filters
Signed-off-by: Sam Lau <samalau@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This change removes the no_batch_alters option.
When working with real world databases, you really want to use batch
alters to group together changes. Otherwise, the output becomes
too overwhelming/unwieldy.
On real world databases, I'm noticing big discrepancies, although
I think most of the time this is a difference in the collation
at the column level.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This script can take a connected database handle and compare that database
schema against kohastructure.sql to see what changes the database would
need in order to match kohastructure.sql
NOTE: It uses SQL::Translation::Diff, which is installed with DBIx::Class.
WARNING: The diff doesn't seem to compare comments, so that difference
won't appear in the output. If we wanted, we could easily enhance the
audit_database.pl script to also compare comments.
WARNING: The output is a proposed series of SQL commands. While they
are useful to review, they won't always duplicate the changes done
by updatedatabase.pl, so it's important to carefully analyze the output.
The key purpose of this audit script is to just highlight the differences
between the two.
Test plan:
0. Apply patch
1. vi ./installer/data/mysql/kohastructure.sql
2. Comment out some columns, change NULL status, or whatever you like
3. perl misc/maintenance/audit_database.pl \
--filename /kohadevbox/koha/installer/data/mysql/kohastructure.sql
4. Note that the output includes SQL commands to change the database
to match the new kohastructure.sql
5a. Try using koha-foreach and note that the database name appears above
the database comparison
5b. koha-foreach "perl misc/maintenance/audit_database.pl \
--filename /kohadevbox/koha/installer/data/mysql/kohastructure.sql"
Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This script was never fully promoted or documented and it requires third
party libraries are installed to use. This patch removes it from the
repository and the Makefile so it is no longer installed.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Test plan:
Run misc/cronjobs/membership_expiry.pl --help
misc/cronjobs/membership_expiry.pl --man
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Note: I did not add an example to the debian cron daily file, since
we have no commented lines there. And automatic renewal is no default
behavior.
Test plan:
Read new lines in misc/cronjobs/crontab.example. Just comments.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Just as passing another expiry notice.
Test plan:
Pick a patron to expire.
Create another membership renewal notice, say X.
Run misc/cronjobs/membership_expiry.pl -c -n -v -letter_renew X
Check output on command line.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Test plan:
[1] Pick an account to expire soon.
[2] Choose MembershipExpiryDaysNotice so that it will be selected.
(Note: You can extend the selection with -before or -after.)
[3] Run misc/cronjobs/membership_expiry.pl -c -v -renew
[4] Check new expiry date in the notice. Does it correspond with
enrollment period of category and pref BorrowerRenewalPeriodBase?
[5] If the patron category has an enrollment fee, check account too.
[6] Reset dateexpiry of this patron to original date.
[7] Run misc/cronjobs/membership_expiry.pl -c -v
[8] Check if you got an expiry notice instead and dateexpiry did not
change.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Removed $fees and $substitute in favor of notice approach
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Bug 33245 has been adjusted; the fallback has been removed.
We should check now if the -active or -inactive parameter
actually has been passed.
Test plan:
See earlier test plan.
Run the script without active/inactive.
Run the script with active.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Test plan:
Enable TrackLastPatronActivity.
Set MembershipExpiryDaysNotice to X days.
Pick a patron, make it expire in X days. Add email address.
Update borrowers.lastseen for this patron to make it active.
Run membership_expiry.pl -c -v -active 6
You should see at least one enqueued notice.
Run membership_expiry.pl -c -v -inactive 6
You should see that it skipped a number of active patrons (>=1).
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Sam Lau <samalau@gmail.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
How to test:
1) Run tests in t/db_dependent/Circulation/MarkIssueReturned.t
Sponsored by: Gothenburg University Library
Signed-off-by: Michaela <michaela.sieber@kit.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch makes the pre-formatting of notice templates conditional on the content type.
Test plan:
1) Navigate to tools > Notices and slips
2) Edit the notice with the code of HOLD
3) Under print, paste the following
<p>Please collect within 10 days from the date of this letter and don't forget to bring your library card with you.</p>
<p>Be aware that opening hours vary at different library branches so if in doubt double check the times before you visit.</p>
<p>Should you no longer require this item please contact us so we can make it available for another customer.</p>
<p>Thank you</p>
4) Check the box that says HTML message and save the notice
5) In Patrons, select a patron and in Patron messaging preferences, check the box in the Email column for Hold filled
6) Find a record in the catalogue
7) Copy the barcode for one of the items
8) Place a hold on the item for the patron you selected
9) Navigate to Circulation > Check-in
10) Paste the barcode in the Check-in box and click Check-in
11) In the pop-up box, click Confirm hold
12) In the command line, run perl misc/cronjobs/gather_print_notices.pl --letter_code=HOLD --html /tmp
13) cd /tmp
14) There should be a file in there called 'notices_HOLD-date.html
15) cat this file
16) The html you pasted in the template will be wrapped in a <pre> tag
17) Apply patch and restart_all
18) rm the notices file that was created
19) cd /kohadevbox/koha
20) Select a new record from the catalogue and repeat steps 7-15 for this record, using the same patron
21) This time, the html file should not contain the <pre> tag
22) Sign off!
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Currently --expiration-from-today has the same definition as --update-expiration. The description has now been updated to reflect the fact that it will set the date from today, and not the patron's enrollment date
Test plan:
Look at the patch file and confirm that the POD now mentions today's date rather than the patron's enrollment date
Signed-off-by: Laura Escamilla <laura.escamilla@bywatersolutions.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
For consistency with other options I opted to have the fee acknowledged
parameter to expect a string and as it's an optional parameter I've
dropped the default value of 'N' too.
Test plan
1) Prior to this patch
1a) Attempt a checkout without passing -fa/--fee_acknowledged flag
`./misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL --patron 23529000035676 --item 39999000007756 -m checkout`
The final field of the SIP request will be '|BON', the default
1b) Attempt a checkout passing -fa/--fee_acknowledged flag
`./misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL --fee-acknowledged --patron 23529000035676 --item 39999000007756 -m checkout`
The final field of the SIP request will still be '|BON', failure
1c) Attempt a checkout passing a string for fee_acknoewledeged flag
`./misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL --fee-acknowledged Y --patron 23529000035676 --item 39999000007756 -m checkout`
The final field of the SIP request will still be '|BON', failure
2) Apply the patch
2a) Attempt a checkout without passing -fa/--fee_acknowledged flag
`./misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL --patron 23529000035676 --item 39999000007756 -m checkout`
The optional `|BO` element should not be present
2b) Attempt a checkout passing -fa/--fee_acknowledged flag
`./misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL --fee-acknowledged N --patron 23529000035676 --item 39999000007756 -m checkout`
The final field of the SIP request will now be '|BON', success
2c) Attempt a checkout passing a string for fee_acknoewledeged flag
`./misc/sip_cli_emulator.pl -a localhost -p 6001 -su term1 -sp term1 -l CPL --fee-acknowledged Y --patron 23529000035676 --item 39999000007756 -m checkout`
The final field of the SIP request will now be '|BOY', success
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Add an alias for added-before, added-after (Using an alias makes it
backwards compatible) and update the POD.
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This adds, as requested, the `added_after` filter option to
writeoff_debts.
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
misc/cronjobs/holds/holds_reminder.pl help says one can use arge -t for --triggered
But not implemented in code.
Should be like in misc/cronjobs/overdue_notices.pl :
't|triggered' => \$triggered,
Test plan :
Play with misc/cronjobs/holds/holds_reminder.pl with arg -t and
--trigger
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Koha overdue notice “print” email in Outlook is incorrectly encoded in czech language.
Tested in KohaTestingDocker - https://gitlab.com/koha-community/koha-testing-docker
This patch fixes that error.
Test plan:
Overdue notice, set print in UI, https://snipboard.io/FLkIxf.jpg.
Message body:
<table style="padding:0; margin: 0; border-collapse: collapse; font-family: 'Trebuchet MS', Helvetica, sans-serif; width: 100%; "><tr><td style="vertical-align: top; padding: 0"> </td><td style="width: 75mm; vertical-align: top;"><table style="margin-top: 2.5cm; font-size: 11pt;"><tr><td><<borrowers.cardnumber>></td></tr><tr><td><<borrowers.firstname>> <<borrowers.surname>></td></tr><tr><td><<borrowers.address>> <<borrowers.streetnumber>></td></tr><tr><td><<borrowers.city>></td></tr><tr><td><<borrowers.zipcode>></td></tr></table></td></tr><tr><td colspan="2" style="padding: 0;"><table style="width: 100%; margin: 3cm 0 0 0;"><tr><td style="width: 5mm; height: 5mm; border-bottom: 1px dotted #000;"> </td><td style="height: 1mm;"></td><td style="width: 5mm; height: 1mm; border-bottom: 1px dotted #000;"> </td></tr></table></td></tr><tr><td colspan="2"><table><tr><td style="padding: 0.5cm 0.3cm">V Chrudimi dne <<today>></td></tr><tr><td style="padding: 0 0.3cm">Upozorňujeme Vás, že byla již o 30 dnů překročena doba, na kterou jsme Vám půjčili dokumenty (celkem<<count>>). Žádáme Vás o brzké vrácení těchto výpůjček:</td></tr><tr><td style="padding: 0.5cm 0.3cm;"><table><tr><th>Od</th><th>Do</th><th>Název</th><th>Autor</th><th>Částka</th></tr><item><tr><td><<issues.issuedate>></td><td><strong><<issues.date_due>></strong></td><td><<biblio.title>></td><td><<biblio.author>></td><td><<items.fine>> Kč</td></tr></item></table></td></tr><tr><td style="padding: 0 0.3cm">Zpozdné je účtováno dle ceníku knihovny.</td></tr><tr><td style="padding: 0 0.3cm">Dokumenty můžete vrátit na jakémkoli oddělení knihovny a pobočce v jejich provozní době.</td></tr><tr><td style="padding: 0 0.3cm">Nesplní-li čtenář svou povinnost vyrovnat závazky s knihovou, má knihovna právo postupovat podle příslušných ustanovení občanského zákoníku.</td></tr></table></td></tr></table>
Set delay 1 day, https://snipboard.io/boAd3F.jpg.
Checkout any book.
Set in database issues->date_due = today() - 1
for checkouted book, https://snipboard.io/PrylGB.jpg.
Run this scripts:
koha-shell kohadev -c "/kohadevbox/koha/misc/cronjobs/overdue_notices.pl -v -t"
You must have KOHA set up for sending e-mails (SMTP server). Replace xxx@email.cz to your e-mail and run this script:
koha-shell kohadev -c "/kohadevbox/koha/misc/cronjobs/gather_print_notices.pl /var/spool/koha/kohadev/ --email xxx@email.cz"
Signed-off-by: Michal Denar <black23@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Test plan:
Run misc/cronjobs/sitemap.pl -where "biblionumber>X" (replace X
by some clever value)
Verify results by browsing sitemap files.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Kyle M Hall
Martin Renvoize
Pedro Amorim
Tomas Cohen Arazi
Nick Clemens
Katrin Fischer
Alex Buckley
Matthias Meusburger
Wainui Witika-Park
Jan Kissig
Matt Blenkinsop
Jonathan Druart
Andreas Roussos
Magnus Enger
Thomas Klausner
Emily Lamancusa
Fridolin Somers
David Cook
Rudolf Byker
Michael Hafen
Janusz Kaczmarek
Lucas Gass
Julian Maurice
Hammat Wele
Matthias Le Gac
Marcel de Rooy
David Schmidt
David Gustafsson
David Gustafsson
Caroline Cyr La Rose
Katrin Fischer
Koha translators
Emmi Takkinen
Olli-Antti Kivilahti
Aleisha Amohia
Victor Grousset/tuxayo
Thibaud Guillot
Laura Escamilla
Stefan Berndtsson
Petr Svoboda
