The migration from SIPconfig.xml to the database ( bug 37893 ) leaves
holds_get_captured NULL for any account that didn't define it in the
config file. At SIP checkin an unset holds_get_captured defaults to
on, but the REST API returns NULL boolean columns as false, so the
SIP account editor shows "No" selected. Saving the account without
touching the option then stores 0 and holds are no longer captured
at SIP checkin.
This patch changes the column default to 1 and sets existing NULL
values to 1 so the stored value matches the actual checkin behavior.
Test Plan:
1) If you have no SIP accounts, create one
2) Simulate an account migrated from a SIPconfig.xml that doesn't define holds_get_captured:
UPDATE sip_accounts SET holds_get_captured = NULL;
3) Edit the account, note "Holds get captured" shows "No" even though
holds *are* captured at SIP checkin while the value is unset
4) Apply this patch
5) Run updatedatabase.pl
6) SELECT login_id, holds_get_captured FROM sip_accounts;
7) Note holds_get_captured is now 1!
8) Edit the account again, note "Yes" is now selected for "Holds get captured"!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Test plan:
root@kohadevbox:koha$ perl /kohadevbox/misc4dev/run_tests.pl --run-db-compare-only --compare-with v24.05.00
should pass with this patch
Without this patch you get:
Running [diff /tmp/current_db.sql /tmp/upgraded_db.sql && diff -q /tmp/current_db.sql /tmp/upgraded_db.sql || { echo "ERROR - DB structures are not identical" && exit 1; }]...
7207c7207
< /*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Miranda Nero <mnero@oslri.net>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Bob Bennhoff <bbennhoff@clicweb.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
MySQL 8.4 requires foreign keys to reference PRIMARY KEY or UNIQUE keys.
Since authorised_values.authorised_value is only unique in combination
with category (composite UNIQUE key av_uniq), the FK constraint is not
compatible with MySQL 8.4+.
Data integrity for illrequests.status_alias is maintained at the application layer:
- Koha::ILL::Request->status_alias() validates values against
authorised_values (category='ILL_STATUS_ALIAS') before storing
- All ILL request operations in Koha use this object layer
- CASCADE behaviors (ON DELETE SET NULL, ON UPDATE CASCADE) are already
handled by the application layer when modifying/deleting authorised values.
Test Plan, using KTD:
1. Prepare KTD for MySQL 8.4
- In koha-testing-docker, edit 'compose/db-mysql.yml'
- Comment out the line 5:
`# command: --default-authentication-plugin=mysql_native_password`
Note: This is required because MySQL 8.4 doesn't support the native password plugin.
2. Start a new instance with MySQL 8.4
- `DB_IMAGE=mysql:8.4 ktd up`
- Observe that it fails to start with the errors such as 'DBD::mysql::st execute failed'
3. Apply the patch
- `git bz apply 39107`
4. Start the instance again
- `DB_IMAGE=mysql:8.4 ktd up`
- Observe that it starts successfully
5. Update the database (Optional, only for existing instances)
- `perl installer/data/mysql/updatedatabase.pl`
6. Access Koha Staff and OPAC interfaces and verify that everything works as expected
7. Sign off
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Christopher Brannon <cbrannon@cdalibrary.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Adds a new nullable longtext column `composite_scores` to the
`import_record_matches` table to store a JSON object mapping each
matchpoint search_index to the score it contributed to the total.
This allows the UI to display a breakdown of match scores (e.g.
"title:50, author:50") rather than just the aggregate total.
Sponsored-by: Main Library Alliance <https://www.mainlib.org/>
Signed-off-by: Jeanne Mauriello <jeanne.mauriello@mainlib.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The SIP fields in the database are too small for the templates of some configurations, and are truncated during upgrade resulting in invalid templates. It's likely we need more than 255 characters for some templates. We should convert all sip template fields to TEXT fields.
Test Plan:
1) For each template field on a sip account attempt to save a template like:
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXY
2) Note error preventing you from doing so
3) Apply this patch
4) Run updatedatabase.pl
5) Repeat step 1
6) Note you can now do so without error!
Signed-off-by: Charlie Arthur <carthur@slolibrary.org>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This has the same goals of bug 24975 but with a different approach.
Bug 24975 tried the "POT/PO files as database tables" approach but
it appeared that is not what best fits our needs.
Instead of a "source text / translated text" mapping, we need
translations to be attached to a specific "object" (or table row), which
has the added benefit of keeping translations unaffected when the
original string changes. It may or may not be what you want but at least
this avoids the problem where fixing an error in the original string
forces users to re-translate everything.
We also need to be able to translate all translatable properties of a
particular entity.
The solution proposed here is to add a DBIC component that will add
several DBIC relations to the `localization` table:
* a relation named `localizations` that will return all translations for
all translatable properties for all languages.
* for each translatable property:
* a relation named `{$property}_localizations` that will return all
translations for a given property, for all languages
* a relation named `{$property}_localization` that will return the
translation for a given property, for the current language
To enable translations for a particular entity, we will need to:
* add two method calls in the "result class" corresponding to the
localizable object type
* in the admin interface, add a link to the the translation popup.
Unlike bug 24975 there is not a page where you can translate
everything. I don't know if it is really useful (probably better to
translate itemtypes when you are on the itemtype page). It can be
added later if needed.
To give an example, this is what needs to be added manually in
Koha/Schema/Result/Itemtype.pm:
__PACKAGE__->load_components('+Koha::DBIx::Class::Localization');
__PACKAGE__->localization_add_relationships(
'itemtype', # the PK column
'description' # a list of translatable columns
);
(see POD in Koha/Schema/Component/Localization.pm for more information)
Final notes:
* I wanted this patch to have as few changes as possible so I kept
methods like `search_with_localization`, `children_with_localization`
even if I believe they can be removed (replaced by search/children)
Localizations do not need to be prefetched/joined unless you want to
sort on them.
For the same reason, $itemtype->unblessed now always return a
`translated_description` key.
This can be fixed in followup patches
* There is cache involved (see
Koha::Schema::Component::Localization::localization).
Localizations in cache are grouped by object type and language (keys
look like 'KOHA:localization:Itemtype:en')
* By keeping a single table for all translations of all entities, we
cannot have data integrity at the RDBMS level, but automatic deletion
of translations is handled by the DBIC `cascade_delete` feature, so
integrity should be fine as long as no raw SQL update/delete operation
is involved.
Signed-off-by: Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The categorycode column comment in the categories table had a typo:
'idenfity' instead of 'identify'. This fixes it in kohastructure.sql
and adds an atomic update to correct existing installations.
Test plan:
1. Apply this patch
2. Run:
$ updatedatabase
=> SUCCESS: The atomic update is applied
3. Verify:
echo "SHOW FULL COLUMNS FROM categories WHERE Field='categorycode';" | koha-mysql kohadev
=> SUCCESS: The comment reads 'unique primary key used to identify the patron category'
4. Sign off :-D
Signed-off-by: Laura_Escamilla <laura.escamilla@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
This also includes a 'manager' accessor which returns the Koha::Patron
object
Signed-off-by: Jeremy Evans <Jeremy.Evans@ukhsa.gov.uk>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
- Add order_by => ['name'] to record_sources fetch in marc-overlay-rules.pl
so filter dropdown options are consistently sorted
- Add COMMENT to record_sources.is_system column in kohastructure.sql
to match convention of other columns in that table
- Remove commented-out Data::Dumper debug lines from biblios.t
- Remove redundant explicit object deletes inside subtests that are
already wrapped in txn_rollback; the rollback undoes them anyway
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
This changes the structure of record source with an is_system column which controls whether this value can be deleted.
Also predefined values are inserted to reflect the default hard coded values from 'Record overlay rules' sources
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
This patch alters the items_last_borrowers table to allow for NULL borrowers
It also adds two new system preferences to allow controlling the anonymization of items_last_borrower data
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Peter Moore <peter@mrcpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Sponsored-by: The Main Library Alliance <https://www.mainlib.org/>
Signed-off-by: Trevor Diamond <trevor.diamond@mainlib.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
- Add comprehensive POD documentation for lowest_ratio_library() method
including parameter descriptions, return values, and algorithm explanation
- Add database index on itemtype column in library_float_limits table
to improve query performance when searching by itemtype
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This commit adds the database migration to move hold fee configuration
from patron categories to circulation rules, providing more granular
control over hold fees based on patron category, item type, and branch.
The migration:
- Creates circulation rules for existing categories.reservefee values
- Removes the deprecated reservefee column from categories table
Sponsored-by: OpenFifth <https://openfifth.co.uk>
Signed-off-by: Jackie Usher <jackie.usher@westsussex.gov.uk>
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Test plan:
Apply the patches and update the database
1) Create a new subscription with items, there should be a checkbox at the bottom of the form to pre-select collection table issues - make sure this is checked
2) Receive some items against the subscription
3) Navigate to the collections table
4) The checkboxes should be pre-selected as per previous behaviour in Koha
5) Edit the subscription and uncheck the checkbox
6) Return to the collections table and the checkboxes should all be unchecked
Signed-off-by: Nathalie <nathalie.savarit@univ-amu.fr>
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
(cherry picked from commit 881f96172a8c1b35d3e74f7f0f6aa44629578021)
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Test plan:
1) Run 'updatedatabase'
2) No errors should be thrown
Sponsored-by: Cheshire West and Chester Council
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Laura_Escamilla <laura.escamilla@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This should pass:
perl /kohadevbox/misc4dev/run_tests.pl --instance=kohadev --db-password=password --run-db-compare-only --compare-with=HEAD
This is a squash of the following:
1) Fix charset in kohastructure.sql
2) Fix NULL => DEFAULT NULL
3) Reorder tables in kohastructure.sql
4) typos
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This follow-up removes the security vulnerability of storing plaintext
passwords in the sip_accounts table. The SIP2 system now relies entirely
on Koha's secure password authentication.
Changes:
- Remove plaintext password comparison from login_core() and telnet_transport()
- Remove login_password column from sip_accounts table
- Update database schema files and atomic update
- Remove password field from API definitions and Vue form
- Update all tests to remove login_password references
Authentication now works in two stages:
1. Access control: Check if userid exists in sip_accounts (allowlist)
2. Authentication: Validate password against hash in borrowers table
This maintains SIP access control while eliminating plaintext password
storage. Staff users authenticate with their Koha password (securely hashed).
Test plan:
1. Run updated tests to verify they pass
2. Verify SIP2 authentication works with staff user credentials
3. Confirm no plaintext passwords are stored
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Kyle M Hall
Jonathan Druart
Pedro Amorim
Lucas Gass
Brendan Lawlor
Saiful Amin
Matt Blenkinsop
Martin Renvoize
Julian Maurice
Tomás Cohen Arazi
Jan Kissig
Michael Hafen
Nick Clemens
Matt Blenkinsop
Pedro Amorim