A cashup is the process of removing cash from the till to send it
to the bank, so labelling the headline cash line "Cash added to
register" was confusing. Re-word it to "Cash removed from register"
in the typical case and "Cash added to register" in the edge case
where the cashier has had to top up the till to cover refunds.
While doing this, align the sign convention of total_grouped with
that of the overall total field on the summary: positive means cash
collected (i.e. removed from the register at cashup time), negative
means cash had to be added back into the register. Previously
total_grouped used the raw accounting sign (credits negative, debits
positive), so a typical $2 cash session would display "Total: 2.00"
and "Cash added to register: -2.00", which was internally
inconsistent. The conditions on the cash and other-payment-type rows
in cashup_modal.js are swapped to match the new convention.
Test plan:
1. Apply the patch and rebuild JS (yarn js:build) inside KTD.
2. As a staff user with permission to manage cash registers, take
payments via several payment types (some cash, some not).
3. Trigger a cashup and view the cashup summary modal.
4. Confirm the "Total" line and the per-payment-type lines all
display as positive amounts in the typical "money was collected"
scenario, with the cash row labelled "Cash removed from
register".
5. Issue some refunds large enough to exceed cash collected, then
cash up again.
6. Confirm the cash row is labelled "Cash added to register" with a
negative amount and the per-non-cash-type rows are labelled
"{type} to add" where applicable.
7. Run prove t/db_dependent/Koha/Cash/Register/Cashup.t and prove
t/db_dependent/Koha/Cash/Register.t inside KTD; both should
pass.
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: Jackie Usher <jackie.usher@westsussex.gov.uk>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch resolves three QA findings on the cashup reconciliation
work:
* Add a CSP nonce to the inline <style> block in cashup_summary.inc
so it is not flagged by the forbidden_patterns check.
* Document the private _get_session_end method on
Koha::Cash::Register::Cashup so pod_coverage passes.
* Mock CashupReconciliationNoteRequired to 0 in
t/db_dependent/Koha/Cash/Register/Cashup.t so the test does not
depend on the site preference value (it previously failed when
the syspref was set to "Require" because surpluses and deficits
are created without a reconciliation note in this test).
Test plan:
1. Apply the patch.
2. Set CashupReconciliationNoteRequired to "Require".
3. Run prove t/db_dependent/Koha/Cash/Register/Cashup.t -- it should
pass.
4. Set CashupReconciliationNoteRequired to "Don't require" and
re-run -- it should still pass.
5. Run koha-qa.pl on the bug branch -- pod_coverage on
Koha/Cash/Register/Cashup.pm and forbidden_patterns on
cashup_summary.inc should now pass.
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: Jackie Usher <jackie.usher@westsussex.gov.uk>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch adds configuration options for cashup reconciliation and
enhances the user interface with preview functionality and improved
interaction patterns.
Configuration Options:
1. CashupReconciliationNoteRequired (System Preference):
- Optional validation requiring reconciliation notes when discrepancies exist
- Ensures documentation of surplus/deficit causes when needed
- Can be disabled for less stringent workflows
- Applied at cashup completion time
2. CashupReconciliationNoteAuthorisedValue (System Preference):
- Allows use of authorized value dropdown for reconciliation notes
- Provides standardized note options (e.g., "Till count error", "Cash removed")
- Falls back to free text textarea when not configured
- Improves consistency and reporting capabilities
UI Enhancements:
1. Cashup Summary Preview:
- Preview cashup summary for in-progress cashups
- Shows current transaction totals before completion
- Clearly indicates preview vs. final summary status
- Helps staff verify amounts before committing
2. Improved User Experience:
- Conditional note fields based on configuration
- Authorized value dropdowns when configured
- Clear validation messages
- Responsive UI updates
Backend changes:
- Enhanced Koha::REST::V1::CashRegisters::Cashups with preview support
- Database atomicupdate script for system preferences
- Validation logic for required notes
Frontend changes:
- Preview modal functionality in cashup_modal.js
- Conditional rendering of note input fields
- Authorized value dropdown integration
- Enhanced summary display logic
Test plan:
1. Apply patches and run database update
2. Configure CashupReconciliationNoteRequired:
- Enable preference
- Perform cashup with discrepancy without note - should fail
- Add note and retry - should succeed
- Disable preference - verify note optional
3. Configure CashupReconciliationNoteAuthorisedValue:
- Create AV category (e.g., CASHUP_NOTE) with values
- Set preference to category name
- Verify dropdown appears in cashup modal
- Clear preference - verify textarea shown
4. Test cashup preview:
- Start cashup on register
- Click "Preview cashup summary"
- Verify current totals displayed
- Add transaction and re-preview - verify updated
- Complete cashup - verify final summary
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: Jackie Usher <jackie.usher@westsussex.gov.uk>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch implements an optional two-phase cashup workflow for point
of sale operations, along with improved error handling and validation.
Two-Phase Cashup Workflow:
- Staff can initiate cashup (CASHUP_START) and complete later
- Prevents new transactions on registers with cashups in progress
- Tracks cashup sessions with start/end timestamps
- Supports both quick cashup (immediate) and staged cashup workflows
- Updates batch cashup workflow operations across multiple registers
Error Handling & Validation:
- Centralized exception handling for cashup operations
- Clear validation messages for missing parameters
- Prevents cashup when no cash transactions exist
- Improved handling of zero/negative amounts
- Informative error messages for user guidance
Negative Amount Support:
- Supports negative cashup amounts for cashup deficits
- Automatic detection when actual cash is less than expected amount
- Creates appropriate CASHUP_DEFICIT records
- UI handles both positive and negative reconciliation amounts
- Updated calculations and display logic throughout
Modal Refactoring:
- Eliminates code duplication between register.tt and registers.tt
- Centralized cashup modal functionality in cashup_modals.js
- Shared confirm_cashup.inc and trigger_cashup.inc templates
- Consistent user experience across all cashup workflows
- Improved maintainability and code organization
Backend changes (Koha::Cash::Register):
- start_cashup(): Creates CASHUP_START action to begin session
- cashup_in_progress(): Detects active cashup sessions
- add_cashup(): Enhanced with two-phase completion support
- outstanding_accountlines(): Respects cashup session boundaries
Frontend changes:
- Interactive modals for starting and completing cashups
- Real-time calculation displays
- Support for reconciliation with actual amounts
- Batch operations UI for multiple registers
- Responsive error messaging and validation feedback
Test plan:
1. Apply patches and restart services
2. Run prove t/db_dependent/Koha/Cash/Register.t
3. Run prove t/db_dependent/Koha/Cash/Register/Cashup.t
4. Test single register workflows:
- Start cashup, add transactions, verify blocked
- Complete cashup with reconciliation amounts
- Test with positive, negative, and zero amounts
5. Test multi-register workflows:
- Select multiple registers on registers page
- Perform batch cashup operations
- Verify each register processes correctly
6. Test error conditions:
- Attempt cashup with no transactions
- Attempt cashup with invalid amounts
- Verify clear error messages displayed
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: Jackie Usher <jackie.usher@westsussex.gov.uk>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Bug 42391 removed the warn from Koha::Object::store when it moved
exception translation to the exception_action hook. This lost useful
debugging information from the Plack error logs.
This patch restores the warning by adding a try/catch in store() that
stringifies and warns the exception before rethrowing. The warning now
contains the translated Koha exception message (e.g. 'Broken FK
constraint') rather than the raw DBI error, since translation happens
before the exception reaches store().
Note: delete() does not warn. Ideally the logging would live in the
ExceptionMapper itself (via an injected logger), covering all code paths
uniformly. That is a larger refactor tracked separately.
Tests that assert on these warnings are updated to match the new
message format.
Test plan:
1. Apply patch
2. Run:
$ ktd --shell
k$ prove t/db_dependent/Koha/Object.t \
t/db_dependent/Koha/Objects.t \
t/db_dependent/Koha/Schema/ExceptionMapper.t \
t/db_dependent/Koha/Item.t \
t/db_dependent/Koha/Suggestions.t \
t/db_dependent/Koha/Account.t \
t/db_dependent/Koha/ApiKey.t \
t/db_dependent/Koha/Checkouts/ReturnClaim.t \
t/db_dependent/Circulation/MarkIssueReturned.t \
t/db_dependent/Suggestions.t \
t/db_dependent/Biblio.t
=> SUCCESS: Tests pass!
3. Sign off :-D
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
The Koha::Schema::Util:: namespace adds unnecessary nesting for a
single-purpose class. This follow-up moves it to
Koha::Schema::ExceptionMapper, which sits alongside Koha::Schema
itself — a better fit given it's tightly coupled to the Schema's
exception_action hook.
Changes:
- Koha::Schema::Util::ExceptionTranslator → Koha::Schema::ExceptionMapper
- Update Koha::Schema to use the new module
- Move test file accordingly
- Remove now-empty Koha/Schema/Util/ directory
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Extend the ExceptionTranslator to handle two additional MySQL error
patterns:
- NOT NULL violations ('Column X cannot be null' and 'X doesn't have
a default value') → Koha::Exceptions::Object::NotNull
- Not in database errors → Koha::Exceptions::Object::NotInStorage
Add Koha::Exceptions::Object::NotNull exception class with a property
field identifying the offending column.
Add Koha::Exceptions::Object::UnhandledDBError as a catch-all for DBI
errors that don't match any known pattern. This closes the gap where
unmatched MySQL errors could leak as DBIx::Class::Exception to callers
(e.g. templates that no longer catch that type).
The exception_action handler now distinguishes three cases:
- Koha exception objects (from txn_do re-throws): passed through via
'die $msg' to preserve the original exception type. Without this,
txn_do's rollback-and-rethrow would stringify the Koha exception
and wrap it in UnhandledDBError, losing the specific type.
- DBI errors (prefixed 'DBI Exception:'): translated if possible,
otherwise wrapped in UnhandledDBError.
- DBIC internal errors (e.g. 'Not in database'): translated if
possible, otherwise thrown as DBIx::Class::Exception.
Note: NotInStorage translation works for update operations but not for
delete-on-detached-Row where DBIC bypasses the Schema's exception_action.
Those cases remain DBIx::Class::Exception.
Update tests:
- OAIServers.t, Z3950Servers.t: assert NotNull with property check
- Hold.t: assert NotNull, remove warn suppression hack
- Objects.t: update-not-in-storage cases assert NotInStorage
- Upload.t: delete-not-in-storage stays DBIx::Class::Exception
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
DBIx::Class provides an exception_action hook that intercepts every
throw_exception call before it becomes a DBIx::Class::Exception. By
installing this hook on Koha::Schema via a connection() override, all
database exceptions are automatically translated into Koha exceptions
at the source.
This eliminates the need for translate_exception catch blocks in every
caller:
- Remove try/catch from Koha::Object::store
- Remove try/catch from Koha::Object::delete
- Remove try/catch from Koha::Item::add_to_bundle
- Remove translate_exception convenience method from Koha::Schema
- Remove safe_do (already removed in previous follow-up)
ExceptionTranslator is refactored to work on plain error message strings
instead of DBIx::Class::Exception objects, since exception_action
receives the message before wrapping.
Unmatched errors fall through to DBIx::Class::Exception->throw() as
the default, preserving backward compatibility.
The warn $_->{msg} that was in Object::store is no longer needed since
exceptions are translated before they propagate. Tests that asserted on
that warning (via warning_like) are updated to use throws_ok directly.
Test plan:
1. Apply patch
2. Run:
$ ktd --shell
k$ prove t/db_dependent/Koha/Schema/Util/ExceptionTranslator.t
=> SUCCESS: Tests pass!
k$ prove t/db_dependent/Koha/Object.t
=> SUCCESS: Tests pass!
k$ prove t/db_dependent/Koha/Item.t
=> SUCCESS: Tests pass!
k$ prove t/db_dependent/Koha/Patron/Modifications.t
=> SUCCESS: Tests pass!
3. Sign off :-D
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
In MySQL8 the order on the PK is not preserved. We need to explicitely
pass it.
Test plan:
export DB_IMAGE=mysql:8.0;
then startup ktd
% prove t/db_dependent/api/v1/biblios.t
should pass with this patch
Without this patch it failed with:
13:00:29 # Failed test 'exact match for JSON Pointer "/4/item_id"'
13:00:29 # at t/db_dependent/api/v1/biblios.t line 276.
13:00:34 # got: '3420'
13:00:36 # expected: '3417'
13:00:37
13:00:38 # Failed test 'Withdrawn comes after lost'
13:00:57 # at t/db_dependent/api/v1/biblios.t line 276.
13:00:59 # got: '3417'
13:01:04 # expected: '3420'
13:01:05 # Looks like you failed 2 tests of 8.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jan Kissig <bibliothek@th-wildau.de>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
check_for_existing_matches() called Koha::Patrons->search(undef) when
PatronDuplicateMatchingAddFields left no usable field to match on,
which matches every patron instead of none.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
On the holdings table the filter by status=recalls is not working correctly.
Test plan:
Add a recall on an item of a bibliographic record that has several
items.
Filter the items table by status=recall
=> Without this patch all items are displayed
=> With this patch applied only the item with a recall is displayed
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Aleisha Amohia <aleisha@catalyst.net.nz>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
When there is a filter with the "status" param, we add a validation step
to secure the use of such a filter.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch allows _status to be in a nested structure.
Cannot be combined in all the possible ways, but at least it should fix
the UI.
Test plan:
Use the filters on the holdings table and confirm that you can combine
the status with other filters.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch updates offline circulation and ILL checkouts to properly
record their nature in circulation logs as forced transactions.
Offline circulation (process_koc.pl):
- Offline transactions have already occurred without access to normal
circulation checks
- Now marks these as forced with 'OFFLINE_CIRCULATION' override code
- This helps identify historical transactions that bypassed checks
ILL checkouts (Koha/ILL/Request.pm):
- ILL checkouts are system-initiated and may bypass certain restrictions
by design
- Now marks these as forced with 'ILL_CHECKOUT' override code
- This helps track and report on ILL circulation activity separately
Display updates (action-logs.inc):
- Added translation strings for both new override codes:
- OFFLINE_CIRCULATION: "Offline circulation checkout"
- ILL_CHECKOUT: "ILL system checkout"
- These will now display properly in Tools > Log viewer
This ensures these special checkout types are properly identified in
reporting and analysis, making it easier to understand the context
of historical transactions.
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
InstallPlugins forces every discovered plugin class to be deleted and
re-required from disk on every call, so plugin_methods reflects the
.pm file as it currently exists rather than whatever was already
loaded into a long-lived Plack worker. That deletion used
Symbol::delete_package, which clears the target package's whole
symbol table -- including any already-loaded packages nested under it
(e.g. a REST API controller a plugin bundles alongside itself, such as
Koha/Plugin/Foo/Controller.pm). Those nested classes keep their own
%INC entry, so they are never re-required afterwards and are left
permanently blanked out for the rest of the worker's life.
This was caught when running
`prove -r t/db_dependent/Koha/Plugins/ t/db_dependent/Koha/REST/Plugin/`
on top of the original patch. Tests fail several files that have nothing
to do with the bug being fixed (Biblio_and_Items_plugin_hooks.t,
Holds_hooks.t, TemplateIncludePathHook.t, REST/Plugin/PluginRoutes.t),
because Koha::Plugin::Test::Controller (a bundled test-fixture
controller living in t/lib/plugins/Koha/Plugin/Test/Controller.pm)
gets its symbol table wiped as collateral damage every time
Koha::Plugin::Test is reloaded.
_forget_plugin_class() replaces the delete_package call with the same
technique used by the CPAN module Class::Unload (not added as a
dependency here, just the technique): only clear the target class's
own symbols and reset its @ISA, explicitly leaving any nested package
entries in its symbol table alone. Module::Pluggable discovers bundled
nested classes as their own entries in the plugin list regardless, so
they still get reloaded correctly on their own turn.
I also annotate the no strict 'refs' block added by the previous commit
with a '## no critic' justification, as flagged by koha-qa.pl.
Test plan:
1. Confirm the bug exists without this patch:
a) git checkout <previous commit> (drop this commit, keeping the
first 3 commits of the bug applied)
b) Inside KTD:
prove -r t/db_dependent/Koha/Plugins/ t/db_dependent/Koha/REST/Plugin/
c) Observe failing files unrelated to this bug's own tests:
Biblio_and_Items_plugin_hooks.t, Holds_hooks.t,
TemplateIncludePathHook.t and REST/Plugin/PluginRoutes.t.
2. Confirm the fix:
a) Re-apply this commit
b) Re-run the same command from 1(b) -- all tests now pass
3. Confirm the original bug this branch set out to fix is still fixed,
both via the manual UI steps from the first commit's test plan
(install Batch Permission Modifier Plugin 1.3.1, upload 1.4.1 over
it via Tools > Plugins, confirm the renamed method takes effect
without restarting Plack) and via:
prove t/db_dependent/Koha/Plugins/Plugins.t
(see the 'InstallPlugins() reloads stale symbol tables tests'
subtest)
4. Confirm QA tooling is clean:
koha-qa.pl -c 4
=> PASS for Koha/Plugins.pm and all touched test files, no critic
or tidiness failures.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
When a plugin is upgraded in place (by uploading a new version over
the old one), the list of methods Koha knows about for that plugin
can get out of date. For example, if a plugin used to have a "tool"
method and the new version renames it to "admin", the staff interface
can still show the old "Run tool" link, and clicking it produces an
error saying the plugin has no "tool" method.
This happens because the plugin upload runs inside a long-lived
Plack worker that has already loaded the old version of the plugin
in memory. When Koha scans the plugin to record its methods, Perl
has no reason to re-read the file from disk, so the scan sees the
old method list and stores that in the plugin_methods table. The
plugin code on disk is new, but the record of what methods it has
is stale.
This patch forces Koha to throw away the in-memory copy of each
plugin and reload it from disk before scanning, so the recorded
list of methods matches the plugin that is actually installed.
Test Plan:
1) Install the Batch Permission Modifier Plugin version 1.3.1
(this version exposes a "tool" method)
2) Go to Tools > Plugins and confirm the plugin row shows
"Run tool" in its actions menu
3) Upload Batch Permission Modifier Plugin version 1.4.1 over
the top of 1.3.1 (this version exposes "admin" instead of
"tool")
4) Reload the Plugins page
5) Note the plugin still shows "Run tool" in its actions menu
6) Click "Run tool"
7) Note the error: the plugin has no "tool" method!
8) Apply this patch
9) Restart all the things!
10) Uninstall the plugin
11) Repeat steps 1-4
12) Note that "Run tool" is gone and "Run admin tool" appears
in its place
13) Click "Run admin tool" and confirm it loads without error!
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
The FK-chain resolution added in the previous patch only handles ONE
level of nesting: it requires $parent_alias to be a direct relationship
of $main_source. This holds for every +count embed currently declared
in api/v1/swagger/ (all exactly one level deep, e.g.
biblio.items+count), but if that assumption is ever violated - a
deeper nested +count, a parent relation with a coderef (custom join)
condition, or no matching FK column name - the previous code silently
fell back to the pre-fix expression ($parent_alias.$fk_self), which can
reference a join alias that isn't available inside DBIC's limiting
subquery, reproducing the original 500 with no diagnostic.
This patch makes _build_count_subquery decline to build a subquery
(return undef) when the correlation can't be safely resolved, instead
of emitting SQL that may be broken. The caller already has an existing,
tested degradation path for "no DBIC relationship info" (prefetch the
association directly; to_api falls back to the Perl-level ->count call;
sorting on it is rejected with a clean 400) - unresolved nested
correlations now go through that same path rather than risking a 500.
Also:
- Extract the repeated "_resultset vs _result" result-source lookup
into a shared _result_source() helper (was duplicated in
dbic_merge_prefetch and _build_count_subquery).
- Document the one-level-deep limitation in _build_count_subquery's POD.
- Add a regression test pinning the 2-level-nested-+count case: it must
fall back to a plain prefetch (no COUNT subquery) rather than error
or emit unsafe SQL.
Test plan:
1. ktd --shell
2. prove t/Koha/REST/Plugin/Query.t t/db_dependent/Koha/Object.t \
t/db_dependent/Koha/REST/Plugin/Objects.t \
t/db_dependent/api/v1/acquisitions_orders.t \
t/db_dependent/api/v1/acquisitions_vendors.t \
t/db_dependent/api/v1/ill_requests.t \
t/db_dependent/api/v1/tickets.t \
t/db_dependent/api/v1/patrons.t
=> All pass, no regressions.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
When nested +count embeds (e.g. biblio.items+count) are used alongside a
has_many prefetch (e.g. biblio.suggestions or the implicitly added
biblio.biblioitem) and pagination, DBIC's limiting subquery optimization
places +select scalar refs in the inner subquery where the parent join
alias (e.g. 'biblio') is not yet available.
This patch fixes _build_count_subquery to resolve the correlation back to
the main table (me) via the FK chain. Instead of generating:
(SELECT COUNT(*) FROM items
WHERE items.biblionumber = biblio.biblionumber)
It now generates:
(SELECT COUNT(*) FROM items
WHERE items.biblionumber = me.biblionumber)
This works because me.biblionumber IS the FK to biblio, holding the same
value. The main table columns are always available regardless of DBIC's
query structure.
Changes:
- Add $main_source parameter to _build_count_subquery for FK resolution
- Thread main_source through _parse_prefetch and dbic_merge_prefetch
- When parent_alias != 'me', resolve through parent relationship condition
- Update Query.t assertion to match the new correlation expression
Test plan:
1. Apply the regression tests from the previous commit
2. Run:
$ ktd --shell
k$ prove t/db_dependent/api/v1/acquisitions_orders.t
=> FAIL: The Bug 42995 subtest fails with 500 errors
3. Apply this patch, repeat from step 2
k$ prove t/Koha/REST/Plugin/Query.t \
t/db_dependent/Koha/Object.t \
t/db_dependent/Koha/REST/Plugin/Objects.t \
t/db_dependent/api/v1/acquisitions_orders.t \
t/db_dependent/api/v1/acquisitions_vendors.t \
t/db_dependent/api/v1/ill_requests.t \
t/db_dependent/api/v1/tickets.t \
t/db_dependent/api/v1/patrons.t
=> SUCCESS: Tests pass!
4. Sign off :-D
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Adds DBIC relationship aliases so that +count embeds advertised in
the API specs are backed by actual database relationships, making
them sortable via the SQL COUNT subquery infrastructure.
New aliases:
- Borrower: checkouts (issues)
- Aqbookseller: baskets (aqbaskets), invoices (aqinvoices)
- Ticket: updates (ticket_updates)
- ErmEholdingsPackage: resources (erm_eholdings_resources)
- Biblio: holds (reserves)
Koha methods are updated to use the new relationship names directly,
following the one-liner style: Koha::Foo->_new_from_dbic( scalar ... )
Note: checkouts on Borrower and subscriptions on Aqbookseller already
had matching DBIC relationships. comments on Illrequest, requests on
IllBatch, and items on Biblio also already existed.
Test plan:
1. Apply patch
2. Run:
$ ktd --shell
k$ prove t/Koha/REST/Plugin/Query.t \
t/db_dependent/api/v1/patrons.t \
t/db_dependent/api/v1/acquisitions_vendors.t \
t/db_dependent/Koha/REST/Plugin/Objects.t \
t/db_dependent/Koha/Biblio.t
=> SUCCESS: All tests pass
3. Verify sorting works:
GET /api/v1/patrons
x-koha-embed: checkouts+count
_order_by: -me.checkouts_count
=> SUCCESS: Patrons sorted by checkout count
4. Sign off :-D
Assisted-by: Sonnet 4.6 (Anthropic)
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Currently, +count embeds (e.g. claims+count) are computed in Perl via
$object->$relation->count during to_api serialization. This means they
cannot be used in _order_by and each count triggers a separate SQL query
per row (N+1).
This patch makes dbic_merge_prefetch detect +count embeds and, when a
matching DBIC relationship exists, inject a correlated COUNT subquery
via +select/+as instead of adding a prefetch.
The order_by entries referencing count aliases are fixed up to strip the
me. prefix, since +as aliases are virtual columns not bound to a table.
In to_api, the is_count handler now checks for the pre-computed column
via get_column before falling back to the Perl-level ->count call. This
preserves backward compatibility for Koha-level methods that don't have
a corresponding DBIC relationship.
Test plan:
1. Apply patches
2. Run:
$ ktd --shell
k$ prove t/Koha/REST/Plugin/Query.t \
t/db_dependent/Koha/Object.t \
t/db_dependent/Koha/REST/Plugin/Objects.t
=> SUCCESS: Tests pass!
3. Sign off :-D
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Apply encode_text function to all text fields in EDIfact Order messages
to properly escape special characters (?, ', :, +) that could break
message parsing. Previously only bibliographic description fields were
escaped, but GIR segments containing collection codes, branch codes,
item types, call numbers, and other data were vulnerable.
Test plan:
- Verify all existing tests pass
- Added comprehensive tests for special character escaping in GIR segments
- Confirmed that fields containing ?, ', :, + are properly escaped
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: Laura_Escamilla <laura.escamilla@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Koha::SIP2::Object::store bumps the timestamp before SUPER::store,
opening a race condition where a worker reloading during db update
transaction caches a config that is missing the new row.
Bump the timestamp after the row has been written so it is not advanced
ahead of the data it represents.
Test Plan:
1) prove t/db_dependent/Koha/SIP2/SIP2ModuleMigration.t
2) prove t/db_dependent/api/v1/sip2_accounts.t
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Some item statuses can correspond to multiple authorised values: Not
for loan, Withdrawn, Lost and Damaged. Until now, only a single entry
could be selected, and filtering applied to all values at once.
This change introduces finer-grained filtering by exposing each
authorised value individually in the filter.
Test plan:
Assign multiple authorised values to items on a bibliographic record
for the four affected attributes: Not for loan, Withdrawn, Lost and
Damaged. Use the "Status" filter and verify that each value behaves as
expected.
Display note:
The current dropdown presentation is not ideal.
Suggestions for improvement are welcome, although avoiding optgroup
would be preferable to keep the scope small.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
When the OPACAuthorIdentifiersAndInformation system preference is
enabled, selected information about authors is displayed. However, if
the main author is a corporate body or a meeting/conference (MARC 21
fields 110/111), the information is not shown.
Test plan:
==========
1. Enable the display of identifiers in
OPACAuthorIdentifiersAndInformation.
2. Add a 024 field with a VIAF ID to any corporate authority record,
e.g.:
024 7# $a 1234 $2 viaf
3. Use this authority heading as the main entry (110 or 111) in any
bibliographic record.
4. In the OPAC view of that bibliographic record, verify that the Author
information tab does not display information from the main entry.
5. Use the same heading as an added entry (710 or 711). In the OPAC
view, verify that the Author information tab now displays information
from the added entry, but still not from the main entry.
6. Apply the patch ; restart_all.
7. Verify that OPAC now displays information from the 110/111
heading fields in the bibliographic record.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Check of biblioitemnumber in MARC uses $biblionumber instead of $biblioitemnumber
It returns for example :
Biblionumber 542620 has biblioitemnumber '542620' but should be '542620' in 090$a
Test plan (for MARC21 only) :
1) Choose a biblio record id=X
2) Change with SQL :
update biblio_metadata set metadata = UpdateXML( metadata , '//datafield[@tag="999"]/subfield[@code="d"]' , CONCAT('<subfield code="d">1111</subfield>') ) where biblionumber = X
3) Run audit on this record
4) Check message is correct
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Update ILL/Batch.pm and ILL/Batch/Status.pm to utilize this new modulename instead
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Test plan:
1. Navigate to Reports > Create from SQL.
2. Create a new report with a name and the following SQL query:
SELECT 'test', 1, '2'
3. Run the report.
4. Under the Download dropdown, select "Open Document Spreadsheet".
5. Verify a ".ods" file is downloaded.
6. Open the downloaded spreadsheet.
7. Verify the two number-like columns are handled as such. This depends
on your choice of spreadsheet software:
- In Excel, the numbers should not have a leading single quote in the
edit-bar when selected.
- In Google Sheets, the numbers should be right-aligned, and
selecting both of them should display their sum in the bottom
toolbar.
8. Run `prove t/Koha/Util/OpenDocument.t` and verify all tests were
successful.
Sponsored-by: Open Fifth
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Anneli Österman <anneli.osterman@koha-suomi.fi>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Before this patch, control fields were not modifiable using this subroutine.
Control fields use character positions instead of subfields. When modifying
control fields, this patch makes the subroutine's parameter "subfield" to indicate
a character position. An example
Koha::SimpleMARC::update_field({
record => $record,
field => '008',
subfield => '2',
values => [09],
});
changes field 008 starting from character position 2 to contain values "09".
This replaces char pos 2 with "0", and char pos 3 with "9".
To test:
1. prove t/SimpleMARC.t
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Anneli Österman <anneli.osterman@koha-suomi.fi>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Add explicit `use JSON::PP ()` to Elasticsearch.pm and the test file
to make the dependency on JSON::PP::false visible rather than relying
on YAML::XS loading it as a side effect.
Extend the get_elasticsearch_mappings() test subtest to assert that
date_detection and numeric_detection are disabled at the mapping root.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
When ElasticsearchMARCFormat is set to ARRAY, the marc_data_array
field uses dynamic:true mapping. Elasticsearch auto-detects field
types based on the first document indexed. If a MARC subfield value
looks like a date (e.g. 2024-01-15 in 952$e), ES locks that
subfield to type date. Subsequent records with non-date values in
the same subfield fail to index with a document_parsing_exception.
This patch sets date_detection and numeric_detection to false at
the mapping root level, so ES treats all dynamically mapped fields
as text instead of guessing types.
Test plan:
1. $ ktd --search-engine es8 --name test-es --proxy up -d
$ ktd --name test-es --wait-ready 180
$ ktd --name test-es --shell
2. k$ koha-mysql kohadev -e "UPDATE systempreferences
SET value='ARRAY'
WHERE variable='ElasticsearchMARCFormat';"
k$ echo 'flush_all' | nc -q1 memcached 11211
k$ koha-mysql kohadev -e "DELETE FROM biblio_metadata;
DELETE FROM items; DELETE FROM biblioitems;
DELETE FROM biblio;"
k$ rebuild_elasticsearch.pl -d -r --biblios
3. Add a record with date-like 952$e and index it:
k$ koha-shell kohadev -p -c 'perl -MC4::Biblio
-MMARC::Record -MMARC::Field -MKoha::Item -e "
my \$r = MARC::Record->new();
\$r->append_fields(
MARC::Field->new(q{245},q{1},q{0},a=>q{Record one}),
MARC::Field->new(q{942},q{ },q{ },c=>q{BK}));
my (\$bn) = C4::Biblio::AddBiblio(\$r, q{});
Koha::Item->new({biblionumber=>\$bn,
biblioitemnumber=>\$bn, homebranch=>q{CPL},
holdingbranch=>q{CPL}, itype=>q{BK},
booksellerid=>q{2024-01-15}})->store;
print qq{Added biblio \$bn\n};"'
k$ rebuild_elasticsearch.pl --biblios -bn <biblionumber>
4. Check the mapping for 952$e:
k$ curl -s es:9200/koha_kohadev_biblios/_mapping/field/\
marc_data_array.fields.952.subfields.e
=> FAIL: type is "date" (should be "text")
5. Add a record with non-date 952$e and index it:
k$ koha-shell kohadev -p -c 'perl -MC4::Biblio
-MMARC::Record -MMARC::Field -MKoha::Item -e "
my \$r = MARC::Record->new();
\$r->append_fields(
MARC::Field->new(q{245},q{1},q{0},a=>q{Record two}),
MARC::Field->new(q{942},q{ },q{ },c=>q{BK}));
my (\$bn) = C4::Biblio::AddBiblio(\$r, q{});
Koha::Item->new({biblionumber=>\$bn,
biblioitemnumber=>\$bn, homebranch=>q{CPL},
holdingbranch=>q{CPL}, itype=>q{BK},
booksellerid=>q{My Vendor Name}})->store;
print qq{Added biblio \$bn\n};"'
k$ rebuild_elasticsearch.pl --biblios -bn <biblionumber> -v -v -v
=> FAIL: document_parsing_exception - failed to parse date field
[My Vendor Name]
6. Apply this patch, repeat from step 2
7. Repeat step 4:
=> SUCCESS: type is "text"
8. Repeat step 5:
=> SUCCESS: No indexing errors
9. Sign off :-D
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This is a quite ugly solution, but that may answer end-user needs.
It add the option the group items by status on the bibliographic record
detail page.
However it will have VERY bad impact on performances for record with a
lot of items, as we are going to determine the status of each items of
the record.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Testing plan:
- Set EmailPurchaseSuggestions to BranchEmailAddress
- Go to create suggestion on the staff side /suggestion/suggestion.pl?op=add_form
- Create a suggestion with Library: Any
- 500 error: inbound_email_address is not a method of undef
- Apply patch & repeat
- Suggestion created
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
The indexation step has been removed by
commit d2820f57d6
Bug 34788: (QA follow-up)
Koha/ERM/EHoldings/Title.pm
- C4::Biblio::ModBiblio( $record, $self->biblio_id, '' );
+ C4::Biblio::ModBiblio( $record, $self->biblio_id, '', { skip_record_index => 1 } );
From the commit message:
2) The background job now uses skip_record_index to avoid queuing indexing jobs for every new biblio and instead queues one job at the end
it's then missing when we add or edit a local title.
Test plan:
1 - Enable ERMModule
2 - Go to ERM
3 - eHoldings->Titles
4 - New title
5 - Populate the form
6 - Check the create bibliographic record box
7 - Save
=> Confirm that the biblio is now indexed
8. Edit the title, modify a value, tick the box, save
=> Confirm that the biblio is now indexed with the new value
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Rather than excluding the entire about.tt files from codespell scanning,
annotate only the specific lines containing the proper noun "Provence"
(SAN-Ouest Provence, a French library co-operative and Koha sponsor) with
an inline TT comment that codespell recognises as a per-line ignore.
This lets codespell continue to check the rest of each file for genuine
spelling mistakes, while still catching accidental "Provence" for
"Province" in any other template not carrying this annotation.
Replaces the file-level exceptions added to Koha::Devel::Files in
favour of the more surgical inline approach.
Test plan:
1. Apply patch
2. Verify "Provence" on the annotated lines produces no codespell output:
codespell -d --ignore-words .codespell-ignore \
koha-tmpl/intranet-tmpl/prog/en/modules/about.tt
=> no output
3. Verify the rest of about.tt is still checked by introducing a
deliberate typo elsewhere in the file and confirming codespell flags it
4. prove xt/author/codespell.t
=> All tests pass
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
The ci-results repo stores all failures from a single test run as one
flat JSON array, regardless of whether they came from the pl, tt, or js
filetype pass. On the next incremental run, get_files_to_test('pl') reads
that entire list -- including any .tt failures -- and calls
remove_exceptions with the 'pl' context. Since .tt files only appear in
the tt exception list, they survive the filter and get codespell'd in the
pl pass where nothing is configured to exclude them.
The fix filters stored failures by filetype before applying exceptions,
so each pass only re-checks failures that belong to its own type.
Test plan:
1. Apply all patches
2. Add a file to the tt codespell exceptions in Koha::Devel::Files that
contains a word codespell would flag
3. Introduce a deliberate failure in another file so the ci-results repo
records the excepted .tt file as a previous failure
4. On the next incremental run, confirm the excepted .tt file is not
re-tested in the pl or js pass
5. prove xt/author/codespell.t
=> Tests pass; no false positives from cross-filetype re-checks
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Correct genuine spelling errors identified by the codespell 2.4.x
full scan after the KTD container OS upgrade. Changes are in
comments, POD, strings, and template text across 55 files.
"SAN Ouest Provence" (the French library co-operative and Koha
sponsor) is preserved as a proper noun. Perl copyright lines
receive an inline # codespell:ignore annotation; about.tt files
(contributor attribution pages) are added to the Koha::Devel::Files
codespell exception list so that future uses of "Provence" as a
typo for "Province" are still caught in all other files.
Notable user-visible fixes:
- "submiting" -> "submitting" in authority and bibliographic merge screens
- "deleteable" -> "deletable" in patron exception messages
- "Acquistions" -> "Acquisitions" in reports dictionary
Test plan:
1. Apply patches
2. In the staff interface, go to Cataloguing > Merge records and
confirm the submit button reads "Submitting" (not "Submiting")
3. In the staff interface, go to Authorities > Merge and confirm
the same correction
4. Attempt to delete a patron with outstanding issues and confirm
the error message reads "deletable" (not "deleteable")
5. prove xt/author/codespell.t
=> All tests pass
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Three issues caused the EdifactModal_spec.ts test suite to fail:
1. insertSampleEdifactMessages inserted a vendor with listprice='USD' and
invoiceprice='USD', but those columns have FK constraints against the
currency table and USD is not pre-populated in KTD's default install.
Drop the currency columns from the INSERT so they default to NULL.
2. Koha::Edifact::File::vendor() and basket() called
Koha::Acquisition::Bookseller and Koha::Acquisition::Basket without
use-ing them. The edifactmsgs.pl table embeds both on every API call,
so a missing module caused a 500 and an empty table.
3. showMessage() already appended &format=json before delegating to
showMessageFromUrl(), which then appended it a second time. Remove the
redundant copy from showMessage().
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The existing 'days: 289 hours, 2 holidays' test asserted 8-1=7 but its own
description named 2 holidays. The expected value matched the buggy behaviour
(loop exiting before the holiday end date) rather than the correct result.
Update to 8-2=6 to reflect the fix.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
In Koha::Calendar::days_between, the holiday-subtraction loop uses a comparison
that includes time. If the end_dt is a holiday and its time is earlier than
the start_dt time (common in renewals), the loop terminates before processing
the end_dt, leading to an extra fine day.
This patch truncates both dates to the beginning of the day after cloning
and setting the time zone, ensuring the loop correctly processes all
calendar days between the two dates.
Test Plan:
1. Set 'finesCalendar' syspref to 'noFinesWhenClosed'.
2. Set 'CalculateFinesOnReturn' syspref to 'Yes'.
3. Set 'finesMode' syspref to 'Production'.
4. Define a special holiday for today in the library calendar.
5. Identify an overdue item for a patron. Manually set its 'date_due' in the
database to several days ago with a time of 23:59:00.
6. Renew the item today at any time BEFORE 23:59:00 (e.g., 10:00 AM).
7. Observe that an extra day of fine is charged because today (the holiday)
was not subtracted.
8. Apply this patch.
9. Repeat the test and observe that the holiday is now correctly subtracted.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Martin Renvoize
Tomás Cohen Arazi
Kyle M Hall
Jonathan Druart
Pedro Amorim
Baptiste Wojtkowski
Nick Clemens
Janusz Kaczmarek
Fridolin Somers
Olivia Reynolds
Andrew Fuerste Henry
Lari Taskula
Eric Phetteplace
apirak