GetNormalizedUPC duplicated logic already available via the
MetadataExtractor (Bug 34828) and exposed as $biblio->normalized_upc.
All callers migrated:
- Scripts with $biblio available: use $biblio->normalized_upc
- C4::Search (only MARC::Record): reuses the extractor instance
already created for OCLC
Assisted-by: Sonnet 4.6 (Anthropic)
Test plan:
1. Apply patch
2. Run:
$ ktd --shell
k$ prove t/db_dependent/Koha.t \
t/db_dependent/Koha/Biblio.t
=> SUCCESS: Tests pass
3. Verify UPC numbers still display in OPAC and staff detail pages
(Syndetics cover images)
4. Sign off :-D
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Replaces all calls to GetNormalizedOCLCNumber() and
Koha::Util::MARC::oclc_number() with the extractor pattern.
OPAC scripts pass their already ViewPolicy-filtered $record directly
to the extractor. This preserves the existing filtering behavior
without introducing new abstraction layers:
Koha::Biblio::Metadata::Extractor->new( { metadata => $record } )->get_normalized_oclc
Intranet scripts and the Koha::Biblio convenience method use the
raw cached extractor (035 visibility is not affected by ViewPolicy
in practice, and intranet doesn't filter):
$biblio->normalized_oclc
Test plan:
1. Verify OCLC numbers display on OPAC detail, ISBD, MARC detail
2. Verify staff catalogue detail and search results
3. prove t/db_dependent/Koha/Biblio.t
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Assisted-by: Sonnet 4.6 (Anthropic)
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Moves get_normalized_oclc from the MARC21-specific extractor to the
base ::MARC class (035$a is flavor-independent) with improved matching:
- Case-insensitive OCoLC detection (matches ocolc, OCOLC, etc.)
- Generic orgcode prefix stripping (any parenthesized prefix, not
just literal "(OCoLC)") for resilience against cataloging typos
Test plan:
$ prove t/db_dependent/Koha/Biblio/Metadata/Extractor/MARC/MARC21.t
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Assisted-by: Sonnet 4.6 (Anthropic)
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Koha::Biblio::Metadata->store() now repairs recoverable MARCXML faults
before saving, so this test's invalid data no longer reaches the DB
and detail.pl never hits a decoding error.
Write the metadata directly via the DBIC row instead, bypassing
store()'s repair, so the test still exercises detail.pl's own
defensive handling of corrupt metadata.
Test plan:
prove t/db_dependent/selenium/regressions.t
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Tests verify that EDI ORDRSP cancellations now use
Koha::Acquisition::Order->cancel() rather than a raw ModOrder call:
- Order status, cancellation reason and date are set correctly
- On-order items linked via aqorders_items are deleted
- Purchase suggestions linked to the biblio are reverted from
ORDERED back to ACCEPTED
- Non-cancellation actions update suppliers_report only
- Missing ordernumbers are handled gracefully without dying
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: Sharon Dugdale <sharon.dugdale@cumberland.gov.uk>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Test plan:
1) Apply just the unit test patch
2) Run t/db_dependent/Koha/Patrons.t
3) Test should fail
4) Apply remaining patch
5) Repeat step 3
6) Test should pass
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
ISBNs beginning with 979 do not have an ISBN-10 version, and need to be normalized to an ISBN-13 format. All others, if valid, can safely be normalized to ISBN-10 format (current behaviour).
To set up
1. Download the attached sample records with 979 ISBNs
2. Log into the staff interface and go to Cataloguing
3. Go to Stage MARC records for import and upload the sample MARC file
4. Stage and import the biblio records
5. Once imported, open the detail pages for some of the records
6. In another tab, do a catalogue search and open the detail pages for ISBN-10 records, for example biblionumbers 6 and 54
To test
1. Apply the patch and restart services
2. Log into the staff interface and go to Koha Administration, Global system preferences, Enhanced Content tab
3. Cycle through each cover image service, enabled by the system preferences
4. With each cover image service is enabled, refresh the record detail pages you have open. The ISBN-10 records should CONTINUE to show their cover images. The newly imported ISBN-13 records should NOW show their cover images.
5. Confirm tests pass: t/db_dependent/Koha.t
testing with ISBN 9798885795692 should return this image from Amazon: https://images-na.ssl-images-amazon.com/images/P/9798885795692.01.LZZZZZZZ.jpg
testing with ISBN 9798876536914 should return this image from Amazon: https://images-na.ssl-images-amazon.com/images/P/9798876536914.01.LZZZZZZZ.jpg
Sponsored-by: Horowhenua Libraries Trust
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
1- Corrected tests according to the comments
2- Removed unused import
3- Added syspref check of 'patronimages'
4- Tweaked swagger path schema
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Api changes adding GET route for patronimage
Test plan:
1- Apply the patch
2- Run `npm run api:bundle`
2- Go to administration -> sys.pref. -> set patronimages to allow
3- Log in koha using a user with "list_borrowers" or "edit_borrowers" privileges
4- Have a patron with a profile image (add an image if necessary by
searching for a patron and then clicking on the default profile image on
the top left)
5- Open a an API testing tool (such as postman), write
{intranet_url}/api/v1/patrons/{borrowernumber}/default_image
where {intranet_url} is the base intranet url of koha and {borrowernumber}
is the borrowernumber of a borrower that has an image
6- Add OAuth 2.0 credentials coming from koha (patron -> more -> manage
api keys) and generate a token
7- Send the request and notice the image of the patron is returned
8- Do step 5-7 with a borrowernumber of a patron without an image and a
borrowernumber of a patron that doesn't exist (and other edge cases
where there should be an error message displayed such as "Access
forbidden", "Authentication required", etc.)
9- Notice an appropriate error message is displayed
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Bug 35797: Rename test file from 'image.t' to 'patron_images.t'
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
To test:
1. prove t/db_dependent/api/v1/idp.t
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch adds a parameter to call the method intranet_catalog_biblio_enhancements_toolbar_button with a biblionumber in the page catalogue/detail.pl.
Modification of cat-toolbar.inc to display new buttons correctly.
To test :
1. Apply the patch
2. Take a plugin who implements the method intranet_catalog_biblio_enhancements_toolbar_button
3. Check if the parameter biblionumber is passed correctly at this method
4. Create the new button in HTML in the plugin method
(button must be like the ones that are in the toolbar of the page detail.pl)
5. Go to the page in browser and check if the new button is displayed
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Added a unit test to confirm that search_for_display orders manually positioned news first, followed by unnumbered news by publication date descending.
Test passes:
prove t/db_dependent/Koha/AdditionalContents.t
Signed-off-by: Gretchen Maxeiner <MaxeinerGL@gcc.edu>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This test built a "bad" biblio by storing invalid MARCXML directly via
Koha::Biblio::Metadata->store(), which now rejects it. Simulate
legacy corrupt data already in the database by writing it straight to
the DBIC row, bypassing the new validation, instead.
Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com>
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
Sponsored-by: OpenFifth
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
AddBiblio/ModBiblio/ModBiblioMarc gain an optional warnings arrayref
option: any repair messages recorded by Koha::Biblio::Metadata->store()
(see previous commit) are appended to it via object_messages, so a
caller can show the cataloguer what was automatically fixed instead of
the fix happening invisibly.
AddBiblio also now distinguishes Koha::Exceptions::Metadata::Invalid
from other errors in its catch block and rethrows it, so a caller can
tell "this record is unrecoverable" apart from an unexpected internal
error and react accordingly (see following bugs for callers doing so).
The inline StripNonXmlChars calls in TransformHtmlToMarc are removed,
since stripping is now handled centrally and consistently by
Koha::Biblio::Metadata->store() rather than ad-hoc at every call site.
Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com>
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
Sponsored-by: OpenFifth
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Koha::Biblio::Metadata->store() now validates that MARCXML round-trips
through MARC::Record before saving it, rather than saving whatever was
given and letting corruption surface later as an opaque failure
somewhere downstream.
Two faults are recovered automatically, and either or both may apply
to the same record:
- non-XML characters (recorded as nonxml_stripped, recovered via
StripNonXmlChars)
- datafields with no subfields at all, which MARC::File::XML refuses
to parse (recorded as empty_datafield_stripped, recovered by
removing the empty datafield)
Each recovered fault is logged, reported via object_messages for the
caller to act on, and recorded as a biblio_metadata_errors row (tag
and subfield included where identifiable) for later review - except
where an identical row already exists for the same metadata_id, so a
source record that stays dirty across repeated saves (e.g. re-harvested
from an external repository) doesn't pile up duplicate rows for the
same fault. Existing rows are otherwise left alone on a clean save,
since they're review flags requiring explicit resolution, not a cache
to invalidate.
If the MARCXML cannot be recovered at all, Koha::Exceptions::Metadata::Invalid
is thrown. The recovery logic is exposed as a repair_marcxml class
method so other callers that parse MARCXML ahead of a store() (see
following bugs) can reuse it instead of re-implementing it.
Signed-off-by: Baptiste Bayche <baptiste.bayche@inlibro.com>
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
Sponsored-by: OpenFifth
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Signed-off-by: Sanjar Tulkinov Anvar o'g'li <sanjarbiy11@gmail.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
login_core only cleared leftover session state on the failed-auth
branch. The earlier 'SIP access not authorized' return (unknown
terminal id) left any account/ils/institution/policy/credentials from
a prior connection on a reused prefork worker in place.
Move the reset to the top of login_core so every non-success exit path
leaves clean state, and only a successful api_auth re-establishes the
session. This keeps the no-authenticated-state-until-auth invariant in
a single place rather than duplicating the key list per branch.
Also extend the unit test to cover the unknown-terminal path on a
reused worker.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Signed-off-by: Sanjar Tulkinov Anvar o'g'li <sanjarbiy11@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
The tests for desensitize and alert flags were failing because
items were created with random itemtypes that could have
sip_magnetic=1. This caused unexpected behavior since magnetic
media items don't get desensitized and trigger alert flags.
Fix by explicitly creating itemtypes with sip_magnetic=0 in:
- test_checkout_desensitize
- test_renew_desensitize
- test_checkin_v2
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: Marcus Michel <marcus.michel@rbkc.gov.uk>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Add unit tests for SIP magnetic media support.
Test plan
1) Run t/db_dependent/SIP/Transaction.t in ktd and confirm it passes
Sponsored-by: OpenFifth <https://openfifth.co.uk/>
Signed-off-by: Marcus Michel <marcus.michel@rbkc.gov.uk>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch adds age restriction handling to the Koha namespace:
- Koha::Biblio->age_restriction: metadata extractor that parses the
agerestriction field using the AgeRestrictionMarker system preference.
Returns the numeric age or undef. Uses single return with last pattern.
- Koha::Policy::Biblio::AgeRestriction->check($biblio, $patron): policy
class that evaluates whether a patron meets the age restriction.
Returns a Koha::Result::Boolean (with payload on restriction).
Test plan:
1. Apply this patch
2. Run:
$ ktd --shell
k$ prove t/db_dependent/Koha/Policy/Biblio/AgeRestriction.t
=> SUCCESS: Tests pass!
3. Sign off :-D
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
_upload_file, _download_file, _list_files and _rename_file's 'no directory configured' checks still called add_message() directly, skipping the status persistence this bug added elsewhere in the file. Route them through _abort_operation() so all error paths in Local.pm stay consistent.
prove t/db_dependent/Koha/File/Transport/Local.t
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Two related inconsistencies remained across the three transport
backends' error-handling paths:
1) Only Koha::File::Transport::SFTP's _abort_operation() persisted an
"errors" status snapshot to the status column on every failed
operation. FTP never did, and Local had no _abort_operation() at
all - so an SFTP transport's displayed status could reflect a
single transient failure from real usage (e.g. an EDI cron job)
while the exact same failure on FTP/Local left the stored status
untouched (stale, possibly still showing the last explicit
test_connection() result). This is exactly the kind of unclear
status reporting bug 42656 is about.
2) FTP's _abort_operation() carried its extra detail under an
operation_params key, while SFTP (and Local's plain add_message
calls) used path. The change_directory() success payload had the
same split: FTP used directory, SFTP and Local used path. The
admin template's change_directory row display depended on FTP's
mismatched key names, which meant it silently failed to show the
tested directory for SFTP transports.
Adds a shared _record_error($operation, \%payload) to the
Koha::File::Transport base class - add_message() plus status
persistence - and routes all three backends' _abort_operation() (Local
gains one) through it. Renames FTP's operation_params/directory keys
to path throughout, matching SFTP and Local, and simplifies the
template to read operation.detail.path unconditionally instead of
ORing two differently-named keys.
_record_error() persists the full trace of operations recorded on the
transport object so far this session (every success leading up to the
failure), not just the failing operation in isolation - the same
"full trace" shape TestTransport::process() already builds for an
explicit test_connection() run. A long-lived transport used across a
multi-step operation (e.g. an EDI download loop touching several files
before one fails) will therefore show every successful step leading up
to the failure. It also logs the failure - including the built trace -
via Koha::Logger at warn level, so it's visible in the Koha logs
without needing to query the transport's status column.
Test plan:
1. prove -r t/db_dependent/Koha/File/ t/db_dependent/Koha/BackgroundJob/TestTransport.t t/db_dependent/Koha/Edifact/Transport.t t/db_dependent/api/v1/file_transports.t
=> All tests successful (48 tests total), including:
- new "_abort_operation persists status consistently" subtests
in FTP.t, SFTP.t and Local.t that reload the transport from
the DB after a simulated failure and confirm the status
column was updated with the correct operation code and path
detail;
- a new "_record_error() tests" subtest in Transport.t proving
the persisted trace includes a prior success message plus the
failure (not just the failure alone), and that a warn-level
log line mentioning the operation and error was emitted.
2. koha-qa.pl -v 2 --more-tests
=> [PASS], no critic/pod warnings.
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Two issues in the operation_description BLOCK's status text, both
flagged in review:
1) The IF/SET that computed the " passed."/" failed." status text sat
between the SWITCH statement and its first CASE. Template Toolkit
only executes CASE-labelled content inside a SWITCH block, so this
was dead code - status was never actually set, and every operation
row silently rendered with no passed/failed text at all. It also
checked operation.error, a field that doesn't exist on the
operation hash (the error message lives under operation.detail.error,
and whether the operation failed is operation.status eq 'error');
even if the code path were reachable it would always have taken the
"passed" branch.
2) Relying on a leading space baked into the " passed."/" failed."
string literals to separate them from the preceding </span> is
fragile. Made the space explicit in the markup instead, and dropped
the leading space from the string values.
Moves the status computation above the SWITCH (using operation.status),
and adds an explicit space before [% status | html %] on every CASE line.
Also adds t/db_dependent/Koha/BackgroundJob/TestTransport.t, which was
entirely missing: it exercises process() end-to-end (enqueue, then
process, then reload both the job and the transport) and confirms the
persisted trace records every operation in sequence - successes before
the failure that ended the run - rather than collapsing to just the
final error.
Test plan:
1. prove t/db_dependent/Koha/BackgroundJob/TestTransport.t
=> All tests successful, confirming process() persists the full
operation sequence (order preserved, at least one success and
one error present) on both the job and the transport's own
status column.
2. Manually rendered the fixed operation_description BLOCK with
Template Toolkit against mock operations (mixed success/error) and
confirmed every row now shows "Connection passed.", "Change
directory (/incoming) passed.", ..., "Change directory (/outgoing)
failed.- error:No such directory" - status text present with
correct spacing, where before the fix no status text rendered at
all.
3. prove -r t/db_dependent/Koha/File/ t/db_dependent/Koha/Edifact/Transport.t t/db_dependent/api/v1/file_transports.t
=> All tests successful (47 tests total), no regressions.
4. koha-qa.pl -v 2 --more-tests
=> [PASS], no critic/pod warnings.
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Koha::File::Transport's three backends disagree on what
change_directory(undef) means: FTP navigates to the server root,
SFTP falls back to the SFTP session's own home directory, and Local
fell back to the configured download_directory or, if that's also
unset, silently to '.' - the Koha process's own working directory,
which has no relation to any configured transport location.
test_connection() calls change_directory(undef) unconditionally for
both the download and upload directory checks on every config save
(via the auto-enqueued TestTransport background job). For a Local
transport configured with only one of download_directory/
upload_directory, the untested direction's list_files() call would
silently list the Koha process's own cwd (e.g. the Koha install root)
instead of failing cleanly.
Add a private _working_directory($configured_field) helper that
returns the tracked change_directory() override or the given
configured field, but never falls back to '.'. _upload_file,
_download_file, _list_files and _rename_file now use it and record a
clear "not configured" error instead of silently acting on an
unrelated directory when neither is set.
FTP's root and SFTP's home-directory fallbacks are left as-is: unlike
Local's '.', they navigate to a real, meaningful location for that
specific connection rather than leaking unrelated filesystem content,
so they aren't part of this fix.
Test plan:
1. prove t/db_dependent/Koha/File/Transport/Local.t
=> All tests successful, including the new subtest proving that
list_files()/rename_file()/download_file() fail cleanly (with a
"not configured" error) rather than falling back to '.' when
their respective directory isn't configured, while the
configured direction (upload, in the test) still works.
2. prove -r t/db_dependent/Koha/File/ t/db_dependent/Koha/Edifact/Transport.t t/db_dependent/api/v1/file_transports.t
=> All tests successful (43 tests total), no regressions.
3. koha-qa.pl -v 2 --more-tests
=> [PASS], no critic/pod warnings.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
t/db_dependent/Log.t
- Add subtest verifying logscriptaction() logs to the SCRIPTS module,
respects the ScriptLog preference, prefixes loginfo with $0, appends
the info parameter, and honours custom action values.
t/Koha/Script_scrub_argv.t
- New test file exercising all branches of _scrub_argv():
--flag value form, --flag=value form, non-sensitive flags, case-
insensitive matching, multiple sensitive patterns (token, secret,
apikey, api_key), and a flag immediately followed by another flag.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
FTP, Local and SFTP list_files() were unified to return the same flat
shape, but consumers still had no portable way to ask a transport for
its current remote directory - the underlying libraries' own methods
of that name don't agree on what "cwd" means. Net::SFTP::Foreign's
cwd() is a pure read-only accessor, but Net::FTP's cwd() is a mutator:
called with no argument it defaults to "/", issues a real CWD command,
and returns a boolean rather than a path.
The koha-plugin-transport-browser plugin discovered this the hard way:
it read $transport->{connection}->cwd directly to capture the initial
working directory, which happened to work for SFTP but silently reset
the FTP connection to root and captured "1" (a boolean) as the "path"
on every request, breaking directory navigation for FTP transports.
Add a public current_directory() method to the Koha::File::Transport
base class, following the same connect-on-demand pattern as
list_files()/upload_file()/etc, with a _current_directory() hook each
subclass implements using its own real read-only accessor: pwd() for
FTP, cwd() for SFTP, and the tracked {current_directory} (falling back
to download_directory) for Local. Consumers should use this instead of
reaching into transport internals.
Test plan:
1. prove t/db_dependent/Koha/File/Transport.t
t/db_dependent/Koha/File/Transport/FTP.t
t/db_dependent/Koha/File/Transport/Local.t
t/db_dependent/Koha/File/Transport/SFTP.t
=> All tests successful, including the new current_directory()
tests subtest in each transport test file, which mock the
underlying libraries (Net::FTP / Net::SFTP::Foreign) to assert
each backend reads its own real read-only accessor.
2. koha-qa.pl -v 2 --more-tests
=> [PASS] for all four changed modules, no critic/pod warnings.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
The FTP and Local transports both explicitly filter out '.' and '..'
pseudo-entries when listing a directory, but the SFTP transport never
did - it relied on the assumption that Net::SFTP::Foreign's ls() already
excludes them. It doesn't: ls() returns exactly whatever entries the
remote server includes in its SSH_FXP_READDIR response, with no default
filtering. Whether '.'/'..' show up therefore depends entirely on the
remote SFTP server implementation, producing inconsistent results between
otherwise identically configured SFTP accounts.
Test plan:
1. prove t/db_dependent/Koha/File/Transport/SFTP.t
=> fails before this patch (listing includes '.' and '..')
=> passes after this patch
2. prove -r t/db_dependent/Koha/File/Transport/ t/db_dependent/Koha/File/Transports.t
=> all still pass
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
The MLSD path, SFTP, and Local always return all six documented
list_files() keys (filename, longname, size, perms, mtime, type) - undef
where the value is unknown, but the key always present. The NLST
fallback path (servers without MLSD support) only set filename, size,
mtime and type, omitting longname and perms entirely rather than
setting them to undef.
Functionally harmless for consumers reading $file->{perms} (Perl
returns undef for a missing hash key too), but it breaks the "same flat
shape" contract this bug established, and would trip up any consumer
using exists() instead of defined() to check for a key.
Test plan:
1. prove t/db_dependent/Koha/File/Transport/FTP.t
=> All tests successful, including the new assertion that longname
and perms keys are present (as undef) on the NLST fallback path.
2. koha-qa.pl -v 2 --more-tests
=> [PASS], no critic/pod warnings.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Net::SFTP::Foreign's ls() returned file metadata with size, permissions
and modification time nested inside a Net::SFTP::Foreign::Attributes
object, while FTP and Local returned flat hash keys and silently
dropped directories. Callers of list_files() needed backend-specific
parsing to get at file metadata or to know a directory even existed,
defeating the purpose of the shared transport API.
All three transports now return the same flat hashref shape per entry:
filename, longname, size, perms, mtime, type ('file'/'directory'/
'other'). Directories are included everywhere; '.' and '..' remain
excluded everywhere. FTP's MLSD path prefers the UNIX.mode fact for
exact Unix permissions, falling back to an approximation from the RFC
3659 perm capability fact when UNIX.mode isn't available, and parses
the modify fact into mtime; its NLST fallback (servers without MLSD)
probes each entry with SIZE/MDTM to recover size, mtime and a
best-effort type.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Koha::File::Transport::FTP::_list_files parsed the LIST / "ls -l" output
with a non-greedy date capture that only consumed the first of the three
date tokens (the month), leaving the day and time glued to the front of
the filename. Every filename returned for an FTP transport was therefore
prefixed with the listing date, e.g.
-rw-r--r-- 1 user group 1234 Jan 01 12:00 QUOTES_413514.CEQ
=> filename "01 12:00 QUOTES_413514.CEQ"
Callers (Koha::Edifact::Transport, marc_ordering_process.pl) then asked
the server for a file that does not exist, so no files were downloaded,
renamed or deleted over FTP. SFTP and Local transports were unaffected.
This regressed when the EDI transport was moved onto the file_transports
system (bug 38489). The previous Koha::Edifact::Transport::ftp_download
listed files with Net::FTP->ls (NLST, bare names); the refactor switched
to Net::FTP->dir (LIST) so it could return the richer hashref shape used
by the SFTP and Local transports, but the parse was faulty.
The date field in "ls -l" output is a fixed run of three whitespace
separated tokens (Mon DD HH:MM or Mon DD YYYY). Consume it explicitly so
the filename is captured cleanly, preserving spaces and still skipping
directories. The LIST-based approach and the filename/longname/size/perms
return shape are kept so the structure stays consistent across transport
classes (relied on by consumers, including out-of-tree ones).
Test plan:
1) prove t/db_dependent/Koha/File/Transport/FTP.t
2) git checkout Koha/File/Transport/FTP.pm and re-run: the list_files()
subtest fails on the filename/size/perms assertions
3) prove t/db_dependent/Koha/Edifact/Transport.t
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
The default `_match=contains` matching builds a LIKE query even on
integer columns, so a plain `?object=42` also matches object values
like 420 or 142. The test's dynamically-assigned patron borrowernumber
(auto_increment, never reset by transaction rollback) could drift into
a value containing "42" as a substring, intermittently inflating the
match count in CI. Add `_match=exact` since the test's intent is an
exact object id match.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
local_holdgroup_match was being hardcoded to 1 in the two LocalHoldsPriority
allocation paths (GiveLibrary and GiveLibraryGroup), regardless of whether the
chosen item actually satisfied the LocalHoldsExclusivity conditions. This
caused staff to be directed to pull an item that would not trigger the hold at
checkin when the two features used different patron/item controls.
The fix removes the premature local_holdgroup_match => 1 from both LHP paths and
lets _flag_local_holdgroup_matches evaluate all allocated items using the correct
LocalHoldsExclusivity* sysprefs.
Also fixes a pre-existing missing bind-variable placeholder in the
INSERT INTO tmp_holdsqueue statement (17 columns, only 16 ?).
Test updates:
- Existing local_holdgroup_match test gains the matching LocalHoldsExclusivity
sysprefs so the flag is set via the correct code path.
- New subtest reproduces the reported bug: LHP (HomeLibrary/homebranch) targets
a non-local item while exclusivity (PickupLibrary/holdingbranch) should not
flag it — confirms local_holdgroup_match is 0 after the fix.
Signed-off-by: Trevor Diamond <trevor.diamond@mainlib.org>
Sponsored-by: Main Library Alliance <https://www.mainlib.org/>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
LocalHoldsExclusivity and LocalHoldsPriority are adjacent but distinct
features: priority alters *which patron's hold* gets filled first, while
exclusivity restricts *which items* may fill a hold during a time window.
Each should be usable independently of the other.
Changes:
- Rename LocalHoldsPriorityExclusivityPeriod to LocalHoldsExclusivityPeriod
- Add LocalHoldsExclusivityPatronControl (PickupLibrary/HomeLibrary)
- Add LocalHoldsExclusivityItemControl (homebranch/holdingbranch)
- HoldsQueue: add _flag_local_holdgroup_matches() that flags
local_holdgroup_match on allocated items using hold-group matching
and the new exclusivity control prefs, regardless of LocalHoldsPriority
- Reserves: CheckReserves exclusivity check now computes its own
"is this item local" using the new prefs and hold-group siblings,
independent of the LocalHoldsPriority matching variables
- Atomicupdate handles both fresh install and rename-from-old-pref paths
- Pref UI note clarifies independence from LocalHoldsPriority
Test plan:
1. Apply patch, run updatedatabase.pl
2. Verify three new sysprefs exist in Administration > System preferences
> Circulation: LocalHoldsExclusivityPeriod, LocalHoldsExclusivityPatronControl,
LocalHoldsExclusivityItemControl
3. Set LocalHoldsPriority to "Don't give" (None)
4. Create a library hold group with at least two libraries (e.g. Lib-A, Lib-B)
5. Create a third library (Lib-C) NOT in the hold group
6. Create an item at Lib-B
7. Place a hold for a patron at Lib-A (pickup at Lib-A)
8. Set LocalHoldsExclusivityPeriod to 7, PatronControl to HomeLibrary,
ItemControl to homebranch
9. Run the holds queue builder (misc/cronjobs/holds/build_holds_queue.pl)
10. Verify hold_fill_targets.local_holdgroup_match = 1 for that hold
(item at Lib-B is in the same hold group as patron at Lib-A)
11. Check in the item at Lib-C — it should NOT fill the hold (exclusivity
blocks the non-local item within the 7-day window)
12. Check in the item at Lib-B — it SHOULD fill the hold (local match)
13. Reset: delete the hold, create a new one with a patron at Lib-C
(pickup at Lib-C). Run the queue again.
14. Verify hold_fill_targets.local_holdgroup_match = 0 for this hold
15. The item at Lib-B should now fill this hold at Lib-C checkin
(non-local patron, no exclusivity protection)
16. Set LocalHoldsExclusivityPeriod to 0 — verify non-local items can
fill holds normally regardless of hold_fill_targets flag
17. prove t/db_dependent/Holds/LocalHoldsPriority.t — passes
18. prove t/db_dependent/HoldsQueue.t — passes
19. prove t/db_dependent/Reserves.t — passes
Signed-off-by: Trevor Diamond <trevor.diamond@mainlib.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Covers the new LocalHoldsPriorityExclusivityPeriod behaviour and keeps
existing tests working with the schema change:
- t/db_dependent/Holds/LocalHoldsPriority.t: new subtest exercising
all four cases -- pref disabled, pref active on a fresh hold, pref
active past the window, and pref active with no hold_fill_targets
row (queue hasn't run).
- t/db_dependent/HoldsQueue.t: assert that after CreateQueue() with a
local-group match, both tmp_holdsqueue and hold_fill_targets rows
carry local_holdgroup_match = 1.
- t/db_dependent/Reserves.t, t/db_dependent/Koha/Hold.t: qualify the
two direct "INSERT INTO hold_fill_targets VALUES (...)" statements
with explicit column names so they continue to work now that
hold_fill_targets has one extra column.
Test plan:
prove t/db_dependent/Holds/LocalHoldsPriority.t \
t/db_dependent/HoldsQueue.t \
t/db_dependent/Reserves.t \
t/db_dependent/Koha/Hold.t
All tests should pass.
Sponsored-by: Main Library Alliance <https://www.mainlib.org/>
Signed-off-by: Trevor Diamond <trevor.diamond@mainlib.org>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
t/db_dependent/Log.t
- Add subtest verifying cronlogaction() prefixes loginfo with \$0
(the running script path), appends the info parameter, defaults the
action to 'Run', and respects custom action values.
t/Koha/Script.t
- Assert that \$Koha::Script::_cron is false when the module is loaded
without the -cron flag, confirming INIT/END blocks will not log.
t/Koha/Script_cron.t
- Assert that \$Koha::Script::_cron is true when the module is loaded
with the -cron flag, confirming INIT/END blocks will log.
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
This patch implements the library_limit concept for Report/Reports. Staff with the correct permission and superlibrarians can set libaries to which the report should be visible, if the system preference is enabled.
Test plan:
Confirm Initial Settings
1. Log in to Koha staff with username: koha and password: koha and set your library as Centerville if it's not already.
2. Visit the Saved reports page, and create and save three reports: Report 1, Report 2, and Report 3. The SQL query for each report doesn't matter, it should just be something valid.
3. Visit the Patrons page and create two staff users: StaffReportLimitUser and StaffNoLimitPermission. Set their home libraries to Centerville and Fairfield, respectively.
4. Under Allow access to the reports module, ensure both users have report management permissions for Create, Delete, Execute.
Verify Patch Settings
5. Apply the patch and restart services. If using Koha Testing Docker (KTD): from the Koha repo directory, apply the patch (e.g. `git bz apply 16631`, or check out the patched branch with `git fetch && git checkout <branch>`), then run `ktd --shell` to enter the KTD shell and run `restart_all`. If the patched branch is already checked out, just enter the KTD shell and run `restart_all`.
6. Click Patrons in the top navbar to go to the Patrons page.
7. Search for "StaffReportLimitUser" in the patron search field.
8. Click StaffReportLimitUser to edit the patron record for this user.
9. Click More -> Set permissions to access the permissions screens for the StaffReportLimitUser.
10. In the row for Allow access to the reports module (reports), click Show details to view relevant permissions.
11. Confirm Manage Report Limits (manage_report_limits) is an option under Allow access to the reports module (reports).
12. Click the checkbox for Manage Report Limits to give this user this permission, and click Save.
13. Go to the System Preferences page (Click the More dropdown from the top nav bar, click Administration, click System Preferences).
14. Verify Reports is an option in the left side navigation bar, and that when clicked, you see the LimitReportsByBranch option.
15. Enable LimitReportsByBranch as described in step 14.
16. Go back to Saved reports as in Step 2.
17. Click Edit in the row menu for Report 1.
18. At the bottom of the Report 1 edit page, you should see a Library limitation select box.
Verify the Library Limitation Settings are Persisted
19. Choose Centerville from the select box and click Update SQL.
20. Reload the page, or navigate away from the page and go back to the edit screen for Report 1.
21. Verify that Centerville is still in the Library limitation select box.
22. Add Fairfield in the Library limitation select box and click Update and run SQL.
23. Go back to editing Report 1.
24. Confirm the Library limitation select box has both Centerville and Fairfield selected.
25. Duplicate Report 1.
26. Confirm the Library limitation select box has both Centerville and Fairfield selected.
27. Click Save Report.
28. Confirm the Library limitation select box has both Centerville and Fairfield selected.
Verify Reports are Limited Based on Library Limitations
29. Edit Report 1 to add Centerville in the Library limitation select box as the only choice and click Update SQL.
30. Edit Report 2 to add Fairfield in the Library limitation select box as the only choice and click Update SQL.
31. Report 3 should have no library limitations, so it should be visible to all libraries.
32. Go to Saved reports as in Step 2.
33. Confirm you do not see Report 2, and only Report 1 and Report 3.
34. In the upper right corner, click the drop down that says koha Centerville and click Set library to change your library.
35. Set your library to Fairfield.
36. Go to Saved reports as in Step 2.
37. Confirm you do not see Report 1, and only Report 2 and Report 3.
Verify Disabling LimitReportsByBranch Preference Shows All Reports
38. Disable LimitReportsByBranch preference as in steps 13 to 15.
39. Go to Saved reports as in Step 2.
40. Confirm Report 1, Report 2, Report 3 are visible.
41. Click to edit Report 1 and verify there is no Library limitation select box.
Verify Library limitations are Persisted When EnableFilteringReports is Toggled On or Off
42. Enable LimitReportsByBranch preference as in steps 13 to 15.
43. Ensure your library is set to Fairfield as in step 35.
44. Go to Saved reports as in Step 2.
45. Confirm Report 1 is not visible, and Report 2 and Report 3 are visible.
46. Click to edit Report 2 and verify the Library limitation select box is at the bottom of the page and that it has Fairfield selected.
Verify the Show all reports option shows all reports for users with the Manage Report Limits permission and superlibrarians
47. Go to Saved reports as in Step 2 and in the Show reports dropdown, select Show all reports, and click Apply filter.
48. Confirm the page reloads with all reports visible: Report 1, Report 2, and Report 3 should be visible.
49. Select Show my branch reports from the dropdown and click Apply filter.
50. Confirm Report 1 is not visible, and Report 2 and Report 3 are visible (you should only see reports for your branch, Fairfield).
Verify Report filters return the correct results
51. Go to Saved reports as in Step 2.
52. Verify you're viewing only your branch's reports in the filter.
53. Under Filter in the left sidebar, type "2" in the Keyword field and click Apply filter.
54. Confirm only Report 2 is visible.
55. Under Filter in the left sidebar, type "1" in the Keyword field and click Apply filter.
56. Confirm there are no reports visible, and the message "No saved reports match your criteria" is displayed.
Verify Library Limits column is available to superlibrarians and users with the Manage Report Limits permission
57. Go to Saved reports as in Step 2.
58. Click the Columns dropdown and ensure that the Library limits column is available to select for display. Select it.
59. Confirm that the Library limits column is now visible in the list of reports and that it shows the correct library limitations for each report (e.g., Report 1 should show 1 library limitation and Centerville (CPL) on hover, Report 2 should show 1 library limitation and Fairfield (FFL) on hover, and Report 3 should show no limitation).
Verify the Show all reports and Limit reports options are available to users that are not superlibrarian but have the Manage Report Limits permission
60. For the StaffReportLimitUser, add the permission for Manage Report Limits as described in steps 9 to 12. This will allow this user to see the Show all reports and Limit reports options.
61. Log out of the koha admin account. In the upper right corner, click the drop down that says koha Fairfield and click Log out.
62. Log in as StaffReportLimitUser, the previously created staff user with report permissions.
63. Confirm the staff user's library is Centerville in the upper right corner of the screen.
64. Go to Saved reports as in Step 2.
65. Confirm the filter option for Show reports is visible.
66. Click to edit Report 1.
67. Verify there is a Library limitation select box and that it can be interacted with.
Verify the Show all reports and Limit reports options are not available to users that are not superlibrarian and do not have the Manage Report Limits permission
68. Log out of the StaffReportLimitUser account.
69. Log in as StaffNoLimitPermission, the previously created staff user without the Manage Report Limits permission.
70. Confirm the staff user's library is Fairfield in the upper right corner of the screen.
71. Go to Saved reports as in Step 2.
72. Click the Columns dropdown and ensure that the Library limits column is not in the list of available columns to select for display.
73. Confirm the filter option for Show reports is not available.
74. Click to edit Report 2.
75. Verify there is no Library limitation select box at the bottom of the page.
[NEW] Verify library limits are auto-assigned and preserved correctly for non-permissioned users
76. Still logged in as StaffNoLimitPermission (Fairfield) from the previous section. Go to Saved reports and click New report. Enter "Limit Auto-Assign Test" as the report name and SELECT 1 as the SQL. Confirm there is no Library limitations select box visible on the form. Click Save.
77. Go back to Saved reports. Click Edit for Report 2 (limited to Fairfield). Add `-- test` as a trailing SQL comment and click Update SQL. This simulates a non-permissioned user saving an existing report that already has limits.
78. Go back to Saved reports. Click Edit for Report 3 (no library limits). Add `-- test` as a trailing SQL comment and click Update SQL. This simulates a non-permissioned user saving an existing report that has no limits.
79. Log out and log back in as StaffReportLimitUser (Centerville, has manage_report_limits).
80. Go to Saved reports. Use Show all reports if needed. Find "Limit Auto-Assign Test" and click Edit. Confirm the Library limitations select box shows Fairfield as the only selected library — the report was automatically limited to the creator's home library on create.
81. Find Report 2 and click Edit. Confirm the Library limitations select box still shows only Fairfield — the existing limit was preserved when the non-permissioned user saved the report.
82. Find Report 3 and click Edit. Confirm the Library limitations select box is empty — no limits were introduced when the non-permissioned user saved a report that had no existing limits.
Verify limited reports are not visible to users without permissions
83. While still logged in as StaffReportLimitUser from step 79, copy the URL for the edit page of Report 1.
84. Log out of the StaffReportLimitUser account.
85. Log in as StaffNoLimitPermission, the user without the Manage Report Limits permission.
86. Paste the copied URL for the edit page of Report 1 into the browser and navigate to it.
87. Confirm that you see an error message that says "This report is restricted and cannot be accessed by the current user." and that the page does not load the report details.
Run Unit Tests
88. Using Koha Testing Docker, in the Koha home directory run the command: ktd --shell to start the ktd shell.
89. Run command: reset_all to drop and recreate the koha database.
90. Run command prove t/db_dependent/Koha/Reports.t to run the test.
91. This file contains the unit tests for the reports_branches table and the new subtest `library limits enforcement on save and update`, which covers auto-assignment of branch limits on create and preservation of limits on update for non-permissioned users.
92. All tests should pass.
93. Run command prove t/Koha/Auth/Permissions.t
94. Permissions unit tests in this file were updated.
95. All tests should pass.
96. Run command prove t/db_dependent/Reports/Guided.t
97. The Guided report tests were updated to account for the new library limitations and permissions.
98. All tests should pass.
Signoff
99. Sign off if the steps above check out.
Sponsored-by: Ramapo Catskill Library System
Co-authored-by: Casey Conlin <casey.conlin@gmail.com>
Signed-off-by: Emmi Takkinen <emmi.takkinen@koha-suomi.fi>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
Tomás Cohen Arazi
Pedro Amorim
Kyle M Hall
Martin Renvoize
Matt Blenkinsop
Aleisha Amohia
Shi Yao Wang
Lari Taskula
Hammat Wele
Laura_Escamilla
Jonathan Druart
Baptiste Wojtkowski
Casey Conlin
Cameron E. Tidd
