Commit Graph

112 Commits

Author SHA1 Message Date
Tomás Cohen Arazi 713c5c3343
Bug 41950: (follow-up) Add DBIC relationship aliases for +count embeds
Adds DBIC relationship aliases so that +count embeds advertised in
the API specs are backed by actual database relationships, making
them sortable via the SQL COUNT subquery infrastructure.

New aliases:
- Borrower: checkouts (issues)
- Aqbookseller: baskets (aqbaskets), invoices (aqinvoices)
- Ticket: updates (ticket_updates)
- ErmEholdingsPackage: resources (erm_eholdings_resources)
- Biblio: holds (reserves)

Koha methods are updated to use the new relationship names directly,
following the one-liner style: Koha::Foo->_new_from_dbic( scalar ... )

Note: checkouts on Borrower and subscriptions on Aqbookseller already
had matching DBIC relationships. comments on Illrequest, requests on
IllBatch, and items on Biblio also already existed.

Test plan:
1. Apply patch
2. Run:
   $ ktd --shell
   k$ prove t/Koha/REST/Plugin/Query.t \
       t/db_dependent/api/v1/patrons.t \
       t/db_dependent/api/v1/acquisitions_vendors.t \
       t/db_dependent/Koha/REST/Plugin/Objects.t \
       t/db_dependent/Koha/Biblio.t
=> SUCCESS: All tests pass
3. Verify sorting works:
   GET /api/v1/patrons
   x-koha-embed: checkouts+count
   _order_by: -me.checkouts_count
=> SUCCESS: Patrons sorted by checkout count
4. Sign off :-D

Assisted-by: Sonnet 4.6 (Anthropic)
Signed-off-by: Pedro Amorim <pedro.amorim@openfifth.co.uk>
2026-06-29 16:19:49 +00:00
Matt Blenkinsop 080fccc0c5
Bug 39438: Add additional fields support to ERM agreement periods
This patch enables additional fields functionality for agreement periods
in the ERM (Electronic Resource Management) module, allowing libraries
to define custom metadata fields for agreement period records.

Changes include:
- Update Koha::AdditionalField to support erm_agreement_periods table
- Modify API definitions to include extended_attributes in responses
- Add erm_agreement_periods option to additional fields admin interface
- Enhance Vue components to display and manage additional field values
- Update ShowElement component to handle period-specific attributes

Test plan:
1. Navigate to Administration > Additional fields
2. Create new fields for "Agreement periods"
3. Edit an ERM agreement and add/edit period information
4. Verify additional fields appear and can be populated
5. Confirm values are saved and displayed correctly

Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2026-01-30 10:54:10 -07:00
Matt Blenkinsop 806ac743f0
Bug 40192: Add additional fields support to ERM titles
This patch extends the additional fields framework to ERM e-holdings
titles, allowing libraries to configure custom metadata fields for
managing electronic resources.

Implementation details:
- Extended Koha::ERM::EHoldings::Title and ::Titles with
  AdditionalFields mixin for object-level support
- Added 'erm_titles' table mapping to AdditionalField.pm for API
  serialization
- Updated OpenAPI schema definitions to include extended_attributes
  in erm_eholdings_title.yaml and expose extended_attribute_types
  endpoint
- Enhanced REST API controller to handle additional fields in title
  creation/updates
- Updated admin interface to allow configuring additional fields for
  the 'title' table name
- Modified Vue.js EHoldingsLocalTitleResource component to display
  and manage custom fields

This enables institutions to track custom metadata (e.g., license
details, administrative notes, local identifiers) alongside standard
ERM title information.

Test plan:
1. Configure additional fields:
   - Go to Administration > Additional fields
   - Click "New field" and select table name "title"
   - Create a text field: Name="License notes", Searchable=Yes
   - Create an authorized value field: Name="Access level",
     AV category=LOC (or any available category)
   - Save both fields

2. Test with local ERM titles:
   - Navigate to E-resource management > Local titles
   - Create a new local title or edit an existing one
   - Verify the "Additional fields" section appears with your
     configured fields
   - Fill in "License notes" with test text
   - Select a value for "Access level"
   - Save the title

3. Verify persistence:
   - Reopen the title you just edited
   - Confirm both additional field values are displayed correctly
   - Edit one field value and save
   - Confirm the update persists

Signed-off-by: Michaela <michaela.sieber@kit.edu>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2026-01-22 09:25:57 -07:00
Tomás Cohen Arazi 6b478e0da9
Bug 40163: (follow-up) Fix wrong GNU links and update test
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2025-09-24 09:25:04 -06:00
Tomas Cohen Arazi 1873d97c07
Bug 40163: Fix several http addresses in the code docs
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2025-09-23 16:21:29 -06:00
Lucas Gass 99fd185b41
Bug 36831: (follow-up): Add formula injection protection
Without patch:

prove /kohadevbox/koha/xt/author/Text_CSV_Various.t FAIL

With patch:

prove /kohadevbox/koha/xt/author/Text_CSV_Various.t PASS

Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2025-09-19 08:42:35 -06:00
Matt Blenkinsop a869b7297f
Bug 36942: Use an exception for a failed KBART file parse
Test plan:
1) prove t/db_dependent/Koha/BackgroundJob/ImportKBARTFile.t

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2025-09-18 14:31:03 -06:00
Matt Blenkinsop 41e198bfcd
Bug 36831: Allow for multiple file types
This patch allows the KBART import tool to accept different filetypes and then work out whether they are CSV or TSV files.

Test plan:
1) Try to import the file attached to the bug using the KBART import tool (E-resource management > eHoldings > Local > Titles > Import from a KBART file)
2) It should return an error saying that the file must be .csv or .tsv
3) Apply patch
4) yarn build
5) restart_all
6) Repeat step 1
7) The file should now successfully import

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2025-09-18 14:31:02 -06:00
Matt Blenkinsop e4a19c945e
Bug 36831: Move logic from the background job to the Title object class
This patch moves logic out of the background job file and into the Koha::ERM::EHoldings::Title class where it should belong. Methods are also renamed to differentiate between object methods and helper methods

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2025-09-18 14:31:00 -06:00
Matt Blenkinsop fb2c66189f
Bug 36831: Add a method to determine the file delimiter
This patch adds a method that will detect the delimiter and quote character for a file

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2025-09-18 14:31:00 -06:00
Pedro Amorim 9482590909
Bug 39345: (QA follow-up): Add exception handling for unsupported counter release
To test this on the UI, you can add the following line of code at the beginning on 'new' in SushiCounter.pm:
->{response}->{Report_Header}->{Release} = 6;

This will force the release to be '6'. Then you can perform a harvest using the UI. The background job will fail with the error message saying that counter release 6 is not supported

prove t/db_dependent/Koha/BackgroundJobs/ErmSushiHarvester.t

Signed-off-by: Eric Phetteplace <phette23@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2025-09-05 15:10:31 -06:00
Pedro Amorim dcf28e5b8a
Bug 39345: (QA follow-up): Fix QA complaints
QA script still fails but I believe these are false positives caused by the tests files being moved

Signed-off-by: Eric Phetteplace <phette23@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2025-09-05 15:10:30 -06:00
Pedro Amorim ff82cd37f5
Bug 39345: Use data provider report release to establish service URL
Signed-off-by: Eric Phetteplace <phette23@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2025-09-05 15:10:29 -06:00
Pedro Amorim f58e2110ca
Bug 39345: Add COUNTER 5.1 support
This moves the original SUSHI COUNTER logic to a new
Koha::ERM::EUsage::COUNTER::5 class that extends
Koha::ERM::EUsage::SushiCounter.

Adds a new Koha::ERM::EUsage::COUNTER::5_1 class with COUNTER 5.1
specific logic.

Keeps common logic in Koha::ERM::EUsage::SushiCounter.

This ensures we support both COUNTER 5.0 and COUNTER 5.1, but should
hopefully help us better maintain COUNTER 5.1 going forward, without having
to worry about COUNTER 5.0, as the code is seperated.
This should also better set us up for adding support for possible future COUNTER releases.

Test plan, k-t-d:
1) Enable ERMModule
2) Add a new usage data provider at:
   <staff_url>/cgi-bin/koha/erm/eusage/usage_data_providers/add
3) Add the required credential parameters e.g. Customer ID and Requestor ID
4) Update the 'Report release' to '5.1', no single quote.
   (If you add anything other than '5', it'll default to '5.1')
5) Visit the list of data providers and click the 'Run now' button for your data provider:
   <staff_url>/cgi-bin/koha/erm/eusage/usage_data_providers
6) Click the background job link at the top. Verify the background job runs as expected.
7) After the successful harvest, run a custom report at, ensure the data is all correct:
   <staff_url>/cgi-bin/koha/erm/eusage/reports
8) Visit the data provider at:
   <staff_url>/cgi-bin/koha/erm/eusage/usage_data_providers/<provider_id>
9) Click the 'Import logs' tab, download the COUNTER 5.1 counter file.
   Open the file and verify the data is correct.

Signed-off-by: Eric Phetteplace <phette23@gmail.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2025-09-05 15:10:28 -06:00
Tomas Cohen Arazi ca8ef54949
Bug 40579: Fix CSV formula injection vulnerabilities
This patch addresses CSV formula injection vulnerabilities by ensuring
all Text::CSV instances use formula protection.

Changes made:
* Add formula => 'empty' to all Text::CSV constructors in:
  - Koha::Patrons::Import
  - Koha::BackgroundJob::ImportKBARTFile
  - Koha::ERM::EUsage::CounterFile

The 'empty' formula mode causes potentially dangerous formulas (starting
with =, +, -, @) to be treated as empty strings rather than executed,
preventing CSV injection attacks while maintaining data integrity.

Security impact:
* Prevents execution of malicious formulas in CSV imports
* Protects against data exfiltration via CSV formula injection
* Maintains backward compatibility for legitimate CSV data

To test:
1. Apply the previous patch (author test)
2. Run:
   $ ktd --shell
  k$ prove xt/author/Text_CSV_Various.t
=> FAIL: Should show 3 files without formula protection
3. Apply this patch
4. Repeat 2
=> SUCCESS: All tests should now pass
5. Try importing CSV with formula content (=cmd|"/c calc",test,data)
=> SUCCESS: Formula content is rejected/emptied, not executed
6. Import normal CSV data
=> SUCCESS: Normal data imports correctly
7. Sign off :-D

Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2025-08-29 10:43:39 -06:00
Pedro Amorim 023f935360
Bug 39823: Identify error messages based on code
Even if severity does not exist. Additionally, only add severity to message if it exists.

Test plan:
1) Apply tests patches. Run tests (they should fail):
prove t/db_dependent/Koha/BackgroundJobs/ErmSushiHarvester.t
2) Apply this patch and run tests again (they should pass this time).

UI test:
1) Create a new data provider linked to Gale Cengage, service url:
https://api.siqcloud.online/counterapi/r51
2) Add report release 5, customer id: '123', requestor id: '123', report types: 'TR_J1'
3) Run a report for this data provider, from Feb 2025 to Apr 2025. Notice that before applying patches, the job report results in a red bar with no error message.
4) Apply patches, repeat 3. Notice it now results in a nice error message.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
2025-06-02 15:26:19 -06:00
Jonathan Druart 9fbff51f0a
Bug 39367: Automatic changes
Added head2 right before sub with missing POD

Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2025-04-23 09:35:14 +02:00
Koha Development Team d659526b5a
Bug 38664: Tidy the whole codebase
This commit is generated using:
  % perl misc/devel/tidy.pl
*within* ktd, to get the same version of perltidy than what will be used
by our CI (currently v20230309).

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2025-02-11 14:58:24 +01:00
Pedro Amorim 15803e4f29
Bug 36627: (QA follow-up): Rename borrowernumber -> patron_id
To test:
$ prove t/db_dependent/api/v1/erm_counter_logs.t

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-12-30 15:12:44 +01:00
Katrin Fischer 3b86455e02
Bug 36627: Fix typo "embeded"
A search turned up more of these in other files. Fixed as well.

Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-12-23 13:36:16 +01:00
Matt Blenkinsop 7b75586ead
Bug 36627: Rename embed method to patron
This patch renames the embed method from borrowernumber to patron to fit with the use of 'patron' in the API

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-12-23 13:36:14 +01:00
Matt Blenkinsop da475321a5
Bug 36627: Embed patron record for display
Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-12-23 13:36:13 +01:00
Matt Blenkinsop 062e04c9c3
Bug 36627: Record who is importing sushi data
This patch adds functionality for storing the patron (or cronjob) that triggers a harvest. Currently every counter log is stored as being imported by the cronjob

Test plan:
1) In the usage statistics module in ERM, create a data provider (the details in the provider are not important)
2) Click into the provider and click into manual upload
3) Upload the file attached to this bug
4) Once the background job is completed click into the counter logs tab in the provider. The imported by columns should show a user with a clickable link

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-12-23 13:36:13 +01:00
Magnus Enger b62edd4177
Bug 33339: (Follow up) Fix more cases of formula=empty
This patch adds formula=empty to:
- Koha::ERM::EUsage::SushiCounter::_build_COUNTER_report_file()

Updated 2024-09-26: Earlier version also removed unnecessary
use of formula=empty from two places.

Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-11-14 10:46:31 +01:00
Jake Deery 6ece137123
Bug 37577: Add additional fields for ERM packages
This bug adds support for additional fields in the ERM module's
packages, similar to how Bug 35287 added support for additional fields
in the ERM module's licences, and Bug 37576 added support for additional
fields in the ERM module's agreements.

To test (k.t.d):
a)  APPLY PATCH
b)  Run cypress --spec t/cypress/integration/AdditionalFields_spec.ts
    1)  Note how the tests pass
c)  Add new additional fields for 'erm_packages'
    1)  Go to /cgi-bin/koha/admin/additional-fields.pl?tablename=erm_packages
    2)  Create field one called 'text non-repeatable'
    3)  Create field two called 'text repeatable', check 'repeatable'
    4)  Create field three called 'av non-repeatable', pick LANG authorised
        value
    5)  Create field four called 'av repeatable', pick LOC authorised value,
        check repeatable
d)  Create new local package
    1)  Go to /cgi-bin/koha/erm/eholdings/local/packages/add, notice the new
        additional fields section
    2)  Complete all mandatory fields for a new package
    3)  Experiment with the four different additional field types – it
        may be useful to watch the Cypress-generated test video for
        guidance
    4)  Select some AV options from the AV fields, deselect them, notice
        the repeatable one allows for multiple selection, the non-repeatable
        one does not
e)  Save the package, then view it
    1)  To view the package, click its name in the table list on
        /cgi-bin/koha/erm/eholdings/local/packages
    2)  notice the additional fields on display
    3)  also notice that the authorised values are in their human-readable
        form, and not their coded form
f)  Edit the package, then view it again
    1)  Go to /cgi-bin/koha/erm/eholdings/local/packages/edit/1
    2)  Play around again with the fields. Save.
    3)  Repeat step e, have the edits shown up?
g)  Test searchability
    1)  Go to /cgi-bin/koha/admin/additional-fields.pl?tablename=erm_packages
    2)  Edit one of the fields and make sure searchable is checked (ensure
        you choose a field that has some data in it)
    3)  Go to /cgi-bin/koha/erm/eholdings/local/packages
    4)  Notice that the searchable field(s) are now columns in the table
    5)  Notice that searchable AV fields contain a drop-down
    6)  Notice that searchable text fields have an input field
    7)  Perform a search on the additional field
    8)  Notice that the searching is working as expected
h) SIGN OFF

Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-11-08 16:09:19 +01:00
Jake Deery f348afbae8
Bug 37576: Add additional fields for ERM agreements
This bug adds support for additional fields in the ERM module's
agreements, similar to how Bug 35287 added support for additional fields
in the ERM module's licences.

To test (k.t.d):
a)  APPLY PATCH
b)  Run cypress --spec t/cypress/integration/AdditionalFields_spec.ts
    1)  Note how the tests pass
c)  Add new additional fields for 'erm_agreements'
    1)  Go to /cgi-bin/koha/admin/additional-fields.pl?tablename=erm_agreements
    2)  Create field one called 'text non-repeatable'
    3)  Create field two called 'text repeatable', check 'repeatable'
    4)  Create field three called 'av non-repeatable', pick LANG authorised
        value
    5)  Create field four called 'av repeatable', pick LOC authorised value,
        check repeatable
d)  Create new agreement
    1)  Go to /cgi-bin/koha/erm/agreements/add, notice the new additional
        fields section
    2)  Complete all mandatory fields for a new licence
    3)  Experiment with the four different additional field types – it
        may be useful to watch the Cypress-generated test video for
        guidance
    4)  Select some AV options from the AV fields, deselect them, notice
        the repeatable one allows for multiple selection, the non-repeatable
        one does not
e)  Save the agreement, then view it
    1)  To view the licence, click its name in the table list on
        /cgi-bin/koha/erm/agreements
    2)  notice the additional fields on display
    3)  also notice that the authorised values are in their human-readable
        form, and not their coded form
f)  Edit the agreement, then view it again
    1)  Go to /cgi-bin/koha/erm/agreements/edit/1
    2)  Play around again with the fields. Save.
    3)  Repeat step e, have the edits shown up?
g)  Test searchability
    1)  Go to /cgi-bin/koha/admin/additional-fields.pl?tablename=erm_agreements
    2)  Edit one of the fields and make sure searchable is checked (ensure
        you choose a field that has some data in it)
    3)  Go to /cgi-bin/koha/erm/agreements
    4)  Notice that the searchable field(s) are now columns in the table
    5)  Notice that searchable AV fields contain a drop-down
    6)  Notice that searchable text fields have an input field
    7)  Perform a search on the additional field
    8)  Notice that the searching is working as expected
h) SIGN OFF

Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-11-08 16:09:17 +01:00
Pedro Amorim f1f33fce68
Bug 35287: (Follow-up 37389): ExtendedAttributes
Apply ExtendedAttributes Mixin to AdditionalFields

To test the ExtendedAttributes mixin search is working:
1) Create 2 or more searchable licenses additional fields
2) Create a new license and enter values in these searchable fields
3) Go to the licenses list, use the column filters to search on more
than one column, verify that the search work as intended (AND operator)

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-10-24 13:38:21 +02:00
Pedro Amorim 6037bd78ae
Bug 35287: Preparation: AdditionalFields License Mixin
Koha::Object::Mixin::AdditionalFields needs to come before Koha::Object
in the inheritance so that the strings_map method that is called is
the one from Mixin and not from Koha::Object.
This is only required for Koha classes implementing the AdditionalFields Mixin and expose data through the REST API, because strings_map is implement in both the aforementioned classes

Signed-off-by: Edith Speller <edith.speller@ukhsa.gov.uk>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-10-24 13:38:18 +02:00
Jonathan Druart 61b3aa2aa2
Bug 38177: Do not use stash to retrieve parameters
The pagination for EBSCO's titles, packages and resources is broken. We
shouldn't rely on $c->stash but use params from the request instead.

Test plan:
1) Setup all the system preferences in ERM:
<staff_url>/cgi-bin/koha/admin/preferences.pl?tab=erm
2) Enable EBSCO in ERMProviders. Set the ERMProviderEbscoApiKey and ERMProviderEbscoCustomerID prefs.
3) Do a search that returns more than 20 records. Click on a page other than 1. Notice the records do not change (Page is the same).
4) Apply patch. Repeat. Notice pagination now works.

Search titles, packages and resources
Use the pagination and modify the number of results per page

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-10-23 19:14:51 +02:00
Pedro Amorim fa6cd2f9b7
Bug 37810: Consider ServiceActive on status response
Some SUSHI providers return ServiceActive instead of the documented Service_Active:
https://countermetrics.stoplight.io/docs/counter-sushi-api/f0dd30f814944-server-status

This ensures the test connection does not fail regardless of what is used

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
2024-10-02 10:17:18 +01:00
Matt Blenkinsop a1f0fc04d1
Bug 37526: Handle redirects in SUSHI requests
Test plan:
1) Apply test patch only. Run:
prove t/db_dependent/Koha/BackgroundJob/ErmSushiHarvester.t
2) Verify tests fail.
3) Apply this patch. Run tests again. Verify they pass.

PA amended: Added test plan

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-09-27 18:00:42 +02:00
Nick Clemens b7600e4801
Bug 37856: (QA follow-up) Tidy
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-09-27 18:00:40 +02:00
Pedro Amorim 72b38a1dbf
Bug 37856: Consider absence of 'Report_Items'
The SUSHI provider in question for this patchset also does not return any
Report_Items entry. This patch accounts for that.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-09-27 18:00:38 +02:00
Pedro Amorim 7c175c2160
Bug 37856: Consider 'Exceptions' not being part of 'Report_Header'
When fixing the issue around the missing platform parameter, we
noticed this particular provider also returns 'Exceptions' as
its own thing, not part of 'Report_Header', despite documentation:
https://countermetrics.stoplight.io/docs/counter-sushi-api/7cp0h7n1hm2en-tr-j1-journal-requests-controlled

From the link above, search for 'Exceptions'. Notice it should be under 'Report_Header'.
Either way, this patch accounts for this possibility.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-09-27 18:00:38 +02:00
Pedro Amorim dc9a5684f4
Bug 37856: Append platform parameter to SUSHI url if defined
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-09-27 18:00:36 +02:00
Nick Clemens 5b51ef78d4
Bug 37963: (QA follow-up) Tidy
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-09-27 18:00:35 +02:00
Pedro Amorim 23ab84c9b6
Bug 37963: Case 2 - Consider providers returning error messages lower cased
Test plan:
1) Apply only test patch for case 1. Run tests:
prove t/db_dependent/Koha/BackgroundJob/ErmSushiHarvester.t
2) Verify tests fail. Apply the other patch for case 1.
3) Run tests again. Verify they now pass.
4) Repeat the test plan but using patches for case 2

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-09-27 18:00:35 +02:00
Pedro Amorim 5d53bb4bff
Bug 37963: Case 1 - Return if sushi_errors even if response code >= 400
This ensures that SUSHI errors, if present, are shown on the UI even if
response->code >= 400.
This means that if, for example, SUSHI provider returns 401 with SUSHI
error like so:
{"Code": 2020, "Severity": "Error", "Message": "API Key Invalid"}

Then, the 'API Key Invalid' will be shown to the user, without having to
check logs.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-09-27 18:00:34 +02:00
Kyle M Hall 5a06bd0d73
Bug 37308: (QA follow-up) Tidy code
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-08-16 17:33:44 +02:00
Pedro Amorim 8175cccca2
Bug 37308: Add user-agent for SUSHI outgoing requests
Signed-off-by: Belal Ahmadi <belal.ahmadi@uwl.ac.uk>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-08-16 17:33:43 +02:00
Matt Blenkinsop d2820f57d6
Bug 34788: (QA follow-up)
This patch fixes a few things:
1) The blue dialog box now clears when navigating away from the page
2) The background job now uses skip_record_index to avoid queuing indexing jobs for every new biblio and instead queues one job at the end
3) Large files that get chunked now successfully create linked biblios if requested
4) Title matching rules have been expanded to check the package ID so that we can have duplicate titles in different packages
5) A link to the package is now included on the job report page

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-05-10 15:19:09 +02:00
Matt Blenkinsop 08992ab711
Bug 36618: Make biblio creation optional for ERM local titles
This patch makes biblio creation on local title creation optional.

Test plan:
1) Navigate to the local title creation form
2) Create a title and leave the "Create record" box unchecked
3) Submit the form
4) Search the catalogue for the title you have just created - no biblio result will be available
5) Repeat steps 2 - 3 but this time check the box to create a record
6) You should now be able to find a record in the catalogue for the title you just imported

Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
2024-05-08 20:04:07 +02:00
Pedro Amorim 9988925fb5
Bug 35418: Fix _get_usage_months
No longer falls in an endless loop if the last month in the report period is december.

Test plan:
1) Apply just the tests patch, run EUsage tests
prove t/db_dependent/Koha/ERM/EUsage/
2) Wait for a few seconds. Notice SushiCounter.t it hangs. CTRL+C (cancel it)
3) Apply the fix patch.
4) Run the tests again. Notice all EUsage tests pass with no issues.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Andrew Auld <andrew.auld@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-29 17:36:11 -03:00
Pedro Amorim 55f024c0e3
Bug 35115: (QA follow-up):
Spelling.
Tidyness.
Removal of leftover warn

Nick's patch fixes the issue as described, and I agree that it is the ideal solution here.
prove t/db_dependent/api/v1/erm_eholdings* passes
prove t/db_dependent/Koha/BackgroundJob/CreateEHoldingsFromBiblios.t passes

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-29 17:36:10 -03:00
Nick Clemens f984969690
Bug 35115: Move store routine from Resource to Title and update code
These patches move the store routine from Koha::ERM::EHoldings::Resource to Koha::ERM::EHoldings::Title as the code deals exclusively with title fields.

It updates the code to ensure that records are created when a title is not attched to a biblio, and that only the biblio title field is updated when updating an eholdings title.

To test:
 1 - Enable ERMModule sys pref
 2 - Create a new public list, visit:
     /cgi-bin/koha/virtualshelves/shelves.pl
 3 - Click "New list" enter name, set public -> public
 4 - Click "Add items", enter 112\n113\n114 (new line for each), in "Biblio numbers"
 5 - Notice that all added biblios have quite a few MARC fields
 6 - Go to packages, visit:
     /cgi-bin/koha/erm/eholdings/local/packages
 7 - Create a new package, add a name and hit 'Submit'
 8 - Go to titles, visit:
     /cgi-bin/koha/erm/eholdings/local/titles
 9 - Click "import from list"
10 - Pick the package created in 7)
11 - On the row of the list created in 2), click "Import"
12 - Go back to the list, visit:
     /cgi-bin/koha/virtualshelves/shelves.pl?op=view&shelfnumber=1
13 - Notice all the biblios have been stripped of their MARC data, and only title remains.
14 - Go to details page for one of the bibs
15 - Edit -> Replace record via Z39.50
16 - You can search for anything, just make sure to import a mostly full record
17 - Go back to ERM - eHoldings - Local - Titles
18 - Edit the title for the record that was replaced
19 - Save
20 - Go to record details - note the record is gone again
21 - Apply patches
22 - Search the catalog
23 - Check some titles
24 - Add to a new list
25 - repeat 8 - 11 with the new list
26 - View and confirm records are intact
27 - Edit the eholdings title for one of the records, changing the title field
28 - Save and view record details
29 - Confirm record is intact and title updated

Signed-off-by: Pedro Amorim <pedro.amorim@ptfs-europe.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-11-29 17:36:09 -03:00
Nick Clemens 17cf580f19
Bug 34587: (QA follow-up) Tidy
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:32 -03:00
Pedro Amorim a35403cb84
Bug 34587: Add IR_A1 fields to COUNTER report
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:31 -03:00
Pedro Amorim c81112e798
Bug 34587: (QA follow-up): squash
Tidy
$c->validation in API controllers
File permissions
.inc file filters

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:30 -03:00
Pedro Amorim f7e2bc5d9b
Bug 34587: Move YOP to mus/yus instead of erm_usage_titles
Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:29 -03:00
Pedro Amorim ee4db6d763
Bug 34587: Improve title uniqueness rules
Improve robustness of unique matching here, to make sure we always match for same title if the case,

Some report rows may correspond to the same title as the previous row but have an empty match field, or alternatively come with a filled match field (e.g. DOI or Print_ISSN in TR_J4. Because of this we only verify a uniqueness match field if both current row and previous have it non-empty, otherwise we keep checking the remaining uniqueness match fields.

Example of this use-case, COUNTER report:
title    | publisher | platform  | Proprietary_ID | Print_ISSN | DOI     | YOP  | usages
examplet | examplep  | examplepl | 1              | 123        |        | 2020 | usages
examplet | examplep  | examplepl | 1              | 123        |someDOI | 2021 | usages

The above 2 rows is the same title, same publisher, same proprietary_id,
same Print_ISSN, etc. It just so happens that one was returned by SUSHI
with DOI and the other wasnt.
These 2 rows correspond to different usage statistics by YOP, not 2
different usage_titles

Signed-off-by: Jessica Zairo <jzairo@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2023-10-31 16:46:27 -03:00