The way the old `objects.search` was build implied several totals were
calculated there, and passed to the `$c->add_pagination_headers` helper.
With the introduction of `objects.search_rs` and the ability of doing
things to the resultset afterwards, it felt like out of place to have
the pagination headers be implicitly set inside `objects.search_rs`.
This patch makes the search_rs stash some required values (from the original request) and
makes `add_pagination_headers` accept those values. This way other
callers can still build their own pagination values, while allowing this
simplified implementation.
Full-stack tests still pass, and the helper tests are moved to the
`db_dependent` section and are now more meaningful as well
To test:
1. Apply this patch
2. Run:
$ ktd --shell
$ prove t/db_dependent/Koha/REST/Plugin/Pagination.t \
t/db_dependent/api/v1/
=> SUCCESS: Tests pass! Nothing broken!
3. Sign off :-D
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch introduces the mentioned helpers, and reimplements
objects.search and objects.find in terms of them.
To test:
1. Apply patch
2. restart_all
3. Check that any API that uses objects.search helper is still running
(GET /api/v1/items, GET /api/v1/holds, etc)
4. prove t/db_dependent/Koha/REST/Plugin/Objects.t
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch updates the `av_expand` occurences to `strings` to match the
change of the return structure of `_strings`.
We replace `+av_expand` in the headers with `+strings`, the expected
object method name from `api_av_mapping` to `api_strings_mapping` and
the internal hash key from `av_expand` to `strings`.
Test plan
1) Run the included unit tests.. all should still pass.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Instead of a global av-expand flag (through a header) this patch
proposes to allow specifying +av_expand at the x-koha-embed header
level. This allows a more fine-grained control on what objects get avs
expanded. e.g.:
GET /patrons/123
x-koha-expand: +av_expand,checkouts.item+av_expand
=>
{
"_str": {
"city": { "str": "Córdoba", ... }
},
...
"checkouts": [
{
...,
"item": {
"_str": {
"not_for_loan_status": { "str": "Reference material", ... },
...
},
...
}
}
]
}
To test:
1. Run:
$ kshell
k$ prove t/db_dependent/Koha/Object.t \
t/Koha/REST/Plugin/Query.t \
t/db_dependent/Koha/REST/Plugin/Objects.t
=> SUCCESS: Tests pass!
2. Sign off :-D
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch adds the posibility to expand authorised values when to_api
method is called.
The classes where authorised values should expand must implememnt the
_fetch_authorised_values method, and must return a hash like the
following
{
column_name => Koha::AuthorisedValue->unblessed
...
}
This patch will be used in bug 8179, so please test there.
Sponsored-by: Virginia Polytechnic Institute and State University
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
This patch adds 'x-koha-request-id' to the reserved param names list,
and makes the objects.search helper read it and send it back in the
response headers.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Our objects.search helper is written for being used in our API
controllers, which get their query parameters processed by the OpenAPI
plugin, using JSON::Validator.
Particularly, the 'q' parameter is defined as 'multi' on our spec, which
means objects.search always gets it as an arrayref.
As the Objects.t tests are not using the OpenAPI plugin, a hashref is
generated as only one q= is being passed. This patch adds an extra
validation on objects.search, for the non-arrayref use case and does the
right thing.
To test:
1. Run:
$ kshell
k$ prove t/db_dependent/Koha/REST/Plugin/Objects.t
=> FAIL: Tests fail!
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests pass!
4. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This patch changes the q_param definition so the defined query parameter
is repeatable. This way JSON::Validator will always generate an arrayref
for it and won't skip occurences.
The objects.search helper is updated to always consider the 'q'
parameter as an array, as expected.
To test:
1. Apply the regression tests patch
2. Run:
$ kshell
k$ prove t/db_dependent/api/v1/query.t
=> FAIL: Tests fail!
3. Apply this patch
4. Repeat 2
=> SUCCESS: Tests pass!
5. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
This patch makes objects.search implicitly update the passed
*$result_set* to use search_limited. This way no object leaks could
happen without noticing.
To test:
1. Apply the regression tests patch
2. Run:
$ kshell
k$ prove t/db_dependent/Koha/REST/Plugin/Objects.t
=> FAIL: Tests fail because search_limited is not used
3. Apply this patch
4. Repeat 2
=> SUCCESS: Tests pass! Results are correctly filtered based on userenv!
5. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Sponsored-by: Virginia Polytechnic Institute and State University
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Sponsored-by: Virginia Polytechnic Institute and State University
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds the objects.find helper. Much like objects.search, it
returns the to_api method with embedded.
Sponsored-by: Virginia Polytechnic Institute and State University
Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch simply removes the use of path parameters on building the
query in $c->objects->search.
The sample usage in the original tests considers a specific use case in
which the 'nested' resultset has the path param in its attributes:
GET /patrons/:patron_id/holds
In this case, $c->objects->search would be passed a Koha::Holds
resultset into which it would build a query. As the patron_id param can
be mapped into a Koha::Hold attribute, this works.
We don't actually use this approach in code. What we do is searching for
the patron, and properly return a 404 if the patron doesn't exist [1]. If it
exists, we actually call $patron->holds to get the related resultset, so
there's no gain in having the path param on the query afterwards.
That said, if we wanted to build:
GET /holds/:hold_id/pickup_locations
as the latter is a calculated value, from a resultset that doesn't
actually include a (mapped to some attribute) hold_id attribute, there's
no way to use it if it will automatically add the hold_id to the
pickup_locations resultset. It will give errors about hold_id not being
an attribute of Koha::Library (Branches.pm actually).
So this patch removes the automatic use of path parameters in
$c->objects->search. We can extract them in the controller and add them
to the passed resultset if required. But this patch removes a constraint
we have for building routes controllers right now.
[1] If we didn't return the 404, and we just passed a fresh Koha::Holds
resultset to be feeded with the patron_id, we would always return an
empty array in cases where the patron doesn't exist. This would be
misleading, but I'm open to discuss it. Design-wise.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
We already set X-Total-Count to the total number of filtered rows,
but we don't have the total number of non-filtered rows.
Test plan:
This is easy to test on top of bug 27352 or bug 27251, apply them if not
pushed yet.
1. Create 40 items with public notes = "xxx" for biblionumber=4
then, using Postman (or whatever you prefer):
http://kohadev-intra.mydnsname.org:8081/api/v1/biblios/4/items?_page=1&_per_page=20&q=[{"me.public_notes"%3A{"like"%3A"%25x%25"}}]&_match=contains
Check the headers and confirm you see X-Total-Count=40 and
X-Base-Total-Count=44
2. go to /cgi-bin/koha/tools/quotes.pl
You see "Showing 1 to 20 of 28 entries"
Search "he"
Showing 1 to 20 of 22 entries (filtered from 28 total entries)
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch introduces handling for per_page=-1 and actually adds a
missing feature to the API: being able to request all resources on a
route.
To test this:
1. Visit the libraries admin page
2. On the table, choose to display 'All' rows
=> FAIL: it doesn't refresh, the browser console displays a 500 error
code and so the logs
3. Apply the tests patches
4. Run:
$ kshell
k$ prove t/Koha/REST/Plugin/Pagination.t \
t/db_dependent/Koha/REST/Plugin/Objects.t
=> FAIL: Tests fail loudly
5. Apply this patch
6. Restart plack
7. Repeat 2
=> SUCCESS: choosing to display all, works
8. Repeat 4
=> SUCCESS: Tests pass now!
9. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch makes the helper paginate the resultsif no pagination
parameters are passed.
Page number defaults to 1, and the page size to the RESTdefaultPageSize
syspref value.
To test:
1. Apply the regression tests patch
2. Run:
$ kshell
k$ prove t/db_dependent/Koha/REST/Plugin/Objects.t
=> FAIL: Pagination is not enforced by default
3. Apply this patch
4. Repeat 2
=> SUCCESS: Results are paginated
5. Sign off :-D
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
We must utf8 encode the string before json decoding it.
Test plan:
Use "❤" and play with the library search (bug 25288) and Postman to
generate queries using it.
This patch prevents 500:
[2020/06/16 14:11:37] [ERROR] GET /api/v1/libraries: unhandled exception (Mojo::Exception)<<Wide character in subroutine entry at /kohadevbox/koha/Koha/REST/Plugin/Objects.pm line 107.>> Koha::REST::Plugin::Exceptions::__ANON__ /kohadevbox/koha/Koha/REST/Plugin/Exceptions.pm (73)
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch adds the possibility to object.search helper, to also filter by prefetched columns.
In order to dynamically add filter parameters, they must be coded as json and placed in the body of the request, coded as string in 'q' query parameter or as string in 'x-koha-query' header.
The coded json, is in fact dbix syntax.
To test:
1. apply this patch
2. prove t/Koha/REST/Plugin/Query.t t/db_dependent/Koha/REST/Plugin/Objects.t
3. Sign off
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Bug 9978 should have fixed them all, but some were missing.
We want all the license statements part of Koha to be identical, and
using the GPLv3 statement.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch makes the Koha::Object(s) derived classes expose information
about prefetch-able relations. This is then used by a new helper to
generate the prefetch information for the DBIC query.
To test:
1. Apply this patch
2. Run:
$ kshell
k$ prove t/db_dependent/Koha/Object* \
t/db_dependent/Koha/REST/Plugin/Objects.t \
t/Koha/REST/Plugin/Query.t
=> SUCCESS: Tests pass!
3. Sign off :-D
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch separates query parameters from path parameters, and uses exact matching for the later.
To test:
1. Apply this patch
2. prove t/Koha/REST/Plugin/Query.t t/db_dependent/Koha/REST/Plugin/Objects.t
SUCCESS => tests ok
3. Sign off
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch makes the objects.search helper use the koha.embed structure
that is embedded in the authenticate_api_request step.
This way, any controller using it will benefit from automatic embed
handling.
To test:
1. Apply this patch
2. Run:
$ kshell
k$ prove t/db_dependent/Koha/REST/Plugin/Objects.t
=> SUCCESS: Tests pass!
3. Sign off :-D
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch simplifies the objects.search helper so it relies entirely on
the result set object for the attribute mappings.
The result is no more to_api or to_model mappings are passed. The
controllers need to be cleaned up after this patch.
To test:
1. Apply this patch
2. Run:
$ kshell
k$ prove t/db_dependent/Koha/REST/Plugin/Objects.t
=> SUCCESS: Tests pass!
3. Sign off :-D
Note: the original version of this helpers accepted arbitrary mappings
and are now constrianed to real mappings on the Koha::Object level. As
such, the number of tests got reduced.
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This patch makes objects.search pass the to_mode method reference so it
can be used to map API attributes to column names when building the
sorting portion of the query.
To test:
1. Apply the regression tests on this patchset
2. Run:
$ kshell
k$ prove t/db_dependent/Koha/REST/Plugin/Objects.t
=> FAIL: Tests fail!
3. Apply the rest of the patchset
4. Run:
k$ prove t/Koha/REST/Plugin/Query.t
=> SUCCESS: Tests pass! changes to the methods work as expected!
5. Run:
k$ prove t/db_dependent/Koha/REST/Plugin/Objects.t
=> SUCCESS: Tests pass!
6. Sign off :-D
Sponsored-by: ByWater Solutions
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Josef Moravec <josef.moravec@gmail.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
This causes test failures in D8 and would require packaging our own perlcritic
Reverting until future date when consequences do not outweigh benefits
This reverts commit ecc94da6b2.
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Test plan:
1) Check all Koha/REST files and confirm that 'use Modern::Perl;' has
been removed.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
This patch adds (yet) another param to objects.search: a reference
to a to_api function to be applied when processing the search results
for output.
To test:
- Apply this patch
- Run:
$ kshell
k$ prove t/db_dependent/Koha/REST/Plugin/Objects.t
=> SUCCESS: Test count raised, and tests pass!
- Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch introduces a new parameter to the objects.search Mojo helper.
The idea behind this, is that if there's any attribute name mapping
between the DB objects and the API exposed ones, we should be able to
pass it to objects.search so the filtering query params are mapped
correctly for building the DBIC query, like this example:
my $patrons_set = Koha::Patrons->new;
my @patrons = $c->objects->search( $patrons_set,
\&to_model
)->as_list;
# and probably
@patrons = map {to_api($_)} @patrons;
The to_model function needs to avoid autovivification, to prevent
messing with the query parameters (undef params).
To test:
- Apply this patches
- Run:
$ kshell
k$ prove t/db_dependent/Koha/REST/Plugin/Objects.t
=> SUCCESS: Tests pass! Params get mapped!
- Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Bourgault <david.bourgault@inlibro.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch creates the 'build_query_params' helper, instead of the
original function in Koha::Objects.
Unit tests are removed for Koha::Objects::_build_query_params_from_api and
written for the helper plugin.
The objects.search helper gets a call to build_query_params added. Tests for it
updated to match this behaviour change.
To test:
- Apply this patches
- Run:
$ kshell
k$ prove t/Koha/REST/Plugin/Query.t \
t/db_dependent/Koha/Objects.t \
t/db_dependent/Koha/REST/Plugin/Objects.t
=> SUCCESS: Tests pass!
- Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Lari Taskula <lari.taskula@jns.fi>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Lari Taskula <lari.taskula@jns.fi>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Tomas Cohen Arazi
Martin Renvoize
Agustin Moyano
Jonathan Druart
Nick Clemens
Jasmine Amohia
Julian Maurice