Building on bug 41918, it would be good to limit the number of reports
a given user can run at a time. This control is config-file based as
it is tied to the server resources provided by the administrator.
Adds a new koha-conf.xml option total_running_reports_per_user_limit
(0 = disabled). When the configured limit is reached, the user is told
"You have too many reports running at the moment. Please wait for some
of them to finish before running this report again."
Note on implementation: the original patch on bz (attachment 198034)
performed a second pre-flight Koha::Reports->running call in
reports/guided_reports.pl. The bug 41918 QA follow-ups already pushed
the duplicate-running guard into Koha::Report::prep_report so it would
fire across the staff UI, svc/report and opac/svc/report. This patch
extends that same model-layer architecture rather than reintroducing
controller-layer enforcement:
- New Koha::Exceptions::Report::TotalRunning subclass of
Koha::Exceptions::Report.
- prep_report now performs a second check after the duplicate one and
throws TotalRunning when the user has total_running_reports_per_user_limit
reports of any kind already in flight.
- All three callers (guided_reports.pl, svc/report, opac/svc/report)
catch the new exception alongside DuplicateRunning, mapping it to a
new template error key total_running_reports_limit_exceeded for the
staff UI and a JSON {error: total_running_reports_limit_exceeded}
with HTTP 429 for the JSON endpoints.
Test Plan:
1) Set total_running_reports_per_user_limit to 3 in your koha-conf.xml.
2) Create 3 long-running reports like:
SELECT COUNT(*) FROM items i1 JOIN items i2 JOIN items i3 JOIN items i4 JOIN items i5 JOIN items i6
3) Run each report in turn by opening a new tab, browsing to the
reports module, and choosing to run that report.
4) Note the first three continue to run, but a fourth attempt gives
the new "too many reports running" alert in the staff UI.
5) Repeat against /cgi-bin/koha/svc/report?id=N and confirm an HTTP
429 response with JSON {error: total_running_reports_limit_exceeded}.
6) Set total_running_reports_per_user_limit back to 0 and confirm
normal behaviour resumes.
7) prove t/db_dependent/Koha/Reports.t -> 13 tests pass, including a
new subtest for TotalRunning.
Signed-off-by: Alex Carver [Acerock7] <alex@rcls.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The CSV/tab export branch (op=export) of reports/guided_reports.pl
calls Koha::Report::prep_report just like the op=run branch and the
two svc/report endpoints, but unlike the others it had no eval/catch
around the call. The QA follow-ups that pushed enforcement into the
model layer therefore left this one path emitting a 500 (uncaught
Koha::Exceptions::Report::DuplicateRunning) when the configured
duplicate-running limit is exceeded.
Wrap the call in the same pattern as svc/report: on
DuplicateRunning return HTTP 429 Too Many Requests with a plain-text
body explaining the situation. Plain text rather than the staff
template because op=export sets file-download headers and the user is
expecting either the file or an error - we don't have a sensible way
to fall back into the report-run template from this branch.
The CLI runners (misc/cronjobs/runreport.pl, misc/export_records.pl)
still bypass prep_report and therefore the limit, which is intended:
scheduled / batch jobs should not be subject to user-quota
enforcement.
Test plan:
1) Set duplicate_running_reports_per_user_limit=1 in koha-conf.xml,
restart_all.
2) Start a long-running report from the staff UI.
3) While it is running, navigate to op=export with the same report
id (eg. via the "Export" button).
4) Before the patch: 500 with a stack trace in the logs.
After the patch: 429 with body "This report is already running.
Please wait for it to finish before running it again."
5) Set the limit back to 0 and confirm exports work normally again.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The previous patches enforced duplicate_running_reports_per_user_limit
only in reports/guided_reports.pl. Two other code paths run saved SQL
reports for an authenticated user and bypassed the guard entirely:
- svc/report (staff JSON endpoint, eg. used for chart widgets and any
caller hitting /cgi-bin/koha/svc/report?id=N directly)
- opac/svc/report (OPAC JSON endpoint)
A user who hit their staff-UI limit could still spawn additional copies
of the same long-running report via either svc endpoint, defeating the
guard.
Push the check into Koha::Report::prep_report itself, which is the one
function all three callers go through. When the limit is reached
prep_report now throws a new Koha::Exceptions::Report::DuplicateRunning
(carrying report_id, user_id and the configured limit). The guided
reports CGI catches it and re-renders the existing "report is already
running" alert; both svc/report endpoints catch it and return HTTP 429
Too Many Requests with a JSON { error: "duplicate_running_report" }
payload.
Notes:
- The pre-flight Koha::Reports->running call is removed from the
guided-reports CGI; the model now owns that policy and the
controller is just rendering. The TOCTOU window remains identical
in size and is the same intentional trade-off as before.
- Limit=0 (the default) and absent userenv both bypass the check, so
CLI cronjobs (misc/cronjobs/runreport.pl) and direct DB users are
unaffected.
Test plan:
1) prove t/db_dependent/Koha/Reports.t - now includes a new subtest
covering limit=0 (lives), under limit (lives) and at limit (throws
Koha::Exceptions::Report::DuplicateRunning).
2) Set duplicate_running_reports_per_user_limit=1 in koha-conf.xml,
restart_all.
3) Open a long-running report in two staff tabs - second tab shows
the "report is already running" alert (unchanged behaviour).
4) While one run is in flight, hit /cgi-bin/koha/svc/report?id=N as
the same user - now returns HTTP 429 with the JSON error payload
(previously: ran a second copy of the query).
5) Same check on /cgi-bin/koha/opac/svc/report?id=N.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Two small documentation improvements following on from the duplicate-
running guard:
- Koha::Report::prep_report now spells out the tracking-comment format
in its POD. Koha::Reports->running parses this comment, and external
log/audit tooling that consumes Koha SQL relied on the previous
format (/* saved_sql.id: <id> */). Calling out the new format
explicitly makes the public contract visible to anyone reading the
POD or grepping for it, and warns future contributors that changing
the wrapper will break running() too.
- reports/guided_reports.pl gets an inline comment explaining the
TOCTOU window between the count check and the actual report
execution. The window is intentional - the guard exists to stop
accidental double-clicks and two-tab submissions, not to be a
serializing lock - so worst case is one extra concurrent run, which
is acceptable and cheaper than the alternative.
No behaviour change.
Test plan:
1) prove t/db_dependent/Koha/Reports.t still passes.
2) perldoc Koha::Report -> prep_report section now mentions the
saved_sql.id / user_id wrapper format.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
- Fix POD synopsis: invalid syntax and wrong field (userid vs
borrowernumber) replaced with a correct example and parameter list.
- Build the WHERE clause with parameterised bindings via
$schema->storage->dbh, removing string interpolation of
$user_id / $report_id into LIKE patterns and the regex match.
IDs are formatted with %d so non-integer input becomes 0 and cannot
inject regex/LIKE metacharacters.
- Return a Koha::Reports resultset instead of a list of MySQL process
ids. The method name now matches what it returns and callers can use
the standard ->count / iteration interface. The single in-tree caller
in reports/guided_reports.pl is updated; the template error key is
simplified from duplicate_running_report_ids to a boolean
duplicate_running_report (the ids were never displayed).
Test plan:
1) Apply the previous Bug 41918 patches and this follow-up.
2) Set duplicate_running_reports_per_user_limit to 1 in koha-conf.xml.
3) Restart all the things.
4) Open a long-running saved report in two tabs and run it from both.
5) Verify the second tab shows "This report is already running..."
6) Set the limit to 0 and confirm both tabs run normally.
7) prove t/db_dependent/Reports.t still passes.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This commit add the option duplicate_running_reports_per_user_limit to
koha-conf.xml. If this option is empty or 0, the feature is disabled. If
this option is enabled, Koha will limit the number of times a given
report can be run of a single Koha user.
Test Plan:
1) Apply this patch
2) Set duplicate_running_reports_per_user_limit to 1, verify previous
behavior is maintained
2) Set duplicate_running_reports_per_user_limit to 0, verify the feature
is disabled
3) Set duplicate_running_reports_per_user_limit to 2, verify Koha limits
the number of duplicates of the report to the number specified in duplicate_running_reports_per_user_limit
Signed-off-by: Alex Carver [Acerock7] <alex@rcls.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Test Plan:
1) create a long running report such as:
SELECT COUNT(*) FROM items i1 JOIN items i2 JOIN items i3 JOIN items i4 JOIN items i5 JOIN items i6
2) Run this report multiple times
3) Note the processes are running via SHOW FULL PROCESSLIST from koha-mysql
4) Kill those processes
5) Apply this patch
6) Restart all the things!
7) Try to run the report in two tabs, note the second tab opened says
the report is already running!
Signed-off-by: Alex Carver [Acerock7] <alex@rcls.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch splits delete_reports into delete_own_reports and delete_all_reports
While delete_all_reports supercedes delete_own_reports it seems to make more sense than
delete_own_reports and delete_others_reports.
With delete_all_reports we can have a user who can delete all, without specific permission to delete their own
The other way we would have delete others, and you could not delete your own, which seems more odd
To test:
1 - Have 2 users - 1 with delete_reports, 1 without
2 - Apply patches, update database
3 - Confirm user with permission now has 'delete_all_reports' and other user still has no reports deletion permission
4 - Give the second user 'delete_own_reports' permission
5 - Create 4 reports as each user
6 - Confirm view/edit pages show the delete button for each user, and confirm they both work
7 - You shoudl have 2 reports left for each user
8 - Confirm each user can delete their own report from the saved SQL list:
http://localhost:8081/cgi-bin/koha/reports/guided_reports.pl?op=list
9 - There should be 1 report from each
10 - Give the second user 'delete_all_reports' permission in one tab and reload the saved reports in another tab
11 - Select both reports in the second tab
12 - Remove 'delete_all_reports' from the second user, leaving 'delete_own_reports' in the first tab
13 - Click 'Delete selected' in the reports tab
14 - Confirm you get 'You do not have permission to continue'
15 - Hit back, deselect other users report, confirm you can delete your own with 'Delete selected'
16 - Sign off
Signed-off-by: Judy Poyer <jpoyer@aacpl.net>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Test plan:
0. Apply the patch and restart_all
1. Using a user with only "create_reports", "execute_reports", or "delete_reports",
try to perform operations not covered by that permission.
That is, if you have "create_reports", try to run, export, and delete reports.
If you have "execute_reports", try to create, modify, and delete reports
If you have "delete_reports", try to create, modify, and run reports
2. Note that a user with each of these permissions can list and view SQL.
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This bug refactors the preview sql buttons to pass the permissions granted for each report to
the javascript that clones and updates the modal.
That allows us to check for the appropriate permissions when preparing the buttons for the modal
and to remove any that are not allowed.
This allows for different permissions per report for future devs
(see bugs 42406, 16631)
To test:
1. Create some reports
2. Create a user with only execute_reports permissions
3. Login as that user and go to Reports > use saved
4. In the actions column click the ^ and select 'Preview sql'
5. Notice delete button renders and works to delete
6. Apply patch and restart_all
7. Repeat step 4, now only the Run report button renders and works
8. Add permissions for create_reports
9. Repeat step for, now Edit, Duplicate, Schedule and run buttons render and work
10. Add permissions for delete_reports
11. Repeat step 4, confirm the delete button renders and works
12. Confirm the preview sql button works to preview different reports
We also fix some error in the script, several wrong things there:
* delete was not replaced with cud-delete
* missing op for the creation of new reports
* update and run requires two permssions
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Scott Barter <sbarter@aacpl.net>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This is a POC (i.e. not ready for inclusion yet)
It adds a "Open in DataTables" button in the toolbar, that will refetch
the report (with a limit=100000, that needs to be configurable, how?)
and load the results in a kohaTable (DataTables)
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Andrew Fuerste Henry <andrew@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The warnings:
[WARN] Argument "YYYY-MM-DD" isn't numeric in numeric lt (<)
at /kohadevbox/koha/reports/issues_stats.pl line 224.
appears in the plack-intranet-error.log.
It happens because a numerical comparison was used to compare the dates
instead of a string comparison.
To test:
========
1. Observe plack-intranet-error.log.
2. In staff interface go to Reports > Statistics wizards > Circulation.
3. Select date "From" and date "To" then submit.
4. Two warnings appear in plack-intranet-error.log.
5. Apply the patch; restart_all.
6. Repeat 2-3. No warnings appear in log.
Sponsored-by: Ignatianum University in Cracow
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch changes the incorrect usage of output_error
in acquisitions_stats.pl so that it uses a HTTP status
code instead of an arbitrary string.
Test plan:
0. Apply the patch and koha-plack --restart kohadev
1. Go to http://localhost:8081/cgi-bin/koha/reports/acquisitions_stats.pl
2. Use your F12 HTML inspector to find the "Line" radio buttons that you've
selected and remove the values for those elements
3. Click "Submit" at the bottom of the screen
4. Note that you get a 403 error (if bug 30803 applied - otherwise it's a 404 error)
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch makes some updates to the acquisitions, patrons, and catalog
statistics wizards in order to make more information translatable.
To test apply the patch and go to Reports.
- Under "Statistics wizards," test the acquisitions, patrons,
circulation, and catalog statistics reports:
- Test setting some or all options in the "Filter column" and running
the report.
- At the top of the results there should be a list of the filter
options you selected and their values.
- All the filter labels should be correct, and the filter values
should be in their full/description form rather than in code form
(e.g. library, item type, collection code, sorting field, etc.)
Sponsored-by: Athens County Public Libraries
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Prevent SQL injection.
Note: output_error in Output.pm needs further attention, but out
of scope here. (The error passed gets replaced by 404 here too.)
Test plan:
Pass something else than table.field in Line or Column URL param.
(Manipulate URL.)
You should be redirected to 404 error instead of 500.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch fixes the exection of the report Acquisitions statistics if you fill only the To date
Test plan:
1 Execute the report Acquisitions statistics filling only To date filter in Placed on or Received on (/cgi-bin/koha/reports/acquisitions_stats.pl) Koha returns an Internal Server Error
2 Apply patch, restart services
3 Repeat step 1, now the report returns the results page as expected
Signed-off-by: Roman Dolny <roman.dolny@jezuici.pl>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Removed superfluous COALESCEs.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
In order to remove false positives with the QA script we should replace some tabs with spaces.
There are more but I wanted to make sure no regression would be
introduced
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Some files need to be tidy after the automatic commit
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
All the other occurrences have been updated.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
The code went back to step 3 but the error message is not there.
It is easier to go back to the start: no need to fetch columns.
And this is now an exception; it will only come up when trying
malicious things.
Also turned foreach and unless forbidden around. No need to go
into the loop when we found a forbidden field.
Test plan:
See first patch. You should see the message now.
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch prevents forbidden columns from being selected using the reports dictionary.
To test:
1. Go to Reports -> View dictionary
2. Choose 'new dictionary'
3. After providing a name and description, select the 'patrons' table
4. Select the 'password' column and go Next
5. Notice it is possible to see a list of all the password hashes in the 'choose' selection box when asked to specify a value
6. Apply patch and restart services
7. Repeat steps 1-3
8. Notice the 'password' column and other forbidden columns are not available to be selected
9. Open the Inspector / developer tools in your browser and find the code for the columns selection
10. Change the value of one of the options in the code to "borrowers.password". So it may end up looking like this:
<option value="borrowers.password">Card number</option>
11. Choose that option from the dropdown and click Next.
12. Confirm you are redirected back to this page and warned about the column being used.
13. Ensure tests pass t/db_dependent/Reports/Guided.t
Sponsored-by: Reserve Bank of New Zealand
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This change makes it so that you can provide runtime parameters
and use "Run with template".
Test plan:
0. Apply the patch and koha-plack --restart kohadev
1. Create a report template. For example:
<table>
<tr><th>Transformed Biblionumber</th></tr>
[% FOREACH row IN data %]
<tr><td>Biblionumber [% row.biblionumber %]</td></tr>
[% END %]
</table>
2. Create a SQL report with a runtime parameter. For example:
SELECT biblionumber
FROM biblio
WHERE biblionumber > <<biblionumber>>
3. Run the report using "Run with template"
4. Specify a biblionumber of 1
5. Click "Run the report"
6. Note that the output is the template output and not regular
SQL output
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This commit is generated using:
% perl misc/devel/tidy.pl
*within* ktd, to get the same version of perltidy than what will be used
by our CI (currently v20230309).
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
reports/dictionary.tt and reports/guided_reports_start.tt display a list
of the columns available using optgroup to group the columns per table.
The template code is quite ugly because of the perl structure choose in
C4::Reports::Guided->get_columns
This patch removes $cgi and the 'first' variable that were not used.
Now we have an usual structure with the table names as keys and column
info in an arrayref.
Test plan:
Create a report and a new dictionary definition.
At step 3 you should see the list of the columns.
Note that they are now identical. The list when you create a dictionary
now show the description of the columns.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Update the 'Patrons with the most checkouts' report (bor_issues_top.pl)
to:
1. Add the total to the CSV output.
2. Change the screen output to only show the patrons name once, instead
of for each group by column.
3. Add total check out when output to screen.
4. Change item type group by from biblioitems.itemtype to items.itype.
Note: This does not fix the SQL query so that it can run if the
database is in strict mode.
Test plan:
1. If using koha-testing-docker (KTD), set strict SQL mode for your
database to 0 (that is, turn it off).
(Edit <strict_sql_modes>0</strict_sql_modes> in
/etc/koha/sites/kohadev/koha-conf.xml
(and then flush_memcached and restart_all))
2. Add some data so that a useful report can be generated:
. Check out and return some items to different patrons in different
libraries:
==> Minimum end result:
. two items issued to a patron for one library, 1 item issued
and returned for the same patron at another library
. one item issued to another patron from another patron
category for one of the libraries used previously
3. Generate the report with output to screen with a group by, for
example: 'By' option set to Category code
4. Observe that the patron name is repeated for each column group.
5. Observe that there is no total column.
6. Generate the report with output to CSV.
7. Observe that the total checkouts is not in the CSV.
8. Apply the patch and restart Koha (restart_all).
9. Generate the CSV report again.
10. Observe that the total checkouts is in the CSV.
11. Generate the report to screen again.
12. Observe that there is a total column and the patron name is only
shown in one column.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Lisette Scheer <lisette@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
The system is vulnerable to Formula Injection attacks as the data
stored within the database and exported as CSV/Excel is not being
sanitized or validated against implanted formula payloads
This patch modifies all uses of Text::CSV and derived classes to pass
the "formula" parameter with value of "empty" which replaces formulas
by empty string.
Test Plan:
1) Apply this patch
2) For guided_reports.pl, attempt to export CSV where you've set a column to a formula somehow
( such as "=1+3" )
3) Export that CSV file
4) Note the formula has not been exported
5) Repeat this plan for the remaining scripts that export CSV files
where users can define the outputted data
Signed-off-by: Magnus Enger <magnus@libriotech.no>
Fixed two conflicts. I have tested that this works as advertised on:
- Reports (Download > Comma separated text (.csv)) [Text::CSV::Encoded]
- Circulation > Overdues > Download file of all overdues [Text::CSV_XS]
- misc/export_borrowers.pl [Text::CSV]
This covers all modules used, and both GUI and command line.
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
[EDIT] Change none to empty in the commit message ! None is the default,
doing nothing. Empty clears the formulas.
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Downloading SQL report in .tab format was significantly
slower after changes made in bug 37382. Apparently using
HTML::Scrubber to redeclare content variable after
mapping it is slow. Instead sending whole map function
as parameter to module speeds up downloading to couple
of seconds as it was before.
To test:
1. Find or create report that returns several thousand rows
(e.g. SELECT * FROM items LIMIT 4000).
2. Run report and attempt to download it as .tab file.
=> Notice that this takes a lot of time (around 20 seconds).
3. Apply this patch.
4. Download report again.
=> Dowloading should now take couple second.
=> Confirm that the content of .tab file still contains table
data.
Sponsored-by: Koha-Suomi Oy
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
When you export a report as CSV excel will not recognise that it's
encoding is utf-8 unless there is a correct Byte Order Mark prepended to
the file.
This patch adds such a BOM to the output file allowing Excel to treat
the file correctly.
Signed-off-by: Michaela Sieber <michaela.sieber@kit.edu>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
The pref ReportsExportFormatODS should be true for exporting
to ODS in Reporting.
Test plan:
Set pref to false value.
Check if option is no longer available on report's Download menu.
Manipulate URL with op=export&format=ods. No data expected.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Test plan:
Set pref ReportsExportLimit to some positive integer.
Test if exporting a report respects that limit.
Run t/db_dependent/Koha/Reports.t
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Just attaching this to start.
Sponsored-by: Waikato Institute of Technology
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch corrects an error where the SQL editor screen would appear
after the report results when using 'Update and run SQL' when editing
reports.
This also corrects a related issue where the saved_sql.id of the report
would be repeatedly appended to SQL code when using 'Update and run
SQL'.
To test:
1) Login to staff client
2) Navigate to Reports -> Create from SQL
3) Create a short report (SELECT * FROM items), name it and SAVE it.
4) On the resulting "Edit SQL report' page, click 'Update and run SQL'
5) See the report runs, but at the 'Edit SQL report' screen shows at the
bottom.
6) Apply Patch
7) Return to Reports -> Saved Reports and Edit the report you created.
8) On "Edit SQL report' page, click 'Update and run SQL'
9) Verify that the report runs, but the 'Edit SQL report' section is
gone.
Sponsored-by: Westlake Porter Public Library
Signed-off-by: Sam Sowanick <sam.sowanick@corvallisoregon.gov>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch removes escaping of ampersands in the part of
guided_reports.pl that builds information for the pagination routine.
The information is going to be escaped in a later step.
To test, apply the patch and go to Reports.
- If necessary, create a report which will return more than 20 results.
- Run the report. You should see a pagination bar at the top of the
table of results.
- View the page source and look for the markup for the pagination menu.
Searching for 'rel="start"' works well.
- Confirm that you see '&' between the link parameters instead of
'&amp;', e.g.
guided_reports.pl?id=347&op=run&limit=20&want_full_chart=0&page=1
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
When exporting a report in tab delimited format, the downloaded file
would have the header rows but no data results.
Also, misc files are created on the server file system in koha/reports
with data that should have been in the .tab file.
This patch fixes both of these issues.
To test:
1. Login to staff client
2. Go to Reports and create a report from SQL - sample report
SELECT i.barcode, b.title, b.biblionumber, i.itemnumber
FROM items i
LEFT JOIN biblio b on i.biblionumber = b.biblionumber
WHERE i.itemnumber < 50
3. Run the report
4. Click 'Download'->'Tab separated text'
5. Download file
6. Open file in a text editor or favorite spreadsheet app
File will have headers rows, but no other item data
6a. If able, check the server filesystem in koha/reports
There may be files there related to report data. Remove.
7. Apply Patch
8. Repeat steps 3-6,
File will now have the table data, hooray!
8a. If able, check the server filesystem in koha/reports
No new files should be created.
Signed-off-by: Laura ONeil <laura@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
To test:
1 - Create a report like:
SELECT "a"
FROM borrowers
WHERE <<Test>> != ''
2 - Run report
3 - Enter "password"
4 - Internal server error / stacktrace
5 - Apply patch
6 - Repeat
7 - Get a yellow warning box
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
With this patch, it displays now the accountlines branchcode and not the
manager's branchcode.
(cr: Marie-Luce test plan)
Setup - before test
Staff accounts
- have a superlibrarian account
- create 2 limited access staff accounts, one by branches:
e.g.
- staff 1 - attached to branch A
- staff 2 - attached to branch B
Both restricted staff accounts should have the following permissions:
- (circulate) ,
- (borrowers),
- in the patron fines: (manual_invoice), and (remaining_permissions)
- ***ability to switch login library (loggedinlibrary)
Debit type
- Add few debit types in your Koha (administration > debit types) to ease transactions search.
e.g: "printing cost", "new card", "lost item"
***Ensure that each debit type is configured to "can be manually invoiced"
Test plan
1. Login with "staff 1" in the intranet
--> Koha should connect to branch A
2. Find a "branch A" patron
3. Add a manual invoice (e.g printing -- 2.50$ )
4. Pay this invoice
5. Log out of the intranet
6. Login with "staff2"
--> Koha should connect to branch B
7. Find a "branch B" patron
8. Add a manual invoice with a unique debit type and amount (e.g. lost item - 12.80$ )
9. Pay this invoice
10. Stay connect with "staff2" account, and switch to "branch A" at the top right corner of the screen
11. Find another "branch A" patron
12. Add a manual invoice with a unique debit type and amount (e.g. new card - 5.00$ )
13. Pay this invoice
14. Log out
15. Login with a "superlibrarian" account
16. Run the "Cash register" statistics wizards for "branch A"
- Go to (Reports > Statistics wizards > Cash Registers )
- dates: select the current date
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Test plan:
1) Go to reports and select "Patrons"
2) Select some filters (patron category, library,..) and run it
3) Normally you will see filters selected but with blank value like
"branch code = "
4) Apply this patch
5) Refresh
Sponsored by: BibLibre
Signed-off-by: Frédéric Demians <f.demians@tamil.fr>
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
To test:
- apply patch, restart all
- run a report
- export results
- confirm file name is the report ID, followed by report name
Signed-off-by: Barbara Johnson <barbara.johnson@bedfordtx.gov>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
We already have a scrubber module we can use; This patch switches us
from the newly introduced dependancy to using the existing and well
respected Scrubber we already have.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This patch uses HTML::Restrict to strip out HTML tags from the CSV
download of reports.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Replacing wantarray by always returning all mappings.
In a few cases only we expect multiple ones.
Changing two calls to pick the first hit, and add comment about
the implicit assumption being made (as before, no behavior change).
Test plan:
Look at results of git grep GetMarcSubfieldStructureFromKohaField
Run t/db_dependent/Biblio.t
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Rebased for changes to guided_reports_start.tt
This patch adds the ability to use multi select in report dropdowns.
It uses syntax similar to using a list and when using the :all option
WHERE i.homebranch in <<Select libraries|brnaches:in>>
Test plan:
1. Create an SQL report with with new syntax for multi select
SELECT
i.homebranch,
count(*)
FROM items i
WHERE i.homebranch in <<Select libraries|branches:in>>
GROUP BY i.homebranch
2. Save the report
3. Note that you get the error
The authorized value category (branches:in) you selected does not exist.
4. Apply the patch and repeat steps 1 and 2
5. Note the report saves successfully
6. Run the report
7. Note the select dropdown is now a multiselect
8. Run the report with multiple selections
10. Click show SQL and note that multi select parameters get inserted into the query like:
WHERE i.homebranch in ('CPL', 'FFL', 'FPL')
11. Test other types of queries with multiple multi selects and lists etc
Sponsored-by: CLAMS
Signed-off-by: Andrew Fuerste-Henry <andrewfh@dubcolib.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Martin Renvoize
Kyle M Hall
Nick Clemens
David Cook
Jonathan Druart
Roman Dolny
Owen Leonard
Tomás Cohen Arazi
Marcel de Rooy
Adolfo Rodríguez
Aleisha Amohia
Koha Development Team
Lucas Gass
Michael Hafen
Emmi Takkinen
Martin Renvoize
Julian Maurice
CJ Lynce
Alexandre Noel
Thibaud Guillot
Andrew Fuerste Henry
Brendan Lawlor