Commit Graph

41749 Commits

Author SHA1 Message Date
Nick Clemens f2ccbd4f08 Bug 29284: Unit test
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit fa56c8b069)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2022-01-31 00:45:57 +01:00
Nick Clemens ff0dce19a0 Bug 29284: Handle the case of an exclamation point in parentheses
This expands the regex to handle this specific case

To test:
1 - Load record created for last patch
2 - Note analytics error
3 - Apply patch
4 - Restart and reload
5 - No more errorm also no Analytics link
6 - Add a 773$t to a record with title used before:
    Digger does it all (not really!)
7 - reload the initial record
8 - See 'Show analytics' link
9 - Click the link
10 - You should end up on the record you added the 773 to

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 7c74eb9124)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2022-01-31 00:45:57 +01:00
Nick Clemens 6ec5e00e6a Bug 29284: Don't die on analytics searching error
This patch adds an eval around the call to search for analytic records
It pases a value to the template on the staff side, but logs the warning on
the opac

This seems similar to 'decoding_error' which is noted on staff side, but absent on OPAC

The eval follows the patter used during searching

To test:
1 - Add a title to catalog, with 245a:
    Digger does it all (not really!)
2 - Set searchEngine preference to: Elasticsearch
3 - The record does not load
4 - Apply patch
5 - The record loads, there is a note about analytics at the top fo the record
6 - View record in opac, no note
7 - Check logs on intranet and opac, searching error is logged

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit ab13f33eae)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2022-01-31 00:45:57 +01:00
Martin Renvoize fc229c123e Bug 28316: (QA follow-up) Make clean_search_term public
With all the work that's gone into improving the internal
_clean_search_term method I feel we should expose it publically as it's
going to be more widely helpful

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 641e8e4096)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2022-01-31 00:44:32 +01:00
Petro Vashchuk 31c94c7f96 Bug 28316: escape ES ranges if QueryAutoTruncate is enabled
if QueryAutoTruncate enabled we will have any special operators ruined
for example: "test [6 TO 7]" will be converted to "test* [6* TO* 7]"
so no reason to keep ranges when QueryAutoTruncate set to "enabled"

1) enable QueryAutoTruncate at your sysprefs.
2) perform a search using range, for example: "[1999 TO 2020]",
it shouldn't work the way it's supposed to.
3) apply the patch.
4) perform the same search with range, ensure that it works correctly.

Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit e84759dfee)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2022-01-31 00:44:32 +01:00
Petro Vashchuk 883e3d3095 Bug 28316: avoid messing up regexes in the search queries
This patch ensures that the behavior with
QueryRegexEscapeOptions set to values other than
"Escape" still will works as expected.

It does so by storing the contents of regexes
before escaping special characters and
then restores the contents of regexes back to how
it was before, ensuring that searching with regex is possible.

Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 59c26ce5f3)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2022-01-31 00:44:32 +01:00
Petro Vashchuk 8de31cbb91 Bug 28316: escape exclamation signs in the query
Currently having exclamation sign at the end of the query makes ES
search fail, and when you try to search for a book that has exclamation
sign in the tittle (something like "Words! words") won't show results
correctly as it tries to negate everything that is after exclamation
sign, making it impossible to search for books that have in in the title

This patch escapes exclamation signs if it's at the end of the query or
has a space after it, resolving both of the issues listed above.

To reproduce:
1) with ES enabled, search for the book with title that contains
exclamation sight at the end, like "book!", this search should result
 in error.
2) do another search, but this time find/prepare beforehand book with a
title that has exclamation sign with a space after it,
e.g "exclamation! sign", it shouldn't find it as ES treats everything
after that exclamation sign as negation.
2) apply the patch.
3) perform searches from the steep one and two again.
Search from step one should no longer fail, while search from the step
two should find that book.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit cb156ac132)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2022-01-31 00:44:32 +01:00
Petro Vashchuk b1961172ee Bug 28316: escape brackets in the search query
This patch screens square and curly brackets which have no special
language meaning.

To reproduce:
1) using ES, search for the book with title that contains
square and/or curly brackets, like "book [second edition]", which will
result in error.
2) apply the patch.
3) search for that book again, ensure that it works now.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 952b5a6a46)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2022-01-31 00:44:32 +01:00
Petro Vashchuk 368fedd5c1 Bug 28316: screen unquoted semicolons and all followup colons
Currently searches like: "book:", ":book" and "host-item:test:n"
cause internal server errors.

This patch adds additional regexes that remove the colons at the start
and end of the query, and another regex that screens all follow-up
colons that go after the first colon to avoid errors when searching for
"host-item:test:n".

To reproduce:
1) using ES, search for the book with title that contains
semicolon at the start or at the end of the line, separated with spaces,
this should cause internal server error.
2) try doing the same with something like "host-item:test:n", it should
result in error as well.
3) apply the patch.
4) repeat steps 1-2, ensure that it works now.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 5b4b14e493)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2022-01-31 00:44:32 +01:00
Petro Vashchuk ec6b3f7091 Bug 28316: add tests
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 405812b407)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2022-01-31 00:44:32 +01:00
Victor Grousset/tuxayo 68428d0eeb Update release notes for 20.11.13 release
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-24 15:50:20 +01:00
Victor Grousset/tuxayo c2eb9ee3ce Increment version for 20.11.13 release
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-23 23:08:43 +01:00
Victor Grousset/tuxayo d38e104834 Fix translations for Koha 20.11.13
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-23 23:06:12 +01:00
Koha translators 861b3d6042 Translation updates for Koha 20.11.13
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-23 22:42:43 +01:00
Martin Renvoize c8c179a32e Bug 27801: Fix javascript price calculations
This patch ensures we're formatting the price values consistently for
the table total and the amount to pay input field.

Test plan
1) Add an item to charge at 0.10.
2) Add this same item 8 or 9 times (Do not use the 'quantity')
3) Note that the table total and the 'Amount paid' values do not match
4) Apply the patch and repeat the above steps.. the values should now
   match.

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
Signed-off-by: Sally <sally.healey@cheshiresharedservices.gov.uk>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 7e72a7dde5)

Signed-off-by: Andrew Fuerste-Henry <andrew@bywatersolutions.com>
(cherry picked from commit 9e16f33b37)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-23 01:39:51 +01:00
Victor Grousset/tuxayo ce1a2f9159 Bug 28316: REVERT ALL: due to string freeze
The follow-up bug 29284 introduces a new string.
Both patchsets will be backported for next release.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-22 20:50:43 +01:00
Victor Grousset/tuxayo cbe0d35757 Bug 29284: REVERT ALL: due to string freeze
Will be backported for next release

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-22 20:50:38 +01:00
Nick Clemens e4b8be4578 Bug 29284: (follow-up) Fix code that I copied from too
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit c32cca53bd)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-22 20:21:55 +01:00
Nick Clemens 471712625b Bug 29284: (follow-up) Improve error logging to prevent warns
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit cf29caf39f)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-22 20:21:55 +01:00
Nick Clemens 2c4a38dab5 Bug 29284: Handle the case of an exclamation point in parentheses
This expands the regex to handle this specific case

To test:
1 - Load record created for last patch
2 - Note analytics error
3 - Apply patch
4 - Restart and reload
5 - No more errorm also no Analytics link
6 - Add a 773$t to a record with title used before:
    Digger does it all (not really!)
7 - reload the initial record
8 - See 'Show analytics' link
9 - Click the link
10 - You should end up on the record you added the 773 to

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 7c74eb9124)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-22 20:21:55 +01:00
Nick Clemens 350e60c99e Bug 29284: Unit test
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit fa56c8b069)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-22 20:21:55 +01:00
Nick Clemens 0186ab49f9 Bug 29284: Don't die on analytics searching error
This patch adds an eval around the call to search for analytic records
It pases a value to the template on the staff side, but logs the warning on
the opac

This seems similar to 'decoding_error' which is noted on staff side, but absent on OPAC

The eval follows the patter used during searching

To test:
1 - Add a title to catalog, with 245a:
    Digger does it all (not really!)
2 - Set searchEngine preference to: Elasticsearch
3 - The record does not load
4 - Apply patch
5 - The record loads, there is a note about analytics at the top fo the record
6 - View record in opac, no note
7 - Check logs on intranet and opac, searching error is logged

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit ab13f33eae)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-22 20:21:55 +01:00
Martin Renvoize 4d0232ce72 Bug 28316: (QA follow-up) Make clean_search_term public
With all the work that's gone into improving the internal
_clean_search_term method I feel we should expose it publically as it's
going to be more widely helpful

Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 641e8e4096)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-22 19:55:22 +01:00
Petro Vashchuk 2ff0677f7f Bug 28316: escape ES ranges if QueryAutoTruncate is enabled
if QueryAutoTruncate enabled we will have any special operators ruined
for example: "test [6 TO 7]" will be converted to "test* [6* TO* 7]"
so no reason to keep ranges when QueryAutoTruncate set to "enabled"

1) enable QueryAutoTruncate at your sysprefs.
2) perform a search using range, for example: "[1999 TO 2020]",
it shouldn't work the way it's supposed to.
3) apply the patch.
4) perform the same search with range, ensure that it works correctly.

Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit e84759dfee)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-22 19:55:22 +01:00
Petro Vashchuk 1bfda79f26 Bug 28316: avoid messing up regexes in the search queries
This patch ensures that the behavior with
QueryRegexEscapeOptions set to values other than
"Escape" still will works as expected.

It does so by storing the contents of regexes
before escaping special characters and
then restores the contents of regexes back to how
it was before, ensuring that searching with regex is possible.

Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 59c26ce5f3)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-22 19:55:22 +01:00
Petro Vashchuk c8f5c155f3 Bug 28316: escape exclamation signs in the query
Currently having exclamation sign at the end of the query makes ES
search fail, and when you try to search for a book that has exclamation
sign in the tittle (something like "Words! words") won't show results
correctly as it tries to negate everything that is after exclamation
sign, making it impossible to search for books that have in in the title

This patch escapes exclamation signs if it's at the end of the query or
has a space after it, resolving both of the issues listed above.

To reproduce:
1) with ES enabled, search for the book with title that contains
exclamation sight at the end, like "book!", this search should result
 in error.
2) do another search, but this time find/prepare beforehand book with a
title that has exclamation sign with a space after it,
e.g "exclamation! sign", it shouldn't find it as ES treats everything
after that exclamation sign as negation.
2) apply the patch.
3) perform searches from the steep one and two again.
Search from step one should no longer fail, while search from the step
two should find that book.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit cb156ac132)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-22 19:55:22 +01:00
Petro Vashchuk 00c41f9d95 Bug 28316: escape brackets in the search query
This patch screens square and curly brackets which have no special
language meaning.

To reproduce:
1) using ES, search for the book with title that contains
square and/or curly brackets, like "book [second edition]", which will
result in error.
2) apply the patch.
3) search for that book again, ensure that it works now.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 952b5a6a46)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-22 19:55:22 +01:00
Petro Vashchuk 6ae9d29846 Bug 28316: screen unquoted semicolons and all followup colons
Currently searches like: "book:", ":book" and "host-item:test:n"
cause internal server errors.

This patch adds additional regexes that remove the colons at the start
and end of the query, and another regex that screens all follow-up
colons that go after the first colon to avoid errors when searching for
"host-item:test:n".

To reproduce:
1) using ES, search for the book with title that contains
semicolon at the start or at the end of the line, separated with spaces,
this should cause internal server error.
2) try doing the same with something like "host-item:test:n", it should
result in error as well.
3) apply the patch.
4) repeat steps 1-2, ensure that it works now.

Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 5b4b14e493)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-22 19:55:22 +01:00
Petro Vashchuk 25024d8438 Bug 28316: add tests
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 405812b407)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-22 19:55:22 +01:00
Victor Grousset/tuxayo 8cc0c61081 Bug ?????: Compile SCSS, forgotten in a previous release cycle
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-22 17:45:26 +01:00
Marcel de Rooy 97819f50f2 Bug 29330: (QA follow-up) Change to message/rfc822
This content-type might be more appropriated to use as a temporary
label for the serialized email message with attachments.

WARNING: perl -cw tells you that the constant is redefined. This has
to do with an already existing module dependency loop of Letters.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 53127abc03)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-21 16:03:26 +01:00
Tomas Cohen Arazi 42015a0450 Bug 29330: (QA follow-up) Use passed MIME type
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Amended: tidied.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 6c96102678)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-21 16:03:26 +01:00
Tomas Cohen Arazi 6fff1cecb7 Bug 29330: Restore handling of serialized MIME messages in message_queue
This patch changes how multipart MIME messages are handled on the
message_queue table.

The email, with the required attachments, is now generated using
Koha::Email, and serialized using Koha::Email->as_string.

This bug also adds Koha::Email->new_from_string which is used to read
that data from the DB, and produce a Koha::Email object, that can be
further augmented/modified using regular Koha::Email methods.

This implementation should be considered a middle ground, with
backportability in mind. higher-level methods should encapsulate setting
the default headers and addresses, to clean the area a bit further.

Preparation:
- You need a valid SMTP configuration in koha-conf.xml. If you use Gmail
  you can generate an 'app password' and set things like this:

 <smtp_server>
    <host>smtp.gmail.com</host>
    <port>587</port>
    <timeout>5</timeout>
    <ssl_mode>STARTTLS</ssl_mode>
    <user_name>youraddress@gmail.com</user_name>
    <password>youpassword</password>
    <debug>1</debug>
 </smtp_server>
- Set KohaAdminAddress to your address.

To test:
1. Pick a patron. Make sure it doesn't have any email address (Acevedo?)
2. Set an overdue notice trigger for its category
3. Check something out, with due date in the past to force an overdue
4. Run:
   $ kshell
  k$ misc/cronjobs/overdue_notices.pl -v
  k$ exit
   $ koha-mysql kohadev
   > SELECT * FROM message_queue WHERE borrowernumber=the_borrowernumber;
=> SUCCESS: A notice has been created
5. Run:
   $ kshell
  k$ misc/cronjobs/process_message_queue.pl --verbose
=> SUCCESS: SMTP is ok => Email is sent
=> FAIL: Your inbox shows an email with weird content
6. Apply this patches
7. Run:
   $ koha-mysql kohadev
   > DELETE FROM message_queue;
8. Repeat 4 and 5
=> SUCCESS: You got an email with an attachment!
=> SUCCESS: The attachment contains an email that couldn't be delivered!
9. Try all the things that enqueue messages :-D
=> SUCCESS: No behavior change
10. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Edited the POD, restoring a few lines that describe the needed hash
keys of the attachments.

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 54546da3f0)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-21 15:58:43 +01:00
Tomas Cohen Arazi 3ad75ad185 Bug 29330: Add Koha::Email->new_from_string
This patch introduces a new method to Koha::Email. This method allows us
to parse a MIME email to initialize the Koha::Email object. This is
particularly important when we are restoring emails from the DB. i.e.
from the *message_queue* table.

To test:
1. Apply this patch
2. Run:
   $ kshell
  k$ prove t/Koha/Email.t
=> SUCCESS: Tests pass! Koha::Email->new_from_string is the correct
counterpart for Koha::Email->as_string!
3. Sign off :-D

Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 2d6a189e89)
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
2021-12-19 21:15:44 +01:00
Fridolin Somers b4aec2b7f0 Update release notes for 20.11.12 release
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2021-11-29 20:55:44 -10:00
Fridolin Somers 40281bc6bc Increment version for 20.11.12 release
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2021-11-29 20:47:17 -10:00
Victor Grousset/tuxayo 5ee3dca8e0 Fix translations for Koha 20.11.12
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2021-11-29 20:29:27 -10:00
Fridolin Somers ed4fb9f822 Merge remote-tracking branch 'translate/20.11.12-translate-20211130' into 20.11.x 2021-11-29 20:28:53 -10:00
Koha translators 701619c513 Translation updates for Koha 20.11.12 2021-11-30 03:20:10 -03:00
Mason James 95eadfd00c Bug 29564: Use List::MoreUtils so SIP U16/Xenial does not break
- run prove t/00-load.t, see error
 - apply patch

    00:07:08.189 koha_1       | #   Failed test 'use C4::SIP::Sip::Configuration;'
    00:07:08.189 koha_1       | #   at t/00-load.t line 46.
    00:07:08.189 koha_1       | #     Tried to use 'C4::SIP::Sip::Configuration'.
    00:07:08.189 koha_1       | #     Error:  "uniq" is not exported by the List::Util module

 - run prove t/00-load.t, see tests pass :0)

https://bugs.koha-community.org/show_bug.cgi?id=29478

https://bugs.koha-community.org/show_bug.cgi?id=29564

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
(cherry picked from commit fb2e8e3450)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2021-11-29 20:16:50 -10:00
Jonathan Druart fed1ee12f3 Bug 29524: (bug 28935 follow-up) Restore modification of some patron's attributes
On
  commit 5f37d8d2f4
  Bug 28935: No filtering on patron's data on member entry pages
we restricted the list of the columns from the borrowers table that can
be modified from the patron edit view.
We were too restrictive, the following 3 attributes can be edited from
this form: privacy_guarantor_fines, privacy_guarantor_checkouts,
checkprevcheckout and lang

Test plan:
Turn on the following prefs:
- AllowStaffToSetFinesVisibilityForGuarantor
- AllowStaffToSetCheckoutsVisibilityForGuarantor
- CheckPrevCheckout (set to 'unless overridden *')
- TranslateNotices
Edit a patron and see the 4 different options are now displayed.
Change their value, save, edit again
Confirm that the values have been saved

Signed-off-by: Owen Leonard <oleonard@myacpl.org>

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 0ca1419b26)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2021-11-25 10:57:36 -10:00
Didier Gautheron c8a36a1b95 Bug 28803: (20.11.x) Add invalid email handling in _send_message_by_email
Quickfix version for 20.11.x because of missing dependancies.
Based on Bug 28996.

Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2021-11-24 15:09:47 -10:00
Martin Renvoize 2712dbeca6 Bug 28803: Add unit tests
This patch updates Letters.t to confirm that invalid email addresses in
the message_queue should not throw an exception when sending mail but
instead set the status to failed and pass error details to the end user.

Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>

Signed-off-by: Joonas Kylmälä <joonas.kylmala@iki.fi>
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2021-11-24 15:09:42 -10:00
Fridolin Somers 1c768b1c99 28870: (20.11.x) fix UT on SendQueuedMessages
Taken from Bug 28803 :
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=125345&action=diff#a/t/db_dependent/Letters.t_sec1
2021-11-24 14:50:53 -10:00
Fridolin Somers 80dccab070 Bug 29341: DBIC schema changes
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2021-11-19 16:07:32 -10:00
Kyle Hall 6fb901ac87 Bug 29341: DBRev 20.11.11.002
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit c2430c6ee0)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2021-11-19 15:55:33 -10:00
Jonathan Druart aa2ec37ce1 Bug 29341: Remove foreign keys on pseudonymized_transactions
Behave like the statistics table and don't remove the code even if the
branch or patron's category is removed.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit f4c61260f5)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2021-11-19 14:32:29 -10:00
Martin Renvoize 7fa33842f3 Bug 29405: Fix date_renewed attribute format in patron spec
This patch just adds the 'date' format string to ensure we are
validating the input/output of the date_renewed field correctly.

Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit e165f57f24)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2021-11-19 14:16:56 -10:00
Christophe Croullebois af3dd5945f Bug 29452: remove unnecessary warns in the logs
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 8c8c7d77cf)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2021-11-19 14:15:03 -10:00
Marcel de Rooy 263a949632 Bug 29321: Remove a last without loop context
This reads better when converted to regular if.
Note that last within such a block is allowed in Perl, but it feels
better to use it only in a loop context.

Test plan:
Read the patch ;)
Bonus: Test if you enter the block by using pref OPACSuggestionMandatoryFields
when adding a suggestion from opac.

Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>

Edit: Added a space between 'if' and '(' :-D

Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
(cherry picked from commit 0144bb310e)
Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com>
2021-11-19 14:13:58 -10:00