Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Added head2 right before sub with missing POD
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
This commit is generated using:
% perl misc/devel/tidy.pl
*within* ktd, to get the same version of perltidy than what will be used
by our CI (currently v20230309).
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
On bug 17591 we discovered that there was something weird going on with
the way we export and use subroutines/modules.
This patch tries to standardize our EXPORT to use EXPORT_OK only.
That way we will need to explicitely define the subroutine we want to
use from a module.
This patch is a squashed version of:
Bug 17600: After export.pl
Bug 17600: After perlimport
Bug 17600: Manual changes
Bug 17600: Other manual changes after second perlimports run
Bug 17600: Fix tests
And a lot of other manual changes.
export.pl is a dirty script that can be found on bug 17600.
"perlimport" is:
git clone https://github.com/oalders/App-perlimports.git
cd App-perlimports/
cpanm --installdeps .
export PERL5LIB="$PERL5LIB:/kohadevbox/koha/App-perlimports/lib"
find . \( -name "*.pl" -o -name "*.pm" \) -exec perl App-perlimports/script/perlimports --inplace-edit --no-preserve-unused --filename {} \;
The ideas of this patch are to:
* use EXPORT_OK instead of EXPORT
* perltidy the EXPORT_OK list
* remove '&' before the subroutine names
* remove some uneeded use statements
* explicitely import the subroutines we need within the controllers or
modules
Note that the private subroutines (starting with _) should not be
exported (and not used from outside of the module except from tests).
EXPORT vs EXPORT_OK (from
https://www.thegeekstuff.com/2010/06/perl-exporter-examples/)
"""
Export allows to export the functions and variables of modules to user’s namespace using the standard import method. This way, we don’t need to create the objects for the modules to access it’s members.
@EXPORT and @EXPORT_OK are the two main variables used during export operation.
@EXPORT contains list of symbols (subroutines and variables) of the module to be exported into the caller namespace.
@EXPORT_OK does export of symbols on demand basis.
"""
If this patch caused a conflict with a patch you wrote prior to its
push:
* Make sure you are not reintroducing a "use" statement that has been
removed
* "$subroutine" is not exported by the C4::$MODULE module
means that you need to add the subroutine to the @EXPORT_OK list
* Bareword "$subroutine" not allowed while "strict subs"
means that you didn't imported the subroutine from the module:
- use $MODULE qw( $subroutine list );
You can also use the fully qualified namespace: C4::$MODULE::$subroutine
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
We should remove the debug statements or use Koha::Logger when we want
to keep it.
Test plan:
Confirm that occurrences of remaining occurrences of DEBUG need to be
kept (historical scripts for instance)
Confirm that the occurrences removed by this patch can be removed
Confirm that the occurrences replaced by Koha::Logger are correct
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Looks good to me, noting a few minor points on BZ.
JD amended patch: replace "warn #Finished" with "#warn Finished", and
put the statement on a single line
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
memcached address and namespace are in $KOHA_CONF, so it is required to
read it before being able to access the cache. And after that,
configuration is kept in memory forever. Storing this in memcached is
useless and even counter-productive, since Koha reads both the file and
the cache
This patch addresses this issue by removing the cache-related code from
C4::Context->new.
It means that C4::Context->new will always read the configuration file,
so this patch also replaces inappropriate calls to
C4::Context->new->config by appropriate calls to C4::Context->config
It also fixes a bug where C4::Context->new would ignore the filepath
given in parameters if there was something in cache.
It also removes a problematic call to Koha::Caches->get_instance.
Because this call was outside of any subroutine, it would have happened
before the initialization of $C4::Context::context (which happen in
C4::Context::import)
Test plan:
1. Do not apply the patch yet
2. Add the following line at the beginning of Koha::Config::read_from_file
warn "read_from_file($file)";
This will allow you to check how many times the file is read.
3. Flush memcached and restart starman
4. Check the logs, you should see "read_from_file" a bunch of times
5. Apply the patch
6. Re-add the line from step 2
7. Flush memcached and restart starman
8. Check the logs, you should see "read_from_file" only once
9. Make sure the memcached config from $KOHA_CONF (memcached_servers,
memcached_namespace) is taken into account by checking the About page
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
It is known to be broken, we should remove its implementation
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
When a high enough number of forks try to access for example system
preferences with Koha::Cache using memcached as backend the results of
different cache requests get mixed up.
The problem is fixed by using Cache::Memcached::Fast::Safe that is a
fork safe version of Cache::Memcached::Fast.
Sponsored-by: The National Library of Finland
Signed-off-by: David Cook <dcook@prosentient.com.au>
Works as described, and solves an insidious difficult to debug
problem in Koha.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
When a high enough number of forks try to access for example system
preferences with Koha::Cache using memcached as backend the results of
different cache requests get mixed up.
The problem is fixed by using Cache::Memcached::Fast::Safe that is a
fork safe version of Cache::Memcached::Fast.
Sponsored-by: The National Library of Finland
Signed-off-by: David Cook <dcook@prosentient.com.au>
Works as described, and solves an insidious difficult to debug
problem in Koha.
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
If Plack is running with several workers we must not use Cache::Memory
as L2 cache.
If a value is set from a worker, it will not be available from other
workers as the Cache::Memory instance is not shared (of course!)
Moreover we now have Koha::Cache::Memory::Lite that does the same job,
so we should not expect performance regressions by removing it.
See also the email sent to koha-devel for more info
http://lists.koha-community.org/pipermail/koha-devel/2018-December/045004.html
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Julian Maurice <julian.maurice@biblibre.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
If memcached or the connection to it is misconfigured, show simple warnings to help
identify the problem.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
This patch does the following:
[1] Move some POD lines from Cache to Caches.
[2] Correct C4::Plugins to Koha::Plugins in POD line of Koha::Plugins
[3] POD Koha/AuthorisedValue.pm: lib_opac moved to opac_description
[4] The POD in Koha/Patron.pm uses head2 and head3 inconsistently.
Ran s/^=head2/=head3/ on those lines (7 substitutions on 7 lines)
[5] Correct a copied POD line from reports/issues_stats.pl in
reports/reserve_stats.pl.
[6] Correct a test description in t/db_dependent/Koha/Authorities.t.
You should never delete the library :)
[7] Correct typo shouild in a comment of rebuild_zebra.pl
Test plan:
[1] Read the patch. Does it make sense?
[2] Run perldoc Koha/Cache.pm and Koha/Caches.pm
[3] Run t/db_dependent/Koha/Authorities.t
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
From the POD of Koha::Cache->set_in_cache:
# This is a bit of a hack to support the old API in case things still use it
Let's remove this hack and update old calls.
Test plan:
Look at the results of
git grep set_in_cache
and confirm that there are no more version of the old call (without
hashref as third param)
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Resolve typo inifinite too.
Adds a few lines in order to stress that the thawed key of the L1
cache SHOULD ONLY be used for unsafe calls, and not be mixed with
regular (safe) calls.
Test plan:
Nothing to test, but verify the quality of the added comments.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This patch removes an occurence of an unused var in Context.pm and
also prevents a warning if memcached_servers is not set:
Use of uninitialized value in split at /home/vagrant/kohaclone/Koha/Cache.pm line 91.
t also tidies small things.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
TODO: Would be better to provide a better way to know if memcache is correctly set
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Move C4::Context->read_config_file to Koha::Config
That permits to reuse it from Koha::Cache without needing C4::Context
(and introduce a circular deps).
TODO: Add decent POD to Koha::Config
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
MEMCACHED_SERVERS and MEMCACHED_NAMESPACE should be moved back to
koha-conf instead of an ENV var to let command line scripts use
the cache mechanism.
TODO:
- Add the entries to koha-conf.xml
- Remove occurrences of MEMCACHED_SERVERS in package, apache and plack files
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
https://bugs.koha-community.org/show_bug.cgi?id=11921
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
We need to define several namespaces for our cache system.
For instance sysprefs, koha conf (koha-conf.xml) and unit tests
should be defined in a separate namespace.
This will permit to
- launch the tests without interfering with other cache values
- and flush the sysprefs cache without flushing all other values
To do so, we need to store different Koha::Cache objects at a package
level. That's why this patch adds a new Koha::Caches module.
FIXME: There is an architecture problem here: the L1 cache should be
defined in Koha::Cache
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
https://bugs.koha-community.org/show_bug.cgi?id=11921
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Sereal is not packaged for jessie, so let's use Sereal::Encoder and
Sereal::Decoder instead.
Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
This manages to eke out a bit more performance on my machine.
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Bug 16044 introduced two-level caching mechanism to Koha::Cache. By
default, get_from_cache() returns a deep copy of the data structures
stored in L1 cache (aka "safe mode"). For extremely big and/or complex
data structures (like MARC framework hash-of-hashes-of-hashes returned
by GetMarcStructure() ), deep-cloning is currently sub-optimal;
this patch addresses that particular issue. It also provides an
ability to intermix "safe" and "unsafe" cache feches, in such
way that they don't interfere with each other (i.e., eliminating
the risk involved with "unsafe" calls possibly compromising
further "safe" calls).
Test plan:
1) apply patch
2) flush L2 cache if necessary (restart memcached daemon)
3) profile GetMarcStructure() calls before / after patch, e.g. by
running some script which calls it often (like catalogue search
w/ XSLT processing turned on, etc.) - GetMarcStructure() should be
faster than before, in all possible circumstances (eg. 18 msec
per call -> 10 msec p/call)
4) after testing, before returning to the master branch, flush L2 cache
again (restart memcached daemon) - otherwise all system preferences
returned from L2 cache would be suffixed with '-CF0'
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
No need to cache undef in L2.
Prevent numerous warnings like:
Use of uninitialized value in subroutine entry at /usr/share/koha/masterclone/Koha/Cache.pm line 284
Test plan:
Run t/Cache.t
Run t/db_dependent/sysprefs.t
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Amended patch: remove trailing space char and replace 'you' with 'we' to
match existing wordings.
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
This reverts commit 6b319a2d48.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Tested with memcached and without MEMCACHED_SERVERS.
Cache.t, Context.t and sysprefs.t pass now.
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Storable dclone() is over 3x faster then Clone::clone() when used for
creating a deep copies of the big, complex data structures like MARC
frameworks. See also Bug 16044 comments #18 & #20.
This is a trivial version of Bug 16166. Performance test
results (see Bug 16140 comment #23 for test setup description):
master
146.29 (14.64+14.53+14.51+14.57+14.72+14.56+14.53+15.06+14.56+14.61)
master + Bug 16221
96.5 (9.77+9.63+9.77+9.77+9.68+9.05+9.68+9.83+9.63+9.69)
master + Bug 16166
90.7 (9.15+9.10+9.16+8.41+9.19+9.19+9.11+9.19+9.02+9.18)
Bug 16166 is a bit faster and more comprehensive implementation, but
also more complex and harder to test/evaluate.
Test plan:
1) apply patch
2) profile GetMarcStructure() calls before / after patch, e.g. by
running some script which calls it often (like catalogue search
w/ XSLT processing turned on, etc.)
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Cache::Memory is loaded and Koha::Cache::_initialize_memory is called
even if a memcached cache has been correctly initialize, it does not
make sense.
It should only be initialize if needed.
Test plan:
Correctly configure a memcache server and confirm that the memory cache
is not initialize.
Do not configure correctly a memcache server and confirm that the cache
system default on Cache::Memory
Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Since bug 11998, the following warnings are raised:
Name "Tie::Hash::FIELDS" used only once: possible typo at /usr/share/perl/5.22/fields.pm line 135.
Name "Cache::RemovalStrategy::LRU::FIELDS" used only once: possible typo at /usr/share/perl/5.22/fields.pm line 135.
Name "Cache::RemovalStrategy::FIELDS" used only once: possible typo at /usr/share/perl/5.22/fields.pm line 135.
It comes from the Koha::Cache
103 if ( can_load( modules => { 'Cache::Memory' => undef } ) ) {
Test plan:
perl -wc C4/AuthoritiesMarc.pm
should return green
QA note: this may have an impact on performance but I have not found any other workarounds.
NOTE: I applied 15870 to master, got the error (-v 2 -c 5), applied 16104,
rebased it in front, error disappeared (still only -c 5), rebased it at the end,
and signed off 16104.
Signed-off-by: Mark Tompsett <mtompset@hotmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Brendan Gallagher <bredan@bywatersolutions.com>
Could be useful later.
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Brendan Gallagher <bredan@bywatersolutions.com>
Koha::Cache->set_in_cache should deep copy (if needed) to avoid the
value which has been set in cache to be unintentionally modified later.
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Brendan Gallagher <bredan@bywatersolutions.com>
When a value exists in L2 cache but not in L1 cache, it should be deep
copied if needed (i.e. not a scalar). Otherwise the calling code is able
to modify the value in cache.
Note that is theoretical, it's possible that no code does that.
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Tomas Cohen Arazi <tomascohen@unc.edu.ar>
Signed-off-by: Brendan Gallagher <bredan@bywatersolutions.com>
The whole patch set is not very pertinent if the L1 cache is not
populated when L2 is fetched!
This patch fixes this inconsistency.
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Jacek Ablewicz <abl@biblos.pk.edu.pl>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
If the caller/developer knows what he is doing, he can decide not to
deep copy the structure. It will be faster but unsafe!
If the structure is modified, the cache will also be updated.
This option must be used with care and is not the default behavior.
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
To avoid the cache to be modified unfortunately, the default behavior of
get_from_cache will be to deep copy if we are getting a structure.
If the item is a scalar, it's simply returned.
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Jesse Weaver <jweaver@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan A Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
http://bugs.koha-community.org/show_bug.cgi?id=9987
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
1) Removed 'use C4::Context;' because it can lead to introduction
of circular reference in the near future
2) Put fastmmap initialization code into an eval {} block, to catch
various kinds of errors which can still occur during it's init in
some [less usual] Koha setups and/or more unusual circumstances
3) Do not include UID in the sharefile name (it will be constructed
using namespace + database name + database host instead).
Test plan addendum:
s/and UID//
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Koha::Cache package does not take into account that, when using
fastmmap caching variant, mmaped cache file created in /tmp
(typically: /tmp/sharefile-koha-koha), would only be further
accessible to the one given OS user - the one which created it.
In many Koha setups, in the circumstances when various system scripts
are executed by 2+ users with diffrent UIDs (like multi-tenant servers,
for example) this may cause many kinds of issues. Observable symptom
is usually the appearance of the below error when searching, or looking
at MARC Framework pages and a few other places:
Open of share file /tmp/sharefile-koha-koha failed: Permission denied
at /usr/lib/perl5/Cache/FastMmap.pm line 640.
This patch:
- disables initialisation of fastmmap caching subsystem unless it is
explicitly requested by the user (CACHING_SYSTEM=fastmmap)
- disables fastmmap cache usage for command line scripts
(i.e. when GATEWAY_INTERFACE environment variable is not defined)
- adds the database name, host name and an ID of the OS user to the
mmaped file name created in /tmp, to prevent various kinds of
unintentional conflicts and/or permission problems from happening
To test:
1) remove the /tmp/sharefile-koha-* file[s] (if any)
2) do something which would lead to its re-creation (e.g., performing
any search in OPAC should be sufficient to cause that)
3) observe that /tmp/sharefile-koha-koha got created
4) remove it
5) apply patch
6) redo step 2)
7) observe that aforementioned file is no longer created in /tmp
8) set CACHING_SYSTEM environment variable to 'fastmmap'
9) redo step 2), observe that /tmp/sharefile-koha-* file got created
and that it's name now contains hostname, database name and UID
10) ensure that everything still works like it should and that there
are no regressions of any kinds anywhere in the system ;)
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This fixes a minor POD error detected, and also adds the Koha/ directory
to the regular test case so that the in-repo tests can find these
things.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This makes Koha::Cache behave better by default. It will use memcached
if available to do shared caching, if that's not available it will fall
back to in-memory caching. It also allows for a singleton accessor to
allow a single cache to be shared within a process.
* Added tests to confirm UTF8-cleanness.
* Added minor fixups to stop warnings.
Test plan:
* The t/Cache.t file runs successfully with and without the
MEMCACHED_SERVERS envvar set (and memcached running in the
environment.)
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
At the moment we cache numerous pieces of information in module-level
variables which then do not get updated in other threads/processes when
they are changed by the user. This is a serious usability issue.
Examples of this include the way we treat sysprefs (there is now a
method to disable the syspref cache, but by default it is enabled),
notices, frameworks, field mappings, and koha-conf.xml, at least.
This patch sets the stage for eliminating this problem by making it
possible to convert module-level cache variables into variables that
are actually backed by whatever caching system may be configured. This
is done through a special Koha::Cache::Object class which can be tied
to the variables that are being used for caching and provided with a
constructor method/closure to allow the cache to be reloaded when it
expires. For example:
my $cache = Koha::Cache->new();
my $data = 'whatever';
my $variable = Koha::Cache->create_scalar(
{
'key' => 'whatever',
'timeout' => 2,
'constructor' => sub { return $data; },
}
);
print "$$variable\n"; # Prints "whatever"
The one change this necessitates for accessing the data is that the
variable must be dereferenced an additional time before use (i.e.
$$variable instead of $variable). There is no difference when the
variable tied is a hash (created with Koha::Cache->create_hash). This
is a small price to pay for Koha working in a multi-threaded, persistent
environment. This change will also make caching easier in general.
CHI was incompatible with the variable tying, so this patch also removes
the dependency on CHI, using instead Cache::Memcached::Fast,
Cache::FastMmap, and Cache::Memory, when they are available.
To test:
1) Apply patch.
2) Run unit test t/Cache.t (after setting the MEMCACHED_SERVERS and
CACHING_SYSTEM environment variables). As no changes were made to
the tests already in that file, this passing demonstrates there
are no regressions.
3) With memcached caching enabled (you must set the MEMCACHED_SERVERS
and CACHING_SYSTEM environment variables) and DEBUG turned on (i.e.
the DEBUG environment variable set to 1), try running a report via
the web service ([intranet]/cgi-bin/koha/svc/report?id=1 and check
your web server logs to confirm that there are messages like
"get_from_cache for intranet:report:id:1" in them.
4) If the reports worked, sign off.
NOTE: Technically you could test this without needing memcached by
installing libcache-fastmmap-perl and setting CACHING_SYSTEM to
'fastmmap' instead of 'memcached'. You could also install libcache-perl
and set CACHING_SYSTEM to 'memory' but there would be little point as
the cached variables would go out of scope in between runs.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
This reverts commit 215abc8024.
The 3 patches for bug 8089 have been reverted, because they break
jenkins & Koha.
A follow-up has been provided, but it does not solve the problem on my
test server, it just changes the error message.
After a discussion with jared, Dobrica should work on another patch, so
the best option is to revert.
1. Replace all instances of memoize_memcached with appropriate calls
into Koha::Cache:
* reports/guided_reports.pl
* C4::Biblio::GetMarcStructure
* C4::Languages::getFrameworkLanguages
* C4::Languages::getAllLanguages
* C4::SQLHelper::GetPrimaryKeys
* C4::SQLHelper::_get_columns
2. Replace all references to memcached with the appropriate calls into
Koha::Cache in C4::Context.
Test plan :
* have DEBUG env set to 1
* reach addbiblio page to test the patch in Biblio.pm, or setup more than 1
language
* you should see in the logs that you're reading and writing from cache
* run the test suite twice both with and without the following environment
variables set:
export MEMCACHED_SERVERS=127.0.0.1:11211
export MEMCACHED_NAMESPACE=KOHA
export CACHING_SYSTEM=memcached
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
I'm unsure about some of the caching times 10000 is a long long time,
but other than that, works fine.
Jonathan Druart
Tomas Cohen Arazi
Koha Development Team
Julian Maurice
Joonas Kylmälä
Martin Renvoize
Olli-Antti Kivilahti
Marcel de Rooy
Kyle M Hall
Jesse Weaver
Jacek Ablewicz
Tomas Cohen Arazi
Jonathan Druart
Robin Sheat
Jared Camins-Esakov
Paul Poulain