Signed-off-by: Chris Nighswonger <cnighswonger@foundations.edu>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
http://bugs.koha-community.org/show_bug.cgi?id=9987
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
1) Removed 'use C4::Context;' because it can lead to introduction
of circular reference in the near future
2) Put fastmmap initialization code into an eval {} block, to catch
various kinds of errors which can still occur during it's init in
some [less usual] Koha setups and/or more unusual circumstances
3) Do not include UID in the sharefile name (it will be constructed
using namespace + database name + database host instead).
Test plan addendum:
s/and UID//
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
Koha::Cache package does not take into account that, when using
fastmmap caching variant, mmaped cache file created in /tmp
(typically: /tmp/sharefile-koha-koha), would only be further
accessible to the one given OS user - the one which created it.
In many Koha setups, in the circumstances when various system scripts
are executed by 2+ users with diffrent UIDs (like multi-tenant servers,
for example) this may cause many kinds of issues. Observable symptom
is usually the appearance of the below error when searching, or looking
at MARC Framework pages and a few other places:
Open of share file /tmp/sharefile-koha-koha failed: Permission denied
at /usr/lib/perl5/Cache/FastMmap.pm line 640.
This patch:
- disables initialisation of fastmmap caching subsystem unless it is
explicitly requested by the user (CACHING_SYSTEM=fastmmap)
- disables fastmmap cache usage for command line scripts
(i.e. when GATEWAY_INTERFACE environment variable is not defined)
- adds the database name, host name and an ID of the OS user to the
mmaped file name created in /tmp, to prevent various kinds of
unintentional conflicts and/or permission problems from happening
To test:
1) remove the /tmp/sharefile-koha-* file[s] (if any)
2) do something which would lead to its re-creation (e.g., performing
any search in OPAC should be sufficient to cause that)
3) observe that /tmp/sharefile-koha-koha got created
4) remove it
5) apply patch
6) redo step 2)
7) observe that aforementioned file is no longer created in /tmp
8) set CACHING_SYSTEM environment variable to 'fastmmap'
9) redo step 2), observe that /tmp/sharefile-koha-* file got created
and that it's name now contains hostname, database name and UID
10) ensure that everything still works like it should and that there
are no regressions of any kinds anywhere in the system ;)
Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This fixes a minor POD error detected, and also adds the Koha/ directory
to the regular test case so that the in-repo tests can find these
things.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
This makes Koha::Cache behave better by default. It will use memcached
if available to do shared caching, if that's not available it will fall
back to in-memory caching. It also allows for a singleton accessor to
allow a single cache to be shared within a process.
* Added tests to confirm UTF8-cleanness.
* Added minor fixups to stop warnings.
Test plan:
* The t/Cache.t file runs successfully with and without the
MEMCACHED_SERVERS envvar set (and memcached running in the
environment.)
Signed-off-by: Brendan Gallagher <brendan@bywatersolutions.com>
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@gmail.com>
At the moment we cache numerous pieces of information in module-level
variables which then do not get updated in other threads/processes when
they are changed by the user. This is a serious usability issue.
Examples of this include the way we treat sysprefs (there is now a
method to disable the syspref cache, but by default it is enabled),
notices, frameworks, field mappings, and koha-conf.xml, at least.
This patch sets the stage for eliminating this problem by making it
possible to convert module-level cache variables into variables that
are actually backed by whatever caching system may be configured. This
is done through a special Koha::Cache::Object class which can be tied
to the variables that are being used for caching and provided with a
constructor method/closure to allow the cache to be reloaded when it
expires. For example:
my $cache = Koha::Cache->new();
my $data = 'whatever';
my $variable = Koha::Cache->create_scalar(
{
'key' => 'whatever',
'timeout' => 2,
'constructor' => sub { return $data; },
}
);
print "$$variable\n"; # Prints "whatever"
The one change this necessitates for accessing the data is that the
variable must be dereferenced an additional time before use (i.e.
$$variable instead of $variable). There is no difference when the
variable tied is a hash (created with Koha::Cache->create_hash). This
is a small price to pay for Koha working in a multi-threaded, persistent
environment. This change will also make caching easier in general.
CHI was incompatible with the variable tying, so this patch also removes
the dependency on CHI, using instead Cache::Memcached::Fast,
Cache::FastMmap, and Cache::Memory, when they are available.
To test:
1) Apply patch.
2) Run unit test t/Cache.t (after setting the MEMCACHED_SERVERS and
CACHING_SYSTEM environment variables). As no changes were made to
the tests already in that file, this passing demonstrates there
are no regressions.
3) With memcached caching enabled (you must set the MEMCACHED_SERVERS
and CACHING_SYSTEM environment variables) and DEBUG turned on (i.e.
the DEBUG environment variable set to 1), try running a report via
the web service ([intranet]/cgi-bin/koha/svc/report?id=1 and check
your web server logs to confirm that there are messages like
"get_from_cache for intranet:report:id:1" in them.
4) If the reports worked, sign off.
NOTE: Technically you could test this without needing memcached by
installing libcache-fastmmap-perl and setting CACHING_SYSTEM to
'fastmmap' instead of 'memcached'. You could also install libcache-perl
and set CACHING_SYSTEM to 'memory' but there would be little point as
the cached variables would go out of scope in between runs.
Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com>
Signed-off-by: Paul Poulain <paul.poulain@biblibre.com>
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
This reverts commit 215abc8024.
The 3 patches for bug 8089 have been reverted, because they break
jenkins & Koha.
A follow-up has been provided, but it does not solve the problem on my
test server, it just changes the error message.
After a discussion with jared, Dobrica should work on another patch, so
the best option is to revert.
1. Replace all instances of memoize_memcached with appropriate calls
into Koha::Cache:
* reports/guided_reports.pl
* C4::Biblio::GetMarcStructure
* C4::Languages::getFrameworkLanguages
* C4::Languages::getAllLanguages
* C4::SQLHelper::GetPrimaryKeys
* C4::SQLHelper::_get_columns
2. Replace all references to memcached with the appropriate calls into
Koha::Cache in C4::Context.
Test plan :
* have DEBUG env set to 1
* reach addbiblio page to test the patch in Biblio.pm, or setup more than 1
language
* you should see in the logs that you're reading and writing from cache
* run the test suite twice both with and without the following environment
variables set:
export MEMCACHED_SERVERS=127.0.0.1:11211
export MEMCACHED_NAMESPACE=KOHA
export CACHING_SYSTEM=memcached
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
I'm unsure about some of the caching times 10000 is a long long time,
but other than that, works fine.
Implements cache handlers for Memcached, mmap shared-file persistent,
and in-process memory caches. If CHI is unavailable, Koha::Cache::Memcached
will fall back to using Cache::Memcached::Fast, or caching will be skipped
without croaking.
To test: run t/Cache.t with the following options, before and after
installing CHI:
2. Tests 3-9 should be skipped with the following:
export CACHING_SYSTEM=
2. You should receive two failures with the following if CHI is not installed:
export CACHING_SYSTEM=memory
3. You should receive two failures with the following if CHI is not installed:
export CACHING_SYSTEM=fastmmap
4. You will need memcached activated for the following to work (but it
will work both with and without CHI):
export CACHING_SYSTEM=memcached
export MEMCACHED_SERVERS=127.0.0.1:11211
export MEMCACHED_NAMESPACE=KOHA
5. You should receive two failures with the following:
export CACHING_SYSTEM=thisdoesntexist
Signed-off-by: Chris Cormack <chris@bigballofwax.co.nz>
Works as advertised, now we need a follow up to add the new dependency.
http://bugs.koha-community.org/show_bug.cgi?id=8029
This patch introduces some new features for caching system:
* the type of caching is retrieved from ENV variable in httpd configuration if not forced when the ->new() is called.
* if $ENV{DEBUG} is ON, you'll have feedback when something is set or read from cache
* the Koha::Cache->is_cache_active is now available and will return 1 if there is a caching system available. It's a replacement for C4::Context->ismemcached
* fixes the mistake for debug/compress_threshold and expire_time parameters
The 2 report web service modules have been updated to use this new API
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Unit test for Koha/Cache.pm (which includes Koha/Cache/Memcached.pm)
Note that in order to test Koha::Cache you must export the environment variable
MEMCACHED_SERVERS. For example:
$ export MEMCACHED_SERVERS=127.0.0.1:11211
Signed-off-by: Jared Camins-Esakov <jcamins@cpbibliography.com>
Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de>
Jonathan Druart
Jacek Ablewicz
Robin Sheat
Jared Camins-Esakov
Paul Poulain
Chris Hall