For reusability and to limit files to the git index
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Amended-by: Jonathan Druart
Added + 1 to the plan
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The idea of this patchset is to allow increment runs for some of our
tests.
The current (full) test suite runs in 2h20 (Koha_Main/3253 on Docker_15).
One of the main reason is the recent introduction of new xt tests that are run on all files (valid, tidy, pod, codespell, etc).
From a previous run (last week) I noted this top 5:
xt/vue_tidy.t 105446 ms
xt/perltidy.t 175970 ms
xt/js_tidy.t 239615 ms
xt/author/codespell.t 467041 ms
xt/pl_valid.t 2502354 ms
We are going to merge js_tidy.t and vue_tidy.t as it does no longer make
sense to have a difference between them (we had at the beginning of the
vue work, because of differences in the options, now everything is
handled in the .prettierrc config file)
Once this is pushed, we will be able to create a new project on gitlab
and pass some env variables from Jenkins.
Suggestion: project would be named https://gitlab.com/koha-community/koha-ci-results
When everything will be setup, jenkins will run the tests one the whole
test suite, then the next run will be "incremental": we are going to run
the tests on the files that were failing and the files modified by the
last push.
Example:
* "commit abc": all the files are tested, xt/perltidy.t is failing on Koha.pm
* koha-ci-results/perltidy/abc is created and contains ['Koha.pm']
* "commit def" has a modification on about.pl
* xt/perltidy.t is checking tidy on Koha.pm and about.pl, both are tidy
* koha-ci-results/perltidy/def is created and contains []
Test plan:
* Create an empty project on gitlab (keep the default readme)
eg. https://gitlab.com/me/koha-ci-results
* Generate an access token with the developer role
(https://gitlab.com/me/koha-ci-results/-/settings/access_tokens)
Copy the token (in the following noted as TOKEN_REPLACE_ME)
* Unprotect the "main" branch
* Run the following prove command:
```
KOHA_CI_INCREMENTAL_RUN_REPO_URL=https://gitlab.com/me/koha-ci-results.git \
KOHA_CI_INCREMENTAL_RUNS=1 \
KOHA_CI_INCREMENTAL_RUNS_REPORT=1 \
KOHA_CI_INCREMENTAL_RUNS_TOKEN=TOKEN_REPLACE_ME \
prove xt/perltidy.t
```
=> a new commit is created on your repo
* Add some mess in one of your perl file, commit using `--no-verify` to
bypass the git hook
* Run the prove command again
=> only the file you have modified is tested
=> a new commit is created and the file is reported as failing
* Add another commit, fix the failure, add other mess
=> confirm that what you are seeing makes sense :)
* Ideally redo with the different tests modified by this patchset
* Note that you can use the incremental run without the report (the report will
be only for Jenkins).
It will help RM as xt tests will run (way) faster.
Note that you can also export the following env vars to set git author
detail:
* GIT_COMMITTER_NAME
* GIT_COMMITTER_EMAIL
* GIT_AUTHOR_NAME
* GIT_AUTHOR_EMAIL
Question: What are we going to use on Jenkins?
Discussion for QA: How should we handle concurrency?
It's crucial to prevent multiple tests from pushing simultaneously.
To minimize the risk of the repository becoming outdated, we might need to perform
a pull right before each push. Is that enough? Keep in mind that only one job will
be run at the same time (Koha_Main).
Maybe later we will also use it for stable branches.
Rebased-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com>
Signed-off-by: Tomás Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
We were ignoring whole files and parsing the whole file to check if
location and 201 were together. It's way better to check per subroutine.
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
All was not returning all the files present in the git index, but all
Koha core files. Let's rename it with 'core' to remove ambiguity.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This test ensures all Text::CSV usage in Koha includes proper formula
injection protection to prevent CSV formula injection attacks.
The test scans all Perl modules and verifies that Text::CSV constructors
include the 'formula' parameter set to a safe mode ('empty', 'die',
'croak', 'diag', or numeric 1-5).
To test:
1. Apply this patch
2. Run:
$ ktd --shell
k$ prove xt/author/Text_CSV_Various.t
=> FAIL: Test should fail showing files without formula protection:
- Koha/Patrons/Import.pm
- Koha/BackgroundJob/ImportKBARTFile.pm
- Koha/ERM/EUsage/CounterFile.pm
3. Apply the follow-up patch to fix the violations
4. Repeat 2
=> SUCCESS: All tests should pass
5. Sign off :-D
Signed-off-by: David Cook <dcook@prosentient.com.au>
Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
We have insert_final_newline = true to ensure files will contain a new
line at the end. However we have seen commits with "No new line" added
in the git log.
We should have a xt test to prevent more occurrences.
We should also have a test at the QA script level:
https://gitlab.com/koha-community/qa-test-tools/-/issues/96
Test plan:
prove xt/find-missing-new-lines-at-the-end-of-file.t
should return green.
Remove the new line at the end of a file, eg.
vim -b mainpage.pl
:set noeol
:wq
Now
prove xt/find-missing-new-lines-at-the-end-of-file.t
is failing on mainpage.pl.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Including chmod 755 xt/use-test-no-warnings.t
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Maybe we could remove the warnings from misc/devel/tidy.pl if the file
was originally empty but for now it feels safer to skip the tidy of
empty files
Test plan:
prove xt/tt_tidy.t xt/use-test-no-warnings.t
should return green
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
We've got those tests running without warnings now too.
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch adds a new xt test (xt/use-test-no-warnings.t) to make sure
all our Perl test files will use Test::NoWarnings.
It also adds Test::NoWarnings to some files that were missing it and
that didn't need any other changes.
However we still have 7 non-trivial tests that we will consider
exception for now, until they are fixed.
Test plan:
prove xt/use-test-no-warnings.t
should return green.
And all other tests modified by this patch (the qa script will do that
for you)
You can also test the 2 selenium tests:
1. DROP and CREATE the DB
DROP DATABASE koha_kohadev;
CREATE DATABASE koha_kohadev;
restart_all
2. KOHA_TESTING=1 prove t/db_dependent/selenium/00-onboarding.t
3. Repeat 1
2. KOHA_TESTING=1 prove t/db_dependent/selenium/01-installation.t
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
The test added in xt/api.t was not fully accurate, we skipped the
columns that have the same name as the api attribute (ie. the ones not
part of to_api_mapping)
Test plan:
prove xt/api.t should pass
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch makes a couple changes to the tests:
* Remove check for Copyright line. The Copyright line should not be mandatory.
* Remove dependency of GPL-related checks on the copyrigth line
existence. (this highlights several missed checks).
* Skip testing-related templates. Koha ships some templates that are not
excluded correctly.
This 'fix' will highlight existing errors that were skipped until now.
To test:
1. Run:
$ ktd --shell
k$ prove xt/find-license-problems.t
=> FAIL: Tests pass! Some files should make them fail!
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests fail!
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
And only those files.
Currently the test is failing with
02:52:31 koha_1 | # Failed test at xt/find-license-problems.t line 65.
02:52:31 koha_1 | # got: '0'
02:52:31 koha_1 | # expected: '1'
02:52:33 koha_1 | # File ./misc/translator/po/LICENSE has wrong copyright: hasgpl=1, hasv3=1, hasorlater=1, haslinktolicense=0, hasfranklinst=0
02:52:33 koha_1 | # Looks like you failed 1 test of 1381.
02:52:33 koha_1 | [00:51:24] xt/find-license-problems.t
Because a LICENSE file has been added to koha-l10n, and now this test fails on it.
27ab9ff4d2
But we should not test files that are not part of the git index.
Test plan:
0. Do not apply this patch
1. Edit xt/find-license-problems.t, add
warn scalar(@files);
2. Run `prove xt/find-license-problems.t`
=> Notice that 4056 files are processed
Note that you need to have a clean git repo. You can run the following
git command BUT BEWARE /*\ it will remove all changes not committed
`git clean -d -f`
3. Apply this patch, prove again
=> Tests are passing, with the same number of tests
xt/find-license-problems.t .. ok
All tests successful.
Files=1, Tests=4056
Note that I think we lost one file, as Test::NoWarnings is adding a
test. But I don't think it is worth the trouble to find out which one is
missing.
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch adds a new subtest to xt/api.t
It ensures that maxLength will appear in the rest api spec file when
there is a size constraint at the DB level
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Matt Blenkinsop <matt.blenkinsop@openfifth.co.uk>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
We are ready to reuse Koha::Devel::Files in other places where
`git ls-files` is used.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Lucas Gass <lucas@bywatersolutions.com>
This patch changes the regex so it doesn't expect the exclusion
parameters to only be files but accept directories too.
The risk of this is we might at some point catch more than we want.
I don't think it is worth spending more time in it, though.
To test:
1. Run:
$ ktd --shell
k$ prove xt/author/valid-templates.t
=> SUCCESS: Tests pass!
2. Apply this patch
3. Repeat 1
=> SUCCESS: Tests pass! We don't need to list each individual file
anymore and it will still work!
4. Sign off :-D
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
bullseye has 2.0.0-1
bookworm has 2.2.2-1
Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
TODO: We will need to adjust this path with the db_rev one
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Adjusted path.
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
We certainly want to improve this test and run it in parallel.
Signed-off-by: Owen Leonard <oleonard@myacpl.org>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
We exclude (for now) some files that needs additional work:
1. Koha/Account/Credit.pm, Koha/Account/Debit.pm and Koha/Old/Hold.pm
"Inconsistent hierarchy during C3 merge of class"
2. misc/translator/TmplTokenizer.pm
Can't locate VerboseWarnings.pm in @INC
We also ignore some known warnings.
Signed-off-by: Emily Lamancusa <emily.lamancusa@montgomerycountymd.gov>
Signed-off-by: Martin Renvoize <martin.renvoize@openfifth.co.uk>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Baptiste Wojtkowski <baptiste.wojtkowski@biblibre.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Jenkins is not failing when git command fails.
We edited the config and it broke the git repo:
13:32:35 koha_1 | error: unable to normalize alternate object path: /srv/jenkins/workspace/Koha_Source/.git/objects
13:32:35 koha_1 | fatal: not a tree object
13:32:35 koha_1 | [12:29:31] xt/check_makefile.t
So tests were not running correctly
Maybe it will be a problem for people running the tests without being
inside a git repo, but for now we need to ensure the community CI does
what we think it does.
Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
It was not failing when warnings were found
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
podchecker is used by the QA script and catch more than Test::Pod
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
It only tested files within C4 and Koha directories.
Now we test all the perl files that are in the git index.
Test plan:
0. Do not apply those patches
1. prove xt/author/podcorrectness.t
Notice "Tests=1208" meaning that 1208 files are tested
2. Apply this patch (not the second one)
It fails, and more files are being tested (3254)
xt/author/podcorrectness.t .. 1994/3254
xt/author/podcorrectness.t .. 2726/3254 # Looks like you failed 1 test of 3254.
3. Apply the second patch
=> It passes!
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Now perltidy.t will check 2958 files instead of 2954. Which matches
with the 4 .PL files.
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
They were still using prettier directly. They should use the tidy.pl as
it is doing some wrapper work.
Signed-off-by: David Nind <david@davidnind.com>
Signed-off-by: Nick Clemens <nick@bywatersolutions.com>
Signed-off-by: Katrin Fischer <katrin.fischer@bsz-bw.de>
Jonathan Druart
Lucas Gass
Martin Renvoize
Matt Blenkinsop
Tomás Cohen Arazi
Marcel de Rooy
Katrin Fischer
Victor Grousset/tuxayo