Koha-clms/Koha
Jonathan Druart 38f92df4e5 Bug 14722: Refactor the export tool
Why a refactoring was need for this script?
The export tool (tools/export.pl) can be called from the command line
and some parts of code were unnecessarity complicated (just look at the
code, you will understand).

Worse still, the script does not provide the same options for both
interface. For instance you cannot export records given a range of
biblionumbers, authids, callnumbers, etc. from the commandline.

What does this patch?
1/ Important: The script tools/export.pl does not work anymore if called from
the command line (should be in the release notes).
2/ The code used to generated a file (csv, iso2709 or xml) has been moved to a new
module (Koha::Exporter::Record) and tests have been provided.
3/ No change is done on the web interface
4/ Some new options have been added to the commandline script
(misc/export.pl):
    - starting_authid
    - ending_authid
    - authtype
    - starting_biblionumber
    - ending_biblionumber
    - itemtype
    - starting_callnumber
    - ending_callnumber
    - start_accession
    - end_accession
5/ There is a change in the behavior if an error occurs:
Can't call method "as_usmarc" on an undefined value at Koha/Exporter/Record.pm line 114.
record (number 5530) is invalid and therefore not exported because its reopening generates warnings above at Koha/Exporter/Record.pm line 117.

Before this patch, they were not displayed (using the command line).

What does not do this patch?
It does not provide the 'clean', 'timestamp' and 'deleted_barcodes' options to
the web interface (same as before).

What about the perfs?
With a DB with ~800 biblios (MARC21)
Before: perl tools/export.pl 14.79s user 0.83s system 71% cpu 21.905 total
After:  perl misc/export.pl  17.19s user 0.84s system 75% cpu 24.018 total

With a DB with ~6400 biblios (UNIMARC)
Before: perl tools/export.pl 26.55s user 0.76s system 76% cpu 35.498 total
After:  perl misc/export.pl  26.78s user 0.84s system 80% cpu 34.494 total

How to test this patch?
Test plan:
A. Web interface:
1/ On the current master, export some records, biblios and authorities (with
the 3 differents exports) playing with the different filters (item type,
libraries, callnumber, accession date, don't export items, remove
non-local items, don't export fields, etc.).
2/ Apply this patch, export again the same records, and compare the
generated files. They must be identical!
3/ Confirm that the export features on the checkout list
(circ/circulation.pl) works as before this patch.

B. The command line
1/ On the current master, export some records, biblios and authorities (with
the 2 differents exports) playing with the different options (date,
deleted_barcodes, clean).
2/ Apply this patch, export again the same records, and compare the
generated files. They must be identical!

Signed-off-by: Chris Cormack <chrisc@catalyst.net.nz>

Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de>
Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io>
2015-10-27 17:01:28 -03:00
..
Acquisition Bug 13319: Fix Koha::Acq::Order should set parent_ordernumber 2015-02-10 11:24:08 -03:00
Borrower Bug 14368: Add discharges history. 2015-09-28 11:56:33 -03:00
Cache Bug 12041 - improve Koha::Cache 2014-06-19 13:05:04 -03:00
Config Bug 13967 - System preferences need a package 2015-06-04 10:39:11 -03:00
Exporter Bug 14722: Refactor the export tool 2015-10-27 17:01:28 -03:00
Filter/MARC Bug 14456: EmbedSeeFromHeadings record filter shouldn't process MARC holding fields 2015-07-24 13:39:53 -03:00
Indexer Bug 10143 - Fixing license headers 2013-05-01 06:47:50 -04:00
Item/Search Bug 11425: Add unit tests 2014-11-04 19:10:28 -03:00
List Bug 15065: (follow-up) update license header 2013-10-14 21:50:10 +00:00
Misc Bug 13813: Remove deprecated module C4::Dates from system 2015-09-18 12:40:55 -03:00
Number Bug 13836: Does not separate symbol and value if symbol is not displayed. 2015-03-19 14:10:21 -03:00
Plugins Bug 14467: Security updates break some Koha plugins 2015-07-07 14:50:43 -03:00
QueryParser/Driver Bug 12647: PQF QueryParser driver and unit tests fixes 2014-08-11 10:09:38 -03:00
RecordProcessor Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Schema/Result Bug 14246: Adding Koha::Schema::Result differences 2015-10-27 15:57:10 -03:00
Sitemapper Bug 11190: (QA followup) for tests on jenkins 2015-08-31 10:32:34 -03:00
SuggestionEngine Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Template/Plugin Bug 14100: (follow-up) Language overlay for item types 2015-10-27 12:34:06 -03:00
Util Bug 13437: Preliminary changes for marc21 plugins field 008 2015-06-10 12:51:26 -03:00
AdditionalField.pm Bug 10855: Remove additional field values when subscription is removed 2015-10-02 15:10:31 -03:00
AuthUtils.pm Bug 11944: Authentication 2015-01-13 13:07:14 -03:00
AuthorisedValue.pm Bug 10363: QA Follow-up - fixing POD and spelling 2015-10-19 12:46:27 -03:00
AuthorisedValues.pm Bug 10363: FIX regression - prevent duplicate 2015-10-19 12:46:28 -03:00
Authority.pm Bug 5202: QA follow-up - correct license and POD errors 2013-10-10 21:39:15 +00:00
Biblio.pm Bug 13851: Replace waiting holds logic in circulation.pl with Koha Objects 2015-07-29 15:47:13 -03:00
Biblioitem.pm Bug 14722: Add Koha::Biblioitem[s] 2015-10-27 16:59:52 -03:00
Biblioitems.pm Bug 14722: Add Koha::Biblioitem[s] 2015-10-27 16:59:52 -03:00
Biblios.pm Bug 13851: Replace waiting holds logic in circulation.pl with Koha Objects 2015-07-29 15:47:13 -03:00
Borrower.pm Bug 13019 - Add base classes on which to build Koha objects 2015-02-12 15:21:01 -03:00
Borrowers.pm Bug 13019 - Add base classes on which to build Koha objects 2015-02-12 15:21:01 -03:00
Branch.pm Bug 13851: Replace waiting holds logic in circulation.pl with Koha Objects 2015-07-29 15:47:13 -03:00
Branches.pm Bug 13851: Fix typo in POD 2015-07-29 15:47:47 -03:00
Cache.pm Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Calendar.pm Bug 14522: (QA followup) Remove POD and fix tests 2015-10-02 12:08:04 -03:00
Cities.pm Bug 14888: Add Koha::Cit[y|ies] classes 2015-10-05 12:00:38 -03:00
City.pm Bug 14888: Add Koha::Cit[y|ies] classes 2015-10-05 12:00:38 -03:00
Database.pm Bug 14778: Set PrintError as it was before 2015-10-23 12:01:17 -03:00
DateUtils.pm Bug 13601: The fallback regex should contain the time part 2015-03-30 13:40:07 -03:00
Email.pm Bug 9530: Adding a new Koha::Email module 2014-10-27 10:38:14 -03:00
FrameworkPlugin.pm Bug 9006: Remove AUTOLOAD in C4::Context 2015-07-20 10:11:41 -03:00
Hold.pm Bug 9809: [QA Follow-up] Remove warnings from Hold.pm 2015-08-26 10:26:49 -03:00
Holds.pm Bug 13851: Replace waiting holds logic in circulation.pl with Koha Objects 2015-07-29 15:47:13 -03:00
I18N.pm Bug 11848: (follow-up) update license statment 2014-05-05 04:51:42 +00:00
Item.pm Bug 13853: Show waiting hold expiration date for waiting holds in holds ajax datatable 2015-08-11 14:48:35 -03:00
Items.pm Bug 13851: Replace waiting holds logic in circulation.pl with Koha Objects 2015-07-29 15:47:13 -03:00
Linktracker.pm Bug 8917 : [ Sign followup ] Track clicks to links external to Koha 2013-03-07 11:14:25 -05:00
Localization.pm Bug 14100: Generic solution for language overlay - Item types 2015-10-27 12:34:06 -03:00
Localizations.pm Bug 14100: Generic solution for language overlay - Item types 2015-10-27 12:34:06 -03:00
Logger.pm Bug 14167: (QA followup) use warn instead of just STDERR 2015-07-21 10:50:17 -03:00
MetaSearcher.pm Bug 11559: Rancor: advanced cataloging interface 2015-10-27 12:17:39 -03:00
MetadataRecord.pm Bug 11559: Rancor: advanced cataloging interface 2015-10-27 12:17:39 -03:00
NorwegianPatronDB.pm Bug 14403: Remove warn in Koha::NorwegianPatronDB 2015-06-19 11:34:16 -03:00
Object.pm Bug 10363: There is no package for authorised values. 2015-10-19 12:46:27 -03:00
Objects.pm Bug 13967 - System preferences need a package 2015-06-04 10:39:11 -03:00
Plugins.pm Bug 12412: Add ability for plugins to convert arbitrary files to MARC from record staging tool 2015-05-04 15:33:51 -03:00
RecordProcessor.pm Bug 14646: Koha::RecordProcessor should accept more than one filter 2015-08-20 13:44:15 -03:00
Schema.pm Bug 11891: (follow-up) remove custom storage class 2014-06-19 11:55:03 -03:00
SimpleMARC.pm Bug 14098: Implement the copy_and_replace action for MTT 2015-09-07 11:17:13 -03:00
Sitemapper.pm Bug 11190: (QA followup) for tests on jenkins 2015-08-31 10:32:34 -03:00
SuggestionEngine.pm Bug 9978: Replace license header with the correct license (GPLv3+) 2015-04-20 09:59:38 -03:00
Upload.pm Bug 14321: [QA Follow-up] Additional QA requests 2015-09-25 12:02:52 -03:00
XSLT_Handler.pm Bug 13262 - Add parameters to XSLT Handler transform method 2015-02-05 15:24:29 -03:00