From 12f2657633b8be62fbcfd15e661c48582606a47a Mon Sep 17 00:00:00 2001 From: "Jeroen van Meeuwen (Kolab Systems)" Date: Thu, 6 Jan 2011 17:43:16 +0000 Subject: Update Testing_Reference documentation --- Testing_Reference/en-US/Introduction.xml | 84 ++++++++++++- .../en-US/Test_Suite_000-Overview.xml | 130 +++++++++++++++++++++ Testing_Reference/en-US/Testing_Reference.xml | 3 - Testing_Reference/en-US/part-Test_Suite_000.xml | 10 +- 4 files changed, 214 insertions(+), 13 deletions(-) create mode 100644 Testing_Reference/en-US/Test_Suite_000-Overview.xml diff --git a/Testing_Reference/en-US/Introduction.xml b/Testing_Reference/en-US/Introduction.xml index c6b1bdd..20456ab 100644 --- a/Testing_Reference/en-US/Introduction.xml +++ b/Testing_Reference/en-US/Introduction.xml @@ -3,9 +3,87 @@ %BOOK_ENTITIES; ]> - - Overview of Tests + + Introduction - para + Bugzilla's Testopia extension, along with the Kolab Python utility suite, is leading in the way Kolab executes its testing. Note that there are many components that require testing, and many deployment scenarios (environments) need to be replicated for accurate representation of test results. + + Collecting those test results, and reporting based on these results, is performed through Bugzilla's Testopia extension. Reporting is important, because the premature identification of potentially problematic areas or sour spots enables us to shift attention and resources to that area and give it the appropriate amount of attention before any such problems hit any of our consumers. + +
+ Bugzilla's Testopia + + In Testopia, a test plan attached to a Bugzilla product. Each test plan contains a number of test cases, each case representing a single action to be taken of which the results can be measured. + + + These plans enable concise testing of a single product component, such as the Kontact, but of course the outcome of the tests is not dependent solely on that single product component –Kolab is a complete environment of interconnected, smaller components. As such, the test cases in each plan are ran against a build of the software being tested, said software being deployed into an environment. + + + These builds and environments, as well as the runs of test cases against said builds and environments can be organized and tracked in Testopia. + + + Bugs may be created from failed tests, and test cases may be created from bugs. + + + Actually executing each test case for each build against different environments, however, is a different story; Manually preparing the environment for a test and manually executing the test cases is inefficient, boring, prone to error and not reproducible –thus painstaking, expensive and not even proper testing. + + +
+ +
+ Kolab's PyKolab + + Amongst other things, we try to automate as much testing as possible with the Kolab Python utilities. Test plans are attached to what we call a suite –a single suite can contain multiple relevant products–, each suite containing a number of sets (plans) of tests (cases). + + + The suites are suites and not products, because a single suite may contain tests targetted at more then one product component. Noted though, most suites indeed target only one product. + + + Each "plan", or series of tests, sort of corresponds with an iteration of a reset of the environment. Suppose user/a@b.com wants an extra folder, of course we don't need to reset the complete environment. However, should user a@b.com no longer exist. Additionally, "plans" form a means of categorization. + + + Each "case", or test, represents a single step on our way to completing the report on a Testopia test case, or a fully automated series of steps completing a Testopia test case. The latter obviously assumes the former has been performed before or is trusted to work. + + +
+ +
+ Obtaining the Source Code for PyKolab + + To obtain a copy of PyKolab's source code, please execute the following: + + + +$ git clone git://git.kolab.org/git/pykolab + + + + Software Requirements + + You will need autoconf and automake installed in order to run PyKolab directly from source. No installation packages have been provided yet. + + + + + Now, navigate into the source code repository root directory, and execute the following commands: + + + +$ autoreconf -v &&./configure + + + +
+ +
+ Configuration + + Configuration is performed through files similar to conf/kolab.conf, all in .ini format. We recommend you look at conf/kolab-test-example.conf, copy it, modify it and knock yourself out against your own test environment. + + +
+ + + diff --git a/Testing_Reference/en-US/Test_Suite_000-Overview.xml b/Testing_Reference/en-US/Test_Suite_000-Overview.xml new file mode 100644 index 0000000..ae5038e --- /dev/null +++ b/Testing_Reference/en-US/Test_Suite_000-Overview.xml @@ -0,0 +1,130 @@ + + +%BOOK_ENTITIES; +]> + + Test Suite 000 - Overview + + Test Suite 000 –lovingly called "zpush"– intends to initialize, create, maintain, update and alter a Kolab groupware environment for one or more ActiveSync capable devices to synchronize against. + +
+ Test Environment Cleanup and Initialization + + The first test in a suite typically cleans up and (re-)initializes the (target) environment to form the basis for the rest of the tests to be executed against. + + + In the case of Z-Push testing, we require a couple of users with mailboxes, and so this test prepares that environment. + + + Required Configuration + + Configuration is required in order to be able to run this (re-)initialization of the environment. See the example configuration for more details. + + + + + + + + As Cyrus Administrator, deletes all mailboxes relevant to the test cases included using the configured users in testing/users. + + + + + + Creates the top-level mailboxes for the users configured in testing/users. + + + + + + User credentials are being used to create their own sub-folders hierarchy. + + + + + + + + +
+ +
+ Test 000-001: Simple, Small & Standard + + Actions: + + + Tests: Basic Kolab features (authentication, IMAP storage, etc.) and basic ActiveSync over Z-Push synchronization capabilities. + + +
+ +
+ Test 000-002: High Volume Calendaring + + para + + +
+ +
+ Test 000-003: High Volume Contacts + + para + + +
+ +
+ Test 000-004: High Volume Mail + + para + + +
+ +
+ Test 000-005: Sub-Folders and Nesting + + para + + +
+ +
+ Test 000-006: Shared Folder Namespace + + para + + +
+ +
+ Test 000-007: Shared Folders from Other Users + + para + + +
+ +
+ Test 000-008: Empty Folders + + para + + +
+ +
+ Test 000-009: No Folders + + para + + +
+ + + + diff --git a/Testing_Reference/en-US/Testing_Reference.xml b/Testing_Reference/en-US/Testing_Reference.xml index 74f2399..d388b25 100644 --- a/Testing_Reference/en-US/Testing_Reference.xml +++ b/Testing_Reference/en-US/Testing_Reference.xml @@ -6,11 +6,8 @@ - - - diff --git a/Testing_Reference/en-US/part-Test_Suite_000.xml b/Testing_Reference/en-US/part-Test_Suite_000.xml index d31d5a5..9e1b2c4 100644 --- a/Testing_Reference/en-US/part-Test_Suite_000.xml +++ b/Testing_Reference/en-US/part-Test_Suite_000.xml @@ -3,14 +3,10 @@ %BOOK_ENTITIES; ]> - + Test Suite 000 Z-Push Testing For The Masses - - Overview of Tests - - para - - + + -- cgit v0.12