summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorJeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>2011-01-06 17:43:16 (GMT)
committerJeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>2011-01-06 17:43:16 (GMT)
commit12f2657633b8be62fbcfd15e661c48582606a47a (patch)
tree38d7abe90b19f583ac54dd55ce6df961f2cb122d
parent9b212357203591f1feb113c9c89ee4cef353146a (diff)
downloadkolab-docs-12f2657633b8be62fbcfd15e661c48582606a47a.tar.gz
Update Testing_Reference documentation
-rw-r--r--Testing_Reference/en-US/Introduction.xml84
-rw-r--r--Testing_Reference/en-US/Test_Suite_000-Overview.xml130
-rw-r--r--Testing_Reference/en-US/Testing_Reference.xml3
-rw-r--r--Testing_Reference/en-US/part-Test_Suite_000.xml10
4 files changed, 214 insertions, 13 deletions
diff --git a/Testing_Reference/en-US/Introduction.xml b/Testing_Reference/en-US/Introduction.xml
index c6b1bdd..20456ab 100644
--- a/Testing_Reference/en-US/Introduction.xml
+++ b/Testing_Reference/en-US/Introduction.xml
@@ -3,9 +3,87 @@
<!ENTITY % BOOK_ENTITIES SYSTEM "Testing_Reference.ent">
%BOOK_ENTITIES;
]>
-<chapter>
- <title>Overview of Tests</title>
+<chapter id="chap-Testing_Reference-Introduction">
+ <title>Introduction</title>
<para>
- para
+ Bugzilla's Testopia extension, along with the Kolab Python utility suite, is leading in the way Kolab executes its testing. Note that there are many components that require testing, and many deployment scenarios (<emphasis>environments</emphasis>) need to be replicated for accurate representation of test results.
</para>
+ <para>
+ Collecting those test results, and reporting based on these results, is performed through Bugzilla's Testopia extension. Reporting is important, because the premature identification of potentially problematic areas or sour spots enables us to shift attention and resources to that area and give it the appropriate amount of attention before any such problems hit any of our consumers.
+ </para>
+ <section id="sect-Testing_Reference-Introduction-Bugzillas_Testopia">
+ <title>Bugzilla's Testopia</title>
+ <para>
+ In Testopia, a <emphasis>test plan</emphasis> attached to a Bugzilla product. Each test plan contains a number of <emphasis>test cases</emphasis>, each case representing a single action to be taken of which the results can be measured.
+ </para>
+ <para>
+ These plans enable concise testing of a single product component, such as the <application>Kontact</application>, but of course the outcome of the tests is not dependent solely on that single product component &ndash;Kolab is a complete environment of interconnected, smaller components. As such, the test cases in each plan are ran against a <emphasis>build</emphasis> of the software being tested, said software being deployed into an <emphasis>environment</emphasis>.
+ </para>
+ <para>
+ These builds and environments, as well as the runs of test cases against said builds and environments can be organized and tracked in Testopia.
+ </para>
+ <para>
+ Bugs may be created from failed tests, and test cases may be created from bugs.
+ </para>
+ <para>
+ Actually executing each test case for each build against different environments, however, is a different story; Manually preparing the environment for a test and manually executing the test cases is inefficient, boring, prone to error and not reproducible &ndash;thus painstaking, expensive and not even proper testing.
+ </para>
+
+ </section>
+
+ <section id="sect-Testing_Reference-Introduction-Kolabs_PyKolab">
+ <title>Kolab's PyKolab</title>
+ <para>
+ Amongst other things, we try to automate as much testing as possible with the Kolab Python utilities. Test plans are attached to what we call a <emphasis>suite</emphasis> &ndash;a single suite can contain multiple relevant products&ndash;, each suite containing a number of sets (plans) of tests (cases).
+ </para>
+ <para>
+ The suites are suites and not products, because a single suite may contain tests targetted at more then one product component. Noted though, most suites indeed target only one product.
+ </para>
+ <para>
+ Each "plan", or series of tests, sort of corresponds with an iteration of a reset of the environment. Suppose user/a@b.com wants an extra folder, of course we don't need to reset the complete environment. However, should user a@b.com no longer exist. Additionally, "plans" form a means of categorization.
+ </para>
+ <para>
+ Each "case", or test, represents a single step on our way to completing the report on a Testopia test case, or a fully automated series of steps completing a Testopia test case. The latter obviously assumes the former has been performed before or is trusted to work.
+ </para>
+
+ </section>
+
+ <section id="sect-Testing_Reference-Introduction-Obtaining_the_Source_Code_for_PyKolab">
+ <title>Obtaining the Source Code for PyKolab</title>
+ <para>
+ To obtain a copy of PyKolab's source code, please execute the following:
+ </para>
+ <para>
+
+<screen>$ <userinput>git clone git://git.kolab.org/git/pykolab</userinput></screen>
+
+ </para>
+ <note>
+ <title>Software Requirements</title>
+ <para>
+ You will need <application>autoconf</application> and <application>automake</application> installed in order to run PyKolab directly from source. No installation packages have been provided yet.
+ </para>
+
+ </note>
+ <para>
+ Now, navigate into the source code repository root directory, and execute the following commands:
+ </para>
+ <para>
+
+<screen>$ <userinput>autoreconf -v &amp;&amp;./configure</userinput></screen>
+
+ </para>
+
+ </section>
+
+ <section id="sect-Testing_Reference-Introduction-Configuration">
+ <title>Configuration</title>
+ <para>
+ Configuration is performed through files similar to <filename>conf/kolab.conf</filename>, all in .ini format. We recommend you look at <ulink url="http://git.kolab.org/pykolab/tree/conf/kolab-test-example.conf"><filename>conf/kolab-test-example.conf</filename></ulink>, copy it, modify it and knock yourself out against your own test environment.
+ </para>
+
+ </section>
+
+
</chapter>
+
diff --git a/Testing_Reference/en-US/Test_Suite_000-Overview.xml b/Testing_Reference/en-US/Test_Suite_000-Overview.xml
new file mode 100644
index 0000000..ae5038e
--- /dev/null
+++ b/Testing_Reference/en-US/Test_Suite_000-Overview.xml
@@ -0,0 +1,130 @@
+<?xml version='1.0' encoding='utf-8' ?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
+<!ENTITY % BOOK_ENTITIES SYSTEM "Testing_Reference.ent">
+%BOOK_ENTITIES;
+]>
+<chapter id="chap-Testing_Reference-Test_Suite_000_Overview">
+ <title>Test Suite 000 - Overview</title>
+ <para>
+ Test Suite 000 &ndash;lovingly called "<literal>zpush</literal>"&ndash; intends to initialize, create, maintain, update and alter a Kolab groupware environment for one or more ActiveSync capable devices to synchronize against.
+ </para>
+ <section id="sect-Testing_Reference-Test_Suite_000_Overview-Test_Environment_Cleanup_and_Initialization">
+ <title>Test Environment Cleanup and Initialization</title>
+ <para>
+ The first test in a suite typically cleans up and (re-)initializes the (target) environment to form the basis for the rest of the tests to be executed against.
+ </para>
+ <para>
+ In the case of Z-Push testing, we require a couple of users with mailboxes, and so this test prepares that environment.
+ </para>
+ <formalpara id="form-Testing_Reference-Test_Environment_Cleanup_and_Initialization-Required_Configuration">
+ <title>Required Configuration</title>
+ <para>
+ Configuration is required in order to be able to run this (re-)initialization of the environment. See <ulink url="http://git.kolab.org/pykolab/tree/conf/kolab-test-example.conf">the example configuration</ulink> for more details.
+ </para>
+
+ </formalpara>
+ <para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ As Cyrus Administrator, deletes all mailboxes relevant to the test cases included using the configured users in <literal>testing/users</literal>.
+ </para>
+
+ </listitem>
+ <listitem>
+ <para>
+ Creates the top-level mailboxes for the users configured in <literal>testing/users</literal>.
+ </para>
+
+ </listitem>
+ <listitem>
+ <para>
+ User credentials are being used to create their own sub-folders hierarchy.
+ </para>
+
+ </listitem>
+
+ </itemizedlist>
+
+ </para>
+
+ </section>
+
+ <section id="sect-Testing_Reference-Test_Suite_000_Overview-Test_000_001_Simple_Small_Standard">
+ <title>Test 000-001: Simple, Small &amp; Standard</title>
+ <para>
+ Actions:
+ </para>
+ <para>
+ Tests: Basic Kolab features (authentication, IMAP storage, etc.) and basic ActiveSync over Z-Push synchronization capabilities.
+ </para>
+
+ </section>
+
+ <section id="sect-Testing_Reference-Test_Suite_000_Overview-Test_000_002_High_Volume_Calendaring">
+ <title>Test 000-002: High Volume Calendaring</title>
+ <para>
+ para
+ </para>
+
+ </section>
+
+ <section id="sect-Testing_Reference-Test_Suite_000_Overview-Test_000_003_High_Volume_Contacts">
+ <title>Test 000-003: High Volume Contacts</title>
+ <para>
+ para
+ </para>
+
+ </section>
+
+ <section id="sect-Testing_Reference-Test_Suite_000_Overview-Test_000_004_High_Volume_Mail">
+ <title>Test 000-004: High Volume Mail</title>
+ <para>
+ para
+ </para>
+
+ </section>
+
+ <section id="sect-Testing_Reference-Test_Suite_000_Overview-Test_000_005_Sub_Folders_and_Nesting">
+ <title>Test 000-005: Sub-Folders and Nesting</title>
+ <para>
+ para
+ </para>
+
+ </section>
+
+ <section id="sect-Testing_Reference-Test_Suite_000_Overview-Test_000_006_Shared_Folder_Namespace">
+ <title>Test 000-006: Shared Folder Namespace</title>
+ <para>
+ para
+ </para>
+
+ </section>
+
+ <section id="sect-Testing_Reference-Test_Suite_000_Overview-Test_000_007_Shared_Folders_from_Other_Users">
+ <title>Test 000-007: Shared Folders from Other Users</title>
+ <para>
+ para
+ </para>
+
+ </section>
+
+ <section id="sect-Testing_Reference-Test_Suite_000_Overview-Test_000_008_Empty_Folders">
+ <title>Test 000-008: Empty Folders</title>
+ <para>
+ para
+ </para>
+
+ </section>
+
+ <section id="sect-Testing_Reference-Test_Suite_000_Overview-Test_000_009_No_Folders">
+ <title>Test 000-009: No Folders</title>
+ <para>
+ para
+ </para>
+
+ </section>
+
+
+</chapter>
+
diff --git a/Testing_Reference/en-US/Testing_Reference.xml b/Testing_Reference/en-US/Testing_Reference.xml
index 74f2399..d388b25 100644
--- a/Testing_Reference/en-US/Testing_Reference.xml
+++ b/Testing_Reference/en-US/Testing_Reference.xml
@@ -6,11 +6,8 @@
<book>
<xi:include href="Book_Info.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
<xi:include href="Preface.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
-
<xi:include href="Introduction.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
-
<xi:include href="part-Test_Suite_000.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
-
<xi:include href="Revision_History.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
<index />
</book>
diff --git a/Testing_Reference/en-US/part-Test_Suite_000.xml b/Testing_Reference/en-US/part-Test_Suite_000.xml
index d31d5a5..9e1b2c4 100644
--- a/Testing_Reference/en-US/part-Test_Suite_000.xml
+++ b/Testing_Reference/en-US/part-Test_Suite_000.xml
@@ -3,14 +3,10 @@
<!ENTITY % BOOK_ENTITIES SYSTEM "Testing_Reference.ent">
%BOOK_ENTITIES;
]>
-<part>
+<part id="part-Testing_Reference-Test_Suite_000">
<title>Test Suite 000</title>
<subtitle>Z-Push Testing For The Masses</subtitle>
- <chapter>
- <title>Overview of Tests</title>
- <para>
- para
- </para>
- </chapter>
+ <xi:include href="Test_Suite_000-Overview.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
+
</part>