summaryrefslogtreecommitdiff
path: root/Architecture_and_Design
diff options
context:
space:
mode:
authorJeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>2011-12-12 12:30:35 (GMT)
committerJeroen van Meeuwen (Kolab Systems) <vanmeeuwen@kolabsys.com>2011-12-12 12:30:35 (GMT)
commite7c2d114b816e8e208e6c5918ba173d4ec7d4c56 (patch)
treead9d9622ae3052b01615d23da5b2cce1ad4b5beb /Architecture_and_Design
parente663ccc6babc947fd961156719bb96e7cfc325e4 (diff)
downloadkolab-docs-e7c2d114b816e8e208e6c5918ba173d4ec7d4c56.tar.gz
Add chapter on server-side Akonadi
Diffstat (limited to 'Architecture_and_Design')
-rw-r--r--Architecture_and_Design/en-US/Akonadi_for_Integration_and_Caching.xml562
1 files changed, 562 insertions, 0 deletions
diff --git a/Architecture_and_Design/en-US/Akonadi_for_Integration_and_Caching.xml b/Architecture_and_Design/en-US/Akonadi_for_Integration_and_Caching.xml
new file mode 100644
index 0000000..0c3cf09
--- /dev/null
+++ b/Architecture_and_Design/en-US/Akonadi_for_Integration_and_Caching.xml
@@ -0,0 +1,562 @@
+<?xml version='1.0' encoding='utf-8' ?>
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
+<!ENTITY % BOOK_ENTITIES SYSTEM "Architecture_and_Design.ent">
+%BOOK_ENTITIES;
+]>
+<chapter id="chap-Architecture_and_Design-Akonadi_for_Integration_and_Caching">
+ <title>Akonadi for Integration and Caching</title>
+ <para>
+ In introduction into using Akonadi as the integration and caching layer for server-side clients goes here.
+ </para>
+ <section id="sect-Architecture_and_Design-Akonadi_for_Integration_and_Caching-Server_Side_Akonadi_Architecture_Overview">
+ <title>Server-Side Akonadi Architecture Overview</title>
+ <para>
+ <xref linkend="figu-Architecture_and_Design-Server_Side_Akonadi_Architecture_Overview-Local_Akonadi_Control_Server_Architecture_Overview" /> provides an overview of a simple, single-server deployment with server-side Akonadi being used by the client applications Roundcube and ActiveSync.
+ </para>
+ <figure id="figu-Architecture_and_Design-Server_Side_Akonadi_Architecture_Overview-Local_Akonadi_Control_Server_Architecture_Overview">
+ <title>Local Akonadi Control Server Architecture Overview</title>
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="images/SSA_0a.png" format="PNG" scalefit="1" width="444" />
+ </imageobject>
+
+ </mediaobject>
+
+ </figure>
+ <para>
+ Note that the storage for the Akonadi Server is the lightest of all database technologies, SQLite. Alternatively, a SQL database server can be used, as illustrated in <xref linkend="figu-Architecture_and_Design-Server_Side_Akonadi_Architecture_Overview-Multiple_Networked_Akonadi_Control_Servers_Architecture" />. Using a SQL database server would, at the same time, allow more than one Akonadi Control Server system to be deployed as well.
+ </para>
+ <figure id="figu-Architecture_and_Design-Server_Side_Akonadi_Architecture_Overview-Local_Akonadi_Control_Server_Architecture_with_SQL_Database_Server">
+ <title>Local Akonadi Control Server Architecture with SQL Database Server</title>
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="images/SSA_0b.png" format="PNG" scalefit="1" width="444" />
+ </imageobject>
+
+ </mediaobject>
+
+ </figure>
+ <figure id="figu-Architecture_and_Design-Server_Side_Akonadi_Architecture_Overview-Networked_Akonadi_Control_Server_Architecture">
+ <title>Networked Akonadi Control Server Architecture</title>
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="images/SSA_1a.png" format="PNG" scalefit="1" width="444" />
+ </imageobject>
+
+ </mediaobject>
+
+ </figure>
+ <figure id="figu-Architecture_and_Design-Server_Side_Akonadi_Architecture_Overview-Multiple_Networked_Akonadi_Control_Servers_Architecture">
+ <title>Multiple Networked Akonadi Control Servers Architecture</title>
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="images/SSA_2a.png" format="PNG" scalefit="1" width="444" />
+ </imageobject>
+
+ </mediaobject>
+
+ </figure>
+
+ </section>
+
+ <section id="sect-Architecture_and_Design-Akonadi_for_Integration_and_Caching-Procedural_Descriptions">
+ <title>Procedural Descriptions</title>
+ <procedure id="proc-Architecture_and_Design-Procedural_Descriptions-Client_Connections_to_Akonadi">
+ <title>Client Connections to Akonadi</title>
+ <step>
+ <para>
+ The client (Roundcube, ActiveSync) calls <application>libakonadi</application> API <code>akonadi_open(</code><emphasis>username</emphasis><code>, </code><emphasis>password</emphasis><code>[, authorize_as=</code><emphasis>username</emphasis><code>][, imap_host=</code><emphasis>imap_host</emphasis><code>][, akonadi_session_id=</code><emphasis>akonadi_session_id</emphasis><code>])</code>.
+ </para>
+
+ </step>
+ <step>
+ <para>
+ On behalf of the client application, <application>libakonadi</application> establishes a connection to;
+ </para>
+ <para>
+ <orderedlist>
+ <listitem>
+ <para>
+ The configured or discovered Akonadi Control Server, through a configured mechanism (AF_UNIX, AF_INET, AMQP MessageBus obtained location).
+ </para>
+ <para>
+ See also: <xref linkend="proc-Architecture_and_Design-Procedural_Descriptions-libakonadi_Connections_to_Akonadi" />.
+ </para>
+
+ </listitem>
+ <listitem>
+ <para>
+ The requested IMAP Server, if any.
+ </para>
+
+ </listitem>
+
+ </orderedlist>
+
+ </para>
+
+ </step>
+ <step>
+ <para>
+ The call to <code>akonadi_open()</code> may be synchronous or asynchronous.
+ </para>
+ <para>
+ For synchronous calls (using <code>akonadi_open_s()</code>), <application>libakonadi</application> refrains from returning a link identifier until after;
+ </para>
+ <para>
+ <orderedlist>
+ <listitem>
+ <para>
+ It has fully established a successfully authenticated connection to the Akonadi Control Server, and the Akonadi Control Server has returned the Akonadi Session Identifier.
+ </para>
+
+ </listitem>
+ <listitem>
+ <para>
+ It has fully established a successfully authenticated connection to the IMAP server, if any such had been requested.
+ </para>
+
+ </listitem>
+
+ </orderedlist>
+
+ </para>
+ <para>
+ For asynchronous calls (using <code>akonadi_open()</code>), <application>libakonadi</application> immediately returns a link identifier. The connections to both the Akonadi Control Server and IMAP Server will be established, but will block subsequent access to synchronous calls, and/or queue subsequent asynchronous calls.
+ </para>
+
+ </step>
+ <step>
+ <para>
+ The call returns a link identifier.
+ </para>
+
+ </step>
+ <step>
+ <para>
+ Once the connection is established, authenticated successfully and has obtained a Session Identifier from the Akonadi Control Server, the session identifier is to be made available through API call:
+ </para>
+ <para>
+ <code>akonadi_session_id(</code><emphasis>link identifier</emphasis><code>)</code>
+ </para>
+
+ </step>
+
+ </procedure>
+
+ <procedure id="proc-Architecture_and_Design-Procedural_Descriptions-libakonadi_Connections_to_Akonadi">
+ <title><application>libakonadi</application> Connections to Akonadi</title>
+ <step>
+ <para>
+ <application>libakonadi</application>, using the credentials supplied to it through the <code>akonadi_open()</code> call, authenticates itself to the Akonadi Control Server (ACS) using one of the credential exchange mechanisms advertised by the ACS in the initial greeting, such as PLAIN, LOGIN, CRAM-MD5, DIGEST-MD5.
+ </para>
+ <para>
+ <remark>Note that we may have to consider the authentication mechanism allowed by the Akonadi Control Server to include GSSAPI and SSL certificate based authentication as well. </remark>
+
+ </para>
+
+ </step>
+ <step>
+ <para>
+ After the connection has been successfully authenticated (and authorized, where appropriate), the ACS proxies through the connection to the Akonadi Server associated with the Akonadi Session.
+ </para>
+
+ </step>
+
+ </procedure>
+
+ <procedure id="proc-Architecture_and_Design-Procedural_Descriptions-Akonadi_Control_Server_Connection_Handling">
+ <title>Akonadi Control Server Connection Handling</title>
+ <step>
+ <para>
+ Upon receiving a connection, the ACS identifies itself in a greeting (server_name) and includes its capabilities.
+ </para>
+
+ </step>
+ <step>
+ <para>
+ Upon receiving an authentication request, should the authentication be successful, the ACS proxies through the connection to the Akonadi Server associated with the user account the connection is to be authorized as, or follows the following startup procedure;
+ </para>
+ <procedure id="proc-Architecture_and_Design-Akonadi_Control_Server_Connection_Handling-ACS_Akonadi_Server_and_Agent_Startup">
+ <title>ACS Akonadi Server and Agent Startup</title>
+ <step>
+ <para>
+ The client application user is associated with an Akonadi Session Identifier.
+ </para>
+ <para>
+ It is NOT the client application session that is associated with an Akonadi Server.
+ </para>
+
+ </step>
+ <step>
+ <para>
+ For each Akonadi Server and Akonadi Agent the ACS starts up, a public and private key-pair is generated and along with the Akonadi Session Identifier, handed over to the Akonadi Server or Agent.
+ </para>
+
+ </step>
+ <step>
+ <para>
+ The private key is forgotten.
+ </para>
+
+ </step>
+ <step>
+ <para>
+ The public key is maintained in a Akonadi Session Identifier.
+ </para>
+
+ </step>
+ <step>
+ <para>
+ If successful, the Akonadi Session Identifier is passed back to the client (<application>libakonadi</application>).
+ </para>
+
+ </step>
+ <step>
+ <para>
+ After initial discovery (IMAP Configuration folder), additional Akonadi Agents may need to be started up. For this procedure, please see
+ </para>
+
+ </step>
+
+ </procedure>
+
+
+ </step>
+
+ </procedure>
+
+ <procedure id="proc-Architecture_and_Design-Procedural_Descriptions-Starting_up_Additional_Akonadi_Agents">
+ <title>Starting up Additional Akonadi Agents</title>
+ <step>
+ <para>
+ The following situations may cause a new Akonadi Agent to need to be started:
+ </para>
+ <para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ An Akonadi Agent has died unexpectedly.
+ </para>
+
+ </listitem>
+ <listitem>
+ <para>
+ Discovery of an IMAP Configuration folder with Kolab XML Format Configuration of secondary accounts requires new agents in order to handle said accounts (i.e. Twitter / Facebook / Gmail / ...).
+ </para>
+
+ </listitem>
+ <listitem>
+ <para>
+ Any other case.
+ </para>
+
+ </listitem>
+
+ </itemizedlist>
+
+ </para>
+ <para>
+ Additionally, the exchange of the original user credentials may be needed in the following scenario:
+ </para>
+ <para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ An Akonadi Agent has lost it's connection and no credentials had been cached, causing the agent to be unable to re-establish the connection.
+ </para>
+
+ </listitem>
+
+ </itemizedlist>
+
+ </para>
+
+ </step>
+ <step>
+ <para>
+ The exchange of user credentials happens over D-Bus.
+ </para>
+
+ </step>
+ <step>
+ <para>
+ A request for the user credentials corresponding to the agent is made in a signed and encrypted fashion, sending the signed/encrypted payload containing the overall Akonadi Session Identifier over a D-Bus signal.
+ </para>
+
+ </step>
+ <step>
+ <para>
+ The Akonadi Control Server responds with a signed/encrypted D-Bus signal containing the user credentials.
+ </para>
+
+ </step>
+
+ </procedure>
+
+ <!-- <procedure>
+ <step>
+ <para>
+ The client identifies itself using a token that is a fully qualified, globally unique user identifier, plus a session-specific token, such as a PHP <code>$_SESSION['id']</code>.
+ </para>
+ <para>
+ The user identifier cannot be, for example, 'jdoe', as it is unqualified, and cannot be 'www01.example.org' as it is not tied to the user - it is recommended to use the primary mail address, such as 'jdoe@example.org'.
+ </para>
+ <para>
+ <remark> The aforementioned would require proper canonification to happen at the client application level, in order to allow users to login with their uid/mail/alias/mailAlternateAddress fully qualified or unqualified user identifier. </remark>
+
+ </para>
+ <para>
+ <remark>Also note that 'jdoe@example.org' or someone's primary email address is not necessarily persistent. When Jane.Gi@example.org marries John Doe, her primary email address may change to Jane.Doe@example.org, inherently invalidating the existing copy of one's cache; There are potentially persistent unique attributes out there such as <code>nsuniqueid</code> and SQL's 'user.id' if you will. This would require the client applications to be able to retrieve that information, however. </remark>
+
+ </para>
+ <para>
+ The Session Idenfifier is added to the initial greeting in order to allow the ACS to recognize the client application is a formerly authenticated and authorized client session. If this would not be included, client #2 (ActiveSync) could be associated with client #1's (Roundcube) Akonadi Server without intermediate authentication/authorization/verification.
+ </para>
+
+ </step>
+ <step>
+ <para>
+ For the ASS process steps upon having received a greeting from the client identifying itself as one of potentially multiple sessions associated with a user, see <xref linkend="proc-Architecture_and_Design-Akonadi_for_Integration_and_Caching-Akonadi_Control_Server_Client_Connection_Handling" />.
+ </para>
+
+ </step>
+ <step>
+ <para>
+ The ASS returns a generated token to the client which the client MUST store and MUST supply with subsequent connection requests.
+ </para>
+ <para>
+ This token is an Akonadi Session Identifier.
+ </para>
+
+ </step>
+ <step>
+ <para>
+ User credentials exchange.
+ </para>
+ <para>
+ <remark> Plaintext submission of user credentials should not be allowed, unless using local sockets (i.e. AF_UNIX). Plaintext submission of the user credentials is only allowed under explicit TLS, or implicit SSL. </remark>
+
+ </para>
+
+ </step>
+ <step>
+ <para>
+ The ASS spawns an Akonadi Server (AS), with a time of anticipated shutdown set 30 minutes from now, and proxies the client connection through to the socket created by the corresponding AS.
+ </para>
+
+ </step>
+ <step>
+ <para>
+ The ASS supplies a plaintext version of the user credentials to the AS, of which the password MAY NOT be stored in plain text.
+ </para>
+
+ </step>
+ <step>
+ <para>
+ Upon the return of the greeting containing the encrypted version of the user password, the client MUST verify the ASS token
+ </para>
+
+ </step>
+
+ </procedure>
+
+ <procedure id="proc-Architecture_and_Design-Akonadi_for_Integration_and_Caching-Akonadi_Control_Server_Client_Connection_Handling">
+ <title>Akonadi Control Server Client Connection Handling</title>
+ <step>
+ <para>
+ Upon receiving the user and session identifiers, the ACS attempts to find an existing Akonadi Server associated with the user identifier using an internal table storing;
+ </para>
+ <para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ The user identifier
+ </para>
+
+ </listitem>
+ <listitem>
+ <para>
+ The user password
+ </para>
+
+ </listitem>
+ <listitem>
+ <para>
+ The D-Bus session ID
+ </para>
+
+ </listitem>
+ <listitem>
+ <para>
+ One or more client application session IDs.
+ </para>
+
+ </listitem>
+ <listitem>
+ <para>
+ Public encryption keys to the Akonadi Server and one or more Akonadi Agent(s) for encrypted D-Bus payload.
+ </para>
+
+ </listitem>
+
+ </itemizedlist>
+
+ </para>
+
+ </step>
+ <step>
+ <para>
+ If no existing Akonadi Server exists, the ACS generates an asynchronous encryption key-pair and spawns the Akonadi Server with it, forgetting the private key, and repeats this exercise for all of one or more Akonadi Agent(s).
+ </para>
+
+ </step>
+
+ </procedure> -->
+ </section>
+
+ <section id="sect-Architecture_and_Design-Akonadi_for_Integration_and_Caching-Notes_from_Design_Sessions">
+ <title>Notes from Design Sessions</title>
+ <section id="sect-Architecture_and_Design-Notes_from_Design_Sessions-Authentication_to_Akonadi_Control_Server">
+ <title>Authentication to Akonadi Control Server</title>
+ <para>
+ Authenticating to the Akonadi Control Server for each connection created by or on behalf of a client application is deemed to be the most secure, effective and standardized mechanism to establish session reliability, proxy authorization and the ability to have one client application use one set of credentials for a user over another set of credentials for a user.
+ </para>
+ <para>
+ Note that the client library, <application>libakonadi</application> needs to obtain the user credentials each time a new connection is requested, as a future direct IMAP connection can be used for interactions that are deemed more efficient to execute against the IMAP server directly, such as simple LIST and FETCH commands &ndash;for which libakonadi would be the intelligent broker.
+ </para>
+ <para>
+ A typical scenario for one user being able to use two or more different sets of credentials includes secondary passwords for mobile devices using ActiveSync, so loss of corporate credentials is less likely to occur.
+ </para>
+ <para>
+ Note that we are otherwise unlikely to be able to securely associate user session #1 in client application $x with user session #2 in client application $y passing both through to the same Akonadi Server.
+ </para>
+
+ </section>
+
+ <section id="sect-Architecture_and_Design-Notes_from_Design_Sessions-AMQP_Capabilities_for_libakonadi">
+ <title>AMQP Capabilities for <application>libakonadi</application></title>
+ <para>
+ AMQP capabilities for <application>libakonadi</application> would enable <application>libakonadi</application> to discover the Akonadi Control Server to use for the new or existing session, enabling one or more Akonadi Control Servers to be deployed with Kolab Groupware.
+ </para>
+ <para>
+ The underlying assumption is scaling up the concept of Akonadi Control Servers as the integration and caching layer for client applications will ultimately cause more than one Akonadi Control Server system to need to be added to the deployment.
+ </para>
+
+ </section>
+
+ <section id="sect-Architecture_and_Design-Notes_from_Design_Sessions-AMQP_Capabilities_for_Akonadi">
+ <title>AMQP Capabilities for Akonadi</title>
+ <para>
+ AMQP capabilities for Akonadi in general would enable IMAP to push change notifications to Akonadi services including <application>libakonadi</application> even if that means the client library can only pick them up during the first interaction against the client application. Such notifications could include;
+ </para>
+ <para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ Folder change notifications (new calendar folder, folder dissappeared)
+ </para>
+
+ </listitem>
+ <listitem>
+ <para>
+ New and/or updated message content (update cache, Free/Busy, or more in general, act accordingly)
+ </para>
+
+ </listitem>
+
+ </itemizedlist>
+
+ </para>
+ <para>
+ These notifications (from Cyrus IMAP) and notifications from Akonadi itself, can remain pending on the messagebus for <application>libakonadi</application> to pick up when users start interacting with client applications.
+ </para>
+ <para>
+ Using a push notification mechanism like this may prove crucial for ActiveSync PUSH mode, or continuously operating web application(s) (PHP FPM, FastCGI) with clients connected through <ulink url="http://en.wikipedia.org/wiki/WebSocket">web sockets</ulink>.
+ </para>
+
+ </section>
+
+ <section id="sect-Architecture_and_Design-Notes_from_Design_Sessions-libakonadi_Direct_IMAP_Connections">
+ <title><application>libakonadi</application> Direct IMAP Connections</title>
+ <para>
+ In scenarios where de-duplication of data is important (i.e. many users, large mailboxes, etc.), the Akonadi Server and Agents would be configured to only obtain and insert into cache relevant metadata about folders and messages.
+ </para>
+ <para>
+ It is therefor assumed that, for example, a FETCH of a message is more efficiently executed against IMAP directly, and only complex queries that cannot be answered by IMAP servers efficiently are directed at the Akonadi Server.
+ </para>
+ <para>
+ Queries that do not need to be executed against an Akonadi Server but are very easily and very efficiently answered by an IMAP server are, for example:
+ </para>
+ <para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ <command>LIST</command>
+ </para>
+
+ </listitem>
+ <listitem>
+ <para>
+ <command>LSUB</command>
+ </para>
+
+ </listitem>
+ <listitem>
+ <para>
+ <command>FETCH</command>
+ </para>
+
+ </listitem>
+
+ </itemizedlist>
+
+ </para>
+ <para>
+ The data for the aforelisted commands, which may be presumed to be among the largest amounts of data, does not need to descend client -&gt; ACS -&gt; AS (-&gt; Cache -&gt; AS) -&gt; AA -&gt; IMAP -&gt; AA -&gt; AS (-&gt; Cache -&gt; AS) -&gt; client.
+ </para>
+ <para>
+ Commands that are more complex for the IMAP server to answer because of databases maintained on the IMAP server are less ideally suited to answer the command quickly, could include;
+ </para>
+ <para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ <command>THREAD ALL UNDELETED</command>
+ </para>
+
+ </listitem>
+
+ </itemizedlist>
+
+ </para>
+ <para>
+ Commands that are complex for the client to compile an answer for because they would require a number of iterations in order to successfully compile the answer, could include;
+ </para>
+ <para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ All calendar items for month $x or weeek $y.
+ </para>
+
+ </listitem>
+ <listitem>
+ <para>
+ A list of all calendar folders (the user has write access to).
+ </para>
+
+ </listitem>
+
+ </itemizedlist>
+
+ </para>
+
+ </section>
+
+
+ </section>
+
+
+</chapter>
+