Friday, June 14, 2013

Dynamic web pages with Python

I just got a Raspberry Pi. This small box has just 700Mhz CPU speed and 512 MB main memory, so you can not expect to be able to run a full featured Enterprise Java Application on it. The Raspian Linux for the Pi is very Python focused and I think there are good reasons for that. Python is object oriented, easy to learn and very flexible and there are a lot of libraries and frameworks around it. This includes for instance Template Engines or Database Access. As a 'light' scripting language it seems also to be less resource consuming. So at least I like Python. So my first Raspberry project was to install an Apache Web Server and the Python Module in order to try out to develop dynamic web sites based on the Raspberry platform. First to say: the setup is very easy. Python comes with it's own distribution system (so with a kind of package manager) which allows you to install everything you need in a few minutes. For things like the Apache Web Server the Debian based package management (apt) should be usded:

sudo apt-get install apache2
sudo apt-get install libapache2-mod-python
sudo apt-get install python-pip (This installs the Python Package Index)
pip install Jinja2 (This installs the Template Engine Jinja2)

All you need to do to enable your Apache for Python is to edit the configuration file /etc/apache2/sites-available/default by adding the following handler definition to the <Directory /var/www> - section:

 AddHandler mod_python .py
 PythonHandler mod_python.publisher
 PythonDebug On

Now create a folder which is named 'templates' under /var/www . All your templates have to live inside this folder because we are using a File System Template Loader. Inside the template folder the hello.html fille was created for demonstration purposes:

   <html> Hello {{name}} ! </html>

As you can see there is one placeholder 'name'. Now we want to create Python code which uses the template. The following code example explains it:

###
# Apache could be used together with python. To have a clean
# seperation between the script code behind and the html view
# a template engine can be used. The following controller script
# shows how to render a web page based on a model and an HTML
# template
###

#Import the Jinja2 template engine
from jinja2 import Environment, FileSystemLoader

# Function to handle the reqest
def index(req):

  # Set the environment by using a file system loader
  env = Environment(loader = FileSystemLoader('/var/www/templates'))

  # Get the template hello.html
  template = env.get_template('hello.html')

  # Return the rendered result
  return template.render(name='Your Name')

Your rendered output is now accessible via http://yourhost/hello.py and should look as the following one:

Hello Your Name !





Wednesday, March 6, 2013

Alfresco's Auditing System

Preamble

I just had to think about the monitoring of Alfresco. Things like 'Which user logged in how often' or 'Which document was opened how often' were required. My first idea was to develop the following system:
  • Alfresco Share does communicate with the Repository layer via Web Scripts. So every action should cause an HTTP request.
  • A proxy in front of the Alfresco repository which filters requests those are relevant  (E.G. the AuthenticationService)
  • Each matching request should be logged to a database. So the database contains just the HTTP request text and the request parameters.
  • Then it is possible to use the request log to create specific reports
By further investigating the requirements the question was raised if Alfresco not yet has such a functionality. This is the reason why this article spots some light on the Alfresco Auditing feature. From the first view, this feature includes the previous mentioned idea. I can define Extractors and Generators based on a by path filtering (whereby the path seems to refer to the RESTful service which is used). Further investigations may answer the question if the auditing is suitable to fit the above mentioned requirements.
  • Auditing needs to be enabled.
  • Configure filters.
  • There are DataProducers, DataExtractors and DataGenerators
  • It is possible to define custom AuditApplications
  • The Auditservice is used to retrieve the audit data.
  • The RecordValue element depends on a  DataExtractor by specifying which data trigger and which data source to use
So now let's try to get behind it

Configuration
  • To enable auditing you can set 'audit..enabled=true' in the alfresco-global.properties file. The web script under '/api/audit/control' then gives you further details about the state. To enable specific audit applications (see section below) you can set 'audit.${application id}.enabled = true'.
  • Set logging in audit-log.properties: org.alfresco.repo.audit.AuditComponentImpl=DEBUG
Filters

Filters are applied to events. The DataProducer is identified by a root path. A DataProducer calls a recordAuditValues method which uses the root path and a audit map. The map contains the information which is relevant for auditing purposes. So if the root path is "/alfresco-access/transactio" then the map contains the values 'action' (E.G. MOVE), 'node' (The target node of the action), 'move/from/node', 'move/to/node', 'sub-actions' and so on.

It is now possible to define filters in the alfresco configuration file. The format is:
  • audit.filter.${application part of the root path}.${sub path of the root path}.${property in audit map} = ${; seperated list of regular expressions for values those should match}
So an example is to audit every log-ins of the user jblogs and every user who has an user id which begins with 'd'.
  •  audit.filter.alfresco-access.login.user=jblogs; d.*
Additionally it's required to enable the audting for a specifc filter
  • audit.filter.alfresco-access.login.enabled=true
DataProducers

There are several data producers out of the box available. The documentation says that the 'org.alfresco.repo.audit.access.AccessAuditor' does not resolve any event in detail (preview and download is one single event) whereby the 'AuditMethodInterceptor' producer records seperated events. There is property in the configuration 'audit.alfresco-access.sub-actions.enabled' which seems to be used to tell Alfresco which DataProducer should be used.

DataGenerators

A data generator produces output without any input. So data is produced when a specifc path is set as active. The AuthenticatedUserDataGenerator generates data as soon as a person gets authenticated. So the data generator is responsible for generating data dependent on specific events. Such a generator is not the same a DataProducer. It seems that a producer is used to implement the 'Which events should produce data?' and the DataGenerator is used to implement the 'Which data should generated?'.

A data generator has a registered name or fully qualified class name. In the first case you can access it via the Spring bean id (audit-services-context.xml) in the last case you can reference it via its class name directly. (class or registeredName properties if defining them in the application configuration)

The documentation says that the 'AccessAuditor' generator writes entries like:

${application part of the root path}.${sub path of the root path}.${property in audit map}= ${value in audit map}.
 
DataExtractors

It is a component which uses input data to produce some output. As a DataGenerator you can define it in your application configuration by using its registered name or fully qualified class name. Alfresco provides the SimpleValueExtractor (org.alfresco.repo.audit.extractor.SimpleValueDataExtractor). This default extractor just returns the input without any transofrmation. Another examle is the NodeNameDataExtractor which is able to extract the cm:name value of a node. So in summary the extractor is used to implement the 'How to store the previously generated data?'

Path mappings

We already mentioned the root path and we also know that our audit entry map contains paths as part of the keys. The path mapping can be used to rewrite these paths. So let's assume you want '/ecgaudit/login' as the path in you entry map instead '/alfresco-api/post/AuthenticationService/authenticate' then you can define the following path mapping:

<PathMappings>
  <PathMap source="/alfresco-api/post/AuthenticationService/authenticate" target="/ecgaudit/login"
</PathMappings>

To following the path conventions, please keep in mind that the first part of the path is the application id.

Audit Applications

How exactly auditing behaves depends on the audit application. There is one application provided by Alfresco which is named 'alfresco-access'.

You can add new audit application configurations to <tomcat>/shared/classes/alfresco/extension/audit directory. Just create an XML file ${application id}.xml inside it.

Here an example application from the Alfresco Wiki:

  <Application name="DOD5015" key="DOD5015">
        <AuditPath key="login">
            <AuditPath key="args">
                <AuditPath key="userName">
                    <RecordValue key="value" dataExtractor="simpleValue"/>
                </AuditPath>
            </AuditPath>
            <AuditPath key="no-error">
                <GenerateValue key="fullName" dataGenerator="personFullName"/>
            </AuditPath>
            <AuditPath key="error">
                <RecordValue key="value" dataExtractor="nullValue"/>
            </AuditPath>
        </AuditPath>
    </Application>

Audit trail

As mentioned before there is the 'alfresco-access' application. So the default entries in the trail are coming from this application. Database tables are used to store the audit trail. The following columns are visible in the trail: 'user name', 'application', 'method' (which is similar to 'action'), timestamp, entry (as explained before such an entry contains a map of values. How the entry looks like depends on the data generation and extraction).

AuditService

 The audit service implements the following interface: http://dev.alfresco.com/resource/docs/java/repository/org/alfresco/service/cmr/audit/AuditService.html . Eye catching is that you have to specify an audit query. A query is handled in an AuditQueryCallback (http://dev.alfresco.com/resource/docs/java/repository/org/alfresco/service/cmr/audit/AuditService.AuditQueryCallback.html). The callback has to implement the 'handleAuditEntry' method which gets passed the following parameters:

  • entryId
  • applicationName
  • user
  • time
  • the entry map
 Additionally it seens to be possible to just access the above defined applications RESTfully. If I understood it right, then the audit query is just the following HTTP call.
  • http://localhost:8080/alfresco/service/api/audit/query/${Application id}?${Parameters}
The documentation mentions the following parameters:

  • verbose = true | false
  • limit = ${The number of last n values to return}
  • forward = true | false
  • toId = ${Which id-s should be included. This is not the node id, but the entry id}
Summary

In summary I think that the auditing feature is capable to fit my initial requirements. It is highly customizable. So it should be possible to extend it with own Producers, Generators and Extractors based on the specific requirements. It provides indeed a suiteable framework but by adding some complexity. What should be kept in mind is that this level of complexity may affect the performance of the system in a negative way, but this needs further evaluation. I will start by using the default audit application to get my hands on it.

Thursday, February 21, 2013

Custom form controls in Alfresco

Basics

It's possible to use custom form controls to show your properties. The following snippet explains how to use them:

<config evaluator="node-type" condition="ecg:mytype">

 <forms>
      <form>
        <field-visibility>

            <!-- My property which is only visible in the view edit mode but not in the view mode-->
            <show id="ecg:myprop" for-mode="edit"/>

        </field-visibility>

        <appearance>

             <!-- Define a panel on which my property should be shown -->

             <set id="myPanel" appearance="bordered-panel" label="My Properties"/>

             

             <!-- Add the property to the panel by defining which control should be used -->

             <field id="ecg:myProp" set="myPanel" label="My Property">
                <control template="/de/ecmgeek/alfresco/mycontrol.ftl">
                    <control-param name="myFirstParam">myFirstParamValue</control-param>
                    <control-param name="mySecondParam">mySeceondParamValue</control-param>
                </control>
            </field> 

        </appearance> 

     </form> 

</forms> 

</config> 




Structure of a form control

 So a form control is just a freemarker template wich can be referenced in your form configuration. Therefore you need to store it under for instance 'web-extension/site-webscripts/de/ecmgeek/alfresco'. But how does such a form look like? Here the code of the textfield.ftl which is provided by Alfresco.


<div class="form-field">
   <#if form.mode == "view">
      <div class="viewmode-field">
         <#if field.mandatory && !(field.value?is_number) && field.value == "">
            <span class="incomplete-warning"><img src="${url.context}/res/components/form/images/warning-16.png" title="${msg("form.field.incomplete")}" /><span>
         </#if>
         <span class="viewmode-label">${field.label?html}:</span>
         <#if field.control.params.activateLinks?? && field.control.params.activateLinks == "true">
            <#assign fieldValue=field.value?html?replace("((http|ftp|https):\\/\\/[\\w\\-_]+(\\.[\\w\\-_]+)+([\\w\\-\\.,@?\\^=%&:\\/~\\+#]*[\\w\\-\\@?\\^=%&\\/~\\+#])?)", "<a href=\"$1\" target=\"_blank\">$1</a>", "r")>
         <#else>
            <#if field.value?is_number>
               <#assign fieldValue=field.value?c>
            <#else>
               <#assign fieldValue=field.value?html>
            </#if>
         </#if>
         <span class="viewmode-value"><#if fieldValue == "">${msg("form.control.novalue")}<#else>${fieldValue}</#if></span>
      </div>
   <#else>
      <label for="${fieldHtmlId}">${field.label?html}:<#if field.mandatory><span class="mandatory-indicator">${msg("form.required.fields.marker")}</span></#if></label>
      <input id="${fieldHtmlId}" name="${field.name}" tabindex="0"
             <#if field.control.params.password??>type="password"<#else>type="text"</#if>
             <#if field.control.params.styleClass??>class="${field.control.params.styleClass}"</#if>
             <#if field.control.params.style??>style="${field.control.params.style}"</#if>
             <#if field.value?is_number>value="${field.value?c}"<#else>value="${field.value?html}"</#if>
             <#if field.description??>title="${field.description}"</#if>
             <#if field.control.params.maxLength??>maxlength="${field.control.params.maxLength}"</#if>
             <#if field.control.params.size??>size="${field.control.params.size}"</#if>
             <#if field.disabled && !(field.control.params.forceEditable?? && field.control.params.forceEditable == "true")>disabled="true"</#if> />
      <@formLib.renderFieldHelp field=field />
   </#if>
</div>




As you can see the following elements are out of the box available:
  • Parameters via E.G. 'field.control.params.myFirstParam'
  • The value of the field via 'field.value'. It makes sense to make it available as via ${fieldValue} in order to use it in your HTML or JavaScript. Therefore Freemarker's assign command can be used '<#assign fieldValue=field.value> . (The term 'field.value' is only available within Freemarker directives. But it is possible to assign them to a place holder (interpolation) by using the assign directive.)
  • The form mode via 'form.mode'
  • The enabled/disabled mode of the field by using 'field.disabled'.
  • The information if the field is a madatory one or not by using 'field.mandatory
  • The field description via 'field.description' 
  • The field name directly via ${fieldName}
  • The field id via ${fieldHtmlId}. The field id is important. It will be replaced by an unique id which depends on the form id and the prefix and name of your property. So you can later reference your field by using this unique id.  
 So from what we know it is easy to create static controls those are looking as we want by chaning the HTML code. But what if we want to have a more dynamic control, for instace an autocompletion one or a custom category picker?

Within your control the Share API is available (http://sharextras.org/jsdoc/share/community-4.0.c/symbols/Alfresco.Share.html). Additionally you have access to the YUI stuff. (YUI is the UI framework behind Alfresco Share) (http://developer.yahoo.com/yui/2/). So you can mix the mentioned HTML with Java Script to perform AJAX requests in order to get data from the repository layer (provided by Web-Scripts). This client side Java Script can then directly manipulate your controls DOM tree. Important is that you run the script after your control was completely available (rendered). Here an example:



<script type="text/javascript">//<![CDATA[

(function()
{
    YAHOO.util.Event.onContentReady("${fieldHtmlId}", 
    function ()
    {
        var successHandler = function (response)
        {
             //Handle the AJAX request

             var data = response.json;



             //Use the data ... 
         };    
         
         var failureHandler = function (response) { 
                Alfresco.logger.error("Could not perform the AJAX request: " +               response.serverResponse.responseText);
         };
         

         var myparamValue = "${field.control.params.myFirstParam}";
         var url = Alfresco.constants.PROXY_URI + "ecg/mywebscript.json?myparam=" + myparamValue ;
        
         var config =
         {
            method: "GET",
            url: url,
            successCallback: 
            { 
                fn: successHandler, 
                scope: this
            },
            failureCallback:
            {
                fn: failureHandler,
                scope: this
            }
         };
        
         Alfresco.util.Ajax.request(config);
    }
    ,this);
})();

//]]></script>



 <div class="form-field">

... 

</div>


 
 How the form submission works

So as you can see it is possible to get data from the repository layer by accessing RESTFul web services. Such a RESTFul web service. Such a service can be implemented as Web Script. You then can use the data which was provided by the service in order to manipulate the DOM of your control. An auto completion control could ask the service for matching words based on the input of the user by writing out a list into a DIV container with the id '${fieldHtmlId}-acc'. A user may select multiple (by the autocompletion recommended) values. But how would you submit the selected values. The answer is: Via the value attribute of the input field with the id ' ${fieldHtmlId}'. Dependent on the type of your field, the form processor will handle the values inside this field in a different way. So if you have a classification (or user) property then the value inside the field should be a node reference. If you have a text input then the value should be just the text to show. If you want to submit multiple node references then the value shoud be a comma seperated list of node references. Because it is often not really useful to show the user a field which shows a comma seperated list of values, the trick is to have one input field (or Picker, ...) which is visible but has a different id, E.G. '${fieldHtmlId}-show' whereby storing your seperated list of node references inside a hidden field with the id '${fieldHtml${fieldHtmlId}'. So you show one field for working purposes but you use the value of the on one with the id '${fieldHtml${fieldHtmlId}' during the submission. I used the following Java Script to add new node references to the value attribute of the hidden input field with the id '${fieldHtmlId}'${fieldHtmlId} '${fieldHtmlId}':}${fieldHtml${fieldHtmlId}${fieldHtml${fieldHtmlId}':


 //Add to the hidden field
  var hiddenField = document.getElementById("${fieldHtmlId}");
  var value = hiddenField.getAttributeNode("value");
           
   var bkpValue = value.nodeValue;
   var newValue = "";
               
   if (bkpValue == "")
   {
      newValue = myNodeRef;
   }
   else
   {
        newValue = bkpValue + "," + myNodeRef;
    }
               
    value.nodeValue    = newValue;



Room for improvements

You may use one control for multiple times on one single form. This causes that you can see tha above mentioned Java Script code for multiple times in the client side Java Script code (by using Firebug ...). To avoid this you can put the JavaScript into a custom Java Script library. Which then could be for instance accessed this way:  'MyLib.addToValues(fieldId, nodeRef)'.)nfiibavi