X509 Certificate Mock Servlet Filter

This is a way low-technolgy-solution to simulate, in web application, an SSL mutual authentication.  The obvious solution would be to configure Apache httpd as front-end, and let it to ask for the user certificate, but I had some problems with an android web browser...
So, instead configure Apache with a "SSLVerifyClient require" this is my ashamed solution: to configure a servlet filter (at first position in the chain) and, if there is no certificate in the request, just put one and go on.

import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.security.cert.CertificateFactory;
import java.security.cert.X509Certificate;

import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class X509CertificateFilter implements Filter {
    private static final Logger LOGGER = LoggerFactory.getLogger(X509CertificateFilter.class);
    
    private static final String CERTIFICATE = "-----BEGIN CERTIFICATE-----\r\n" + 
            "MIIIdDCCB1ygAwIBAgIQIDbcJ3psq8FXD6IQBZA15DANBgkqhkiG9w0BAQsFADBN\r\n" +             
            "(...)\r\n" + 
            "740qIwTZlA4=\r\n" + 
            "-----END CERTIFICATE-----";
    
    @Override
    public void init(FilterConfig filterConfig) throws ServletException {                
    }

    @Override
    public void destroy() {        
    }
    
    @Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
            throws IOException, ServletException {
        if (request.getAttribute("javax.servlet.request.X509Certificate") == null) {
            request.setAttribute("javax.servlet.request.X509Certificate", buildCertificates());
        }
        
        chain.doFilter(request, response);    
    }

    private static X509Certificate[] buildCertificates() {
        try {        
            final InputStream is = new ByteArrayInputStream(CERTIFICATE.getBytes());                    
            CertificateFactory cf = CertificateFactory.getInstance("X.509");
            X509Certificate cert = (X509Certificate) cf.generateCertificate(is);
            return new X509Certificate[] {cert};
        }
        catch(Exception e) {
            LOGGER.error("Error injecting certificate", e);
            return null;
        }
    }    
}

You can also configure multiple certificates, for example in a map, and use a request parameter to pick one...

JBoss datasources and transactions practical tests

Abstract

One question which arouse from time to time is how transactions work and what kind of configuration you need to set in your application in order to make it run, and to keep your databases clean (no lost writes or incosistent states). 

The question about how to make your application run is usually solved by developers with the classic try-and-error approximation. When you face an error, you could begin to change settings until the error disappear. Recently one of these problems was related with the access to two different databases through two JTA Datasources, and in the same transaction. The magic solution found was to change one of these Datasources to non JTA. 

The question about how to keep your databases clean depends on how you solved the error. As you will see later, that solution was not much safe, but probably they didn't know.

So, I decided to prepare a simple web application to try access to two different databases, throught different kinds of datasources, in the same transaction and by ussing different combiantions.

The source code of these tests is in my GitHub: https://github.com/evazquezma/jboss/tree/master/jboss-datasources

Glosary

These are the main terms used here:

• Local transaction: A transaction which involves only one transactional resource.
• Global transaction: A transaction which involves multiple transactional resources.
• Distribuited transaction: A global transaction which spans over multiple hosts (in this case, servers).
• XA: eXtended Architecture
• XA Resource: Transactional resource which allows 2PC for global transactions.
• 1PC: One Phase Commit
• 2PC: Two Phase Commit

Test structure

The test application will be running in JBoss EAP 6.4. It will use Spring 3.X and Hibernate 4.X (sorry, I have in my to-do list to migrate all these stuff )

For these test there will a set of datasources provided for JBoss. Two of them will be datasources with JTA flag to false, two more with JTA to true (default value) and finally two XA-Datasources.
In each test the application will use different combinations of these datasources, but there will be always two different databases in the pitch.

Database access will be configured through Hibernate SessionFactory and Spring Transaction Manager:

<bean id="dataSource1" class="org.springframework.jndi.JndiObjectFactoryBean">
 <property name="jndiName" value="java:jboss/datasources/ExampleDSNoJTA"/>
</bean>

<bean  id="sessionFactory1" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean" lazy-init="false">
 <property name="dataSource" ref="dataSource1"/>
 <property name="packagesToScan" value="es.sisifo.jboss.datasources.entity.one"/>
 <property name="hibernateProperties">
  <props>
   <prop key="hibernate.dialect">org.hibernate.dialect.HSQLDialect</prop>
   <prop key="hibernate.show_sql">true</prop>
   <prop key="hibernate.format_sql">true</prop>
   <prop key="hibernate.use_sql_comments">true</prop>
   <prop key="hibernate.hbm2ddl.auto">create</prop>

   <prop key="hibernate.validator.apply_to_ddl">false</prop>
   <prop key="hibernate.validator.autoregister_listeners">false</prop>

   <prop key="hibernate.transaction.factory_class">org.hibernate.engine.transaction.internal.jta.CMTTransactionFactory</prop>
   <prop key="hibernate.transaction.jta.platform">org.hibernate.service.jta.platform.internal.JBossAppServerJtaPlatform</prop>                                
     </props>
 </property>  
</bean>

<!-- The same to sessionFactory2 -->

<!-- TX config -->
<tx:annotation-driven/>
<tx:jta-transaction-manager/>

During startup Spring will try to find the server transaction manager by checking the following JNDI entries:

• java:comp/TransactionManager
• java:appserver/TransactionManager
• java:pm/TransactionManager
• java:/TransactionManager. Bingo, it will use com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate

In each database there will be one table/entity: EntityOne and EntityTwo, with the same dummy structure:

@Entity
@Table(name = "tableOne")
public class EntityOne {
 @Id
 @GeneratedValue(strategy = GenerationType.IDENTITY)
 private Long id;

 private String code;

 private Date time;
 
 (...) 
}

Each test will consists of calling three transactional methods in a service:

@Override
@Transactional
public void testInsertOK() {
 final EntityOne entityOne = new EntityOne();
 entityOne.setTime(new Date());
 entityOne.setCode("Insert OK");
 entityOneDao.save(entityOne);

 final EntityTwo entityTwo = new EntityTwo();
 entityTwo.setTime(new Date());
 entityTwo.setCode("Insert OK");
 entityTwoDao.save(entityTwo);
}

@Override
@Transactional(rollbackFor=Exception.class)
public void testInsertMiddleError() {
 final EntityOne entityOne = new EntityOne();
 entityOne.setTime(new Date());
 entityOne.setCode("Insert with Middle Error");
 entityOneDao.save(entityOne);

 if (System.currentTimeMillis() > 0) {
  throw new RuntimeException("Kaboom in the middle");
 }

 final EntityTwo entityTwo = new EntityTwo();
 entityTwo.setTime(new Date());
 entityTwo.setCode("Insert with Middle Error");
 entityTwoDao.save(entityTwo);
}


@Override
@Transactional(rollbackFor=Exception.class)
public void testInsertFinalError() {
 final EntityOne entityOne = new EntityOne();
 entityOne.setTime(new Date());
 entityOne.setCode("Insert with Final Error");
 entityOneDao.save(entityOne);

 final EntityTwo entityTwo = new EntityTwo();
 entityTwo.setTime(new Date());
 entityTwo.setCode("Insert with Final Error");
 entityTwoDao.save(entityTwo);

 throw new RuntimeException("Kaboom at the end");
}

At the end of the tests, both tables will be listed in order to check which writes persisted and which were rolled back.

Summary

Tests results

Non JTA Datasource + Non JTA Datasource

Result:

EntityOne [id=1, code=Insert OK, time=2018-03-25 20:03:22.577]
EntityOne [id=2, code=Insert with Middle Error, time=2018-03-25 20:03:22.683]
EntityOne [id=3, code=Insert with Final Error, time=2018-03-25 20:03:22.698]

EntityTwo [id=1, code=Insert OK, time=2018-03-25 20:03:22.663]
EntityTwo [id=2, code=Insert with Final Error, time=2018-03-25 20:03:22.701]

Database writes are auto-commited inmeditely in both resources.
You can use both resources inside the same transaction, but they are not enlisted.
Each resource has its own "transaction boundary". If there is an error inside the transaction, as there is no rollback, you can get data inconsistency.

JTA Datasource + Non JTA Datasource

Result:

EntityOne [id=1, code=Insert OK, time=2018-03-24 19:16:51.645]

EntityTwo [id=1, code=Insert OK, time=2018-03-24 19:16:51.709]
EntityTwo [id=2, code=Insert with Final Error, time=2018-03-24 19:16:51.733]

JTA resource is managed inside a JTA Transaction, so transaction boundary is demarcated by the transaction manager.
Database writes are committed on transaction completion for JTA resource, and auto-committed inmediately for non JTA resource .
Thus, non JTA resource is not managed by transaction manager, if there is an error during the transaction, JTA resource is rolled back and non JTA resource is autocommitted.
You can get database inconsistencies.

Non JTA Datasource + JTA Datasource

Result:

EntityOne [id=1, code=Insert OK, time=2018-03-24 19:25:33.252]
EntityOne [id=2, code=Insert with Middle Error, time=2018-03-24 19:25:33.325]
EntityOne [id=3, code=Insert with Final Error, time=2018-03-24 19:25:33.335]

EntityTwo [id=1, code=Insert OK, time=2018-03-24 19:25:33.312]

Similar to previous scenario. In this case, the difference is due to the insertions order.
During an exception inside the transaction, only JTA resource is rolled back.

JTA Datasource + JTA Datasource

You get the following error:

19:37:52,308 WARN  [com.arjuna.ats.arjuna] (http-localhost/127.0.0.1:8080-2) ARJUNA012140: Adding multiple last resources is disallowed. Trying to add LastResourceRecord(XAOnePhaseResource(LocalXAResourceImpl@434c39c8[connectionListener=27aa95d connectionManager=5209886e warned=false currentXid=< formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffffc0a80133:17004d06:5ab68ccc:26e, node_name=1, branch_uid=0:ffffc0a80133:17004d06:5ab68ccc:275, subordinatenodename=null, eis_name=java:jboss/datasources/ExampleDS2 > productName=H2 productVersion=@PROJECT_VERSION@ (2012-07-13) jndiName=java:jboss/datasources/ExampleDS2])), but already have LastResourceRecord(XAOnePhaseResource(LocalXAResourceImpl@414aa356[connectionListener=501a920b connectionManager=1519056b warned=false currentXid=< formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffffc0a80133:17004d06:5ab68ccc:26e, node_name=1, branch_uid=0:ffffc0a80133:17004d06:5ab68ccc:272, subordinatenodename=null, eis_name=java:jboss/datasources/ExampleDS > productName=H2 productVersion=@PROJECT_VERSION@ (2012-07-13) jndiName=java:jboss/datasources/ExampleDS]))
19:37:52,309 INFO  [stdout] (http-localhost/127.0.0.1:8080-2) [2018-03-24 19:37:52,309] (SqlExceptionHelper.java:145) WARN http-localhost/127.0.0.1:8080-2 org.hibernate.engine.jdbc.spi.SqlExceptionHelper SQL Error: 0, SQLState: null

19:37:52,309 INFO  [stdout] (http-localhost/127.0.0.1:8080-2) [2018-03-24 19:37:52,309] (SqlExceptionHelper.java:147) ERROR http-localhost/127.0.0.1:8080-2 org.hibernate.engine.jdbc.spi.SqlExceptionHelper javax.resource.ResourceException: IJ000457: Unchecked throwable in managedConnectionReconnected() cl=org.jboss.jca.core.connectionmanager.listener.TxConnectionListener@27aa95d[state=NORMAL managed connection=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection@1a5a5cf8 connection handles=0 lastUse=1521916672308 trackByTx=false pool=org.jboss.jca.core.connectionmanager.pool.strategy.OnePool@11371f69 pool internal context=SemaphoreArrayListManagedConnectionPool@6f62844[pool=ExampleDS2] xaResource=LocalXAResourceImpl@434c39c8[connectionListener=27aa95d connectionManager=5209886e warned=false currentXid=null productName=H2 productVersion=@PROJECT_VERSION@ (2012-07-13) jndiName=java:jboss/datasources/ExampleDS2] txSync=null]

You are trying to enlist two transactional resources in the same transaction, i.e., you are running in a global transaction. This scenario is only allowed with XA resources.

The error is quite clear:
ARJUNA012140: Adding multiple last resources is disallowed.
Trying to add LastResourceRecord(XAOnePhaseResource(... jndiName=java:jboss/datasources/ExampleDS2])), but already have LastResourceRecord(XAOnePhaseResource(... jndiName=java:jboss/datasources/ExampleDS]))

What it's saying is that you have a 1P resource (ExampleDS) in an active transaction, and you are trying to enlist a second 1P Resource (ExampleDS) in the same transaction. You can only do that with XA resources.

JTA Datasource + JTA XA Datasource

Response:

EntityOne [id=1, code=Insert OK, time=2018-03-25 12:29:42.584]

EntityTwo [id=1, code=Insert OK, time=2018-03-25 12:29:42.695]

OK, this can sound weird. You have one JTA resource (which only supports 1PC) enlisted in a global transaction with other XA resource (which do supports 2PC). This should be an error, because you can not ensure the 2PC, i.e, the transaction coordinator can not ask the JTA resource if it is prepared.

Here is where ARJUNA came to scene. JBoss uses ARJUNA as transaction core, and it uses a LRCO - Last Resource Commit Optimization:

• http://narayana.io/docs/project/index.html#d0e1859
• https://access.redhat.com/documentation/en-us/red_hat_jboss_enterprise_application_platform/6.4/html/development_guide/sect-transaction_optimizations

The doc is surprisingly clear. You do can have a single non-XA resource inside a global transaction, but only if all the other resources do are XA. In order to perform the 2PC, in the first phase the non XA resource is processed last, and only if all the XA resources responded affirmatively to the prepare request, the non-XA Resources is committed immediately. During second phase the rest of XA Resources are committed normally.

The operations are as follow:

1. Prepare 2PC (The XA Resources)
2. Commit LRCO (The non-XA Resource)
3. Write tx log
4. Commit 2PC (The XA Resources)

But there is still an error opportunity if something crash in step 3. So, this is not a fully reliable scenario.

In order to improve LRCO there is the CMR - Commit Markable Resource, but it´s only valid for datasource (a transactional resource can be, for example, a JMS Queue, but you can not use CMR here). You need some extra configuration for this:

• Set the non-XA datasource as connectable
• Create a new table in database (the one for de non-XA datasource) to store XIDs
• Link Jboss transaction subsystem with this datasource and this table

JTA Datasource + JTA XA Datasource

Same result as previous test.

XA Datasource + XA Datasource

Same result as previous test.

This is a full global transaction scenario with XA Resources.

JTA XA Datasource + Non JTA Datasource

Same result as "JTA Datasource + Non JTA Datasource"

Non JTA Datasource + JTA XA Datasource

Same result as "Non JTA Datasource + JTA Datasource"