Buscar este blog

jueves, 28 de julio de 2016

Web service client - Spring cache

This is the third post about cache. The first one was about how to configure persistence cache with Hibernate, and the second one about how to configure a business cache with Spring, both of then by using Infinispan embbede in JBoss.

The objective of this post is to show how to configure the bussines cache with web services clients generated with maven cxf-codegen-plugin. The only special feature here is that you can not use the cache annotations (@Cacheable, @CacheEvict, etc.) because you have not access to the web service client source code. This code is generated by Maven in the compile phase.

In this example I will use the famous currency converter service: http://www.webservicex.com/CurrencyConvertor.asmx?wsdl

The first step is to configure pom.xml in order to parse the WSDL and to generate all needed classes.
<plugin>
 <groupId>org.apache.cxf</groupId>
 <artifactId>cxf-codegen-plugin</artifactId>
 <version>2.7.18</version>
 <executions>
  <execution>
   <id>generate-sources-wsdl</id>
   <phase>generate-sources</phase>
   <configuration>
    <wsdlOptions>
     <wsdlOption>
      <wsdl>${basedir}/src/main/resources/META-INF/wsdl/CurrencyConvertor.wsdl</wsdl>
      <extraargs>
       <extraarg>-xjc-XhashCode</extraarg>
       <extraarg>-xjc-Xequals</extraarg>
       <extraarg>-xjc-XtoString</extraarg>
      </extraargs>
     </wsdlOption>
    </wsdlOptions>
   </configuration>
   <goals>
    <goal>wsdl2java</goal>
   </goals>
  </execution>
 </executions>
 <dependencies>
  <dependency>
   <groupId>org.jvnet.jaxb2_commons</groupId>
   <artifactId>jaxb2-basics</artifactId>
   <version>1.11.1</version>
  </dependency>
 </dependencies>
</plugin>

I have downloaded the wsdl file and have put it inside the project in /resources/META-INF/wsdl directory.

Next, you have to configure de jaxws client, so you could use this client as a managed spring bean. In a context config file (for example, cxf-config.xml) I have this:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:jaxws="http://cxf.apache.org/jaxws"
 xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
  http://cxf.apache.org/jaxws http://cxf.apache.org/schemas/jaxws.xsd">


 <bean id="loggingOutInterceptor" class="org.apache.cxf.interceptor.LoggingOutInterceptor">
    <property name="prettyLogging" value="true"/>
 </bean>

 <bean id="loggingInInterceptor" class="org.apache.cxf.interceptor.LoggingInInterceptor">
    <property name="prettyLogging" value="true"/>
 </bean>


 <jaxws:client id="currencyConversorClient"
  name="{http://www.webserviceX.NET}CurrencyConvertorSoap12" 
  serviceClass="net.webservicex.CurrencyConvertorSoap"
  address="http://www.webservicex.com/CurrencyConvertor.asmx">

  <jaxws:outInterceptors>
   <ref bean="loggingOutInterceptor" />
  </jaxws:outInterceptors>

  <jaxws:inInterceptors>
   <ref bean="loggingInInterceptor" />
  </jaxws:inInterceptors>
 </jaxws:client>

</beans>

With this configuration you can use the currencyConversorClient injected in other Spring managed beans, for example:
@Autowired    
private net.webservicex.CurrencyConvertorSoap currencyService;

And finally, you have to configure the cache in a declarative way, by using another context config file, as is explained in the reference documentation.
<!-- cache manager and  annotation-driven configuration omited-->

<!-- cache definitions -->
<cache:advice id="currencyConversorWebServiceCacheAdvice" cache-manager="cacheManager">
 <cache:caching cache="currency">
  <cache:cacheable method="conversionRate"/>
 </cache:caching>
</cache:advice>

<aop:config>
 <aop:advisor advice-ref="currencyConversorWebServiceCacheAdvice" pointcut="execution(* net.webservicex.CurrencyConvertorSoap.*(..))"/>
</aop:config>

In this snippet you declare a cache named "currency" over the conversionRate method in the CurrencyConvertorSoapCurrencyConvertorSoap service. Note that this is actually an interface which was generated by Maven.


Updated 1
Ultimately, a cache is just a hashmap like structure which store keys (the methods parameters) and values (the method return). In order to work properly, the cache keys must override the equals and hashcode methods, because two instances of a class with the same properties may not be equals.

So, when creating the web service class, you need to configure the cxf-codegen-plugin to add equals and hashcode methods to the DTOs. You get this by using the extra args tag, and by importing jaxb2-basics dependencies.

The currency example is very basic and the method params are simple types, which don't require this approximation, but for complex scenarios this would be mandatory.

Updated 2
Under some circumstances (I don't know exactly when) you need to add a dependency with org.jboss.ironjacamar.jdbcadapters module in the jboss-deployment-structure.xml file. Otherwise you will get a very confusing error.

Check this post: http://trabajosdesisifo.blogspot.com.es/2016/08/jboss-illegalargumentexception-warning.html

miércoles, 27 de julio de 2016

Jboss - Spring - Infinispan cache

In a previous post I explained how to configure Hibernate Second Level Cache with Infinispan in a cluster enviroment, check here. Now I´ll show you how to configure a business cache with Spring, also with Infinispan in a JBoss Cluster enviroment.

When I talk about bussines cache I mean the service tier in a multi-tier JEE application. Usually one service (or use case) involves multiple accesses to database, even to multiple databases and remote services, so this kind of cache is complementary to Hibernate cache.

Spring Framework provides a cache abstraction, an infraestructure arround some spring-managed beans which delegates the cache management over a plugable cache container. In this case, this container will be Infinispan.


JBoss configuration

You need to use, at least, the HA profile (High Availability).

Similar to Hibernate, the cache container will be provided by JBoss. In this example I created a myCustomCache container, configured over UDP in order to share information over the cluster nodes.
/profile=ha/subsystem=infinispan/cache-container=myCustomCache:add(start=EAGER)
/profile=ha/subsystem=infinispan/cache-container=myCustomCache/transport=TRANSPORT:add(lock-timeout="60000", stack="udp", cluster="myCustomCache-cluster")


Application configuration

Spring config

Spring will use the cache container created previously to store and recover the cached objects. This container will be obtained via JNDI and injected in a bean of SpringEmbeddedCacheManager.
But this procedure has a problem, you can not define your own cache regions. When spring asks for a new cache region, if it yet not exists , the cache manager creates it with default values. In a cluster enviroment, as I tested, this is a local cache, so it will not be shared amongst the nodes.

In order to create custom regions, and declare them as distributables, you need to create your own extension of the SpringEmbeddedCacheManager. This class will read an internal infinispan configuration file, parse it and "add" the declared caches to the cache manager.

This is the CustomSpringEmbeddedCacheManager class:
public class CustomSpringEmbeddedCacheManager extends SpringEmbeddedCacheManager implements InitializingBean {
    private static Logger LOGGER = LoggerFactory.getLogger(CustomSpringEmbeddedCacheManager.class);

    private Resource infinispanXMLFile;

 public void setInfinispanXMLFile(final Resource infinispanXMLFile) {
        this.infinispanXMLFile = infinispanXMLFile;
    }
 
 
    public CustomSpringEmbeddedCacheManager(final EmbeddedCacheManager nativeCacheManager) {
        super(nativeCacheManager);
    }

    @Override
    public void afterPropertiesSet() throws Exception {
     final EmbeddedCacheManager cacheManager = getNativeCacheManager();
   
        final ParserRegistry parserRegistry = new ParserRegistry(Thread.currentThread().getContextClassLoader());
        final ConfigurationBuilderHolder configurationBuilderHolder = parserRegistry.parse(infinispanXMLFile.getInputStream());
        final Map<String, ConfigurationBuilder> namedConfigs = configurationBuilderHolder.getNamedConfigurationBuilders();
      
        for (final Map.Entry<String, ConfigurationBuilder> entry : namedConfigs.entrySet()) {
            if (cacheManager.cacheExists(entry.getKey())) {
                LOGGER.trace("Eliminando cache {}", entry.getKey());
                cacheManager.removeCache(entry.getKey());
            }

            LOGGER.trace("Definiendo nueva cache {}", entry.getKey());
            cacheManager.defineConfiguration(entry.getKey(), entry.getValue().build());
        }
    } 
}

Now, in your application context, you need to obtain de Infinispan cache manager and configure the spring cache manager.
<jee:jndi-lookup id="nativeCacheManager" expected-type="org.infinispan.manager.CacheContainer" jndi-name="java:jboss/infinispan/container/myCustomCache"/>
          
<bean id="cacheManager"  class="es.cixtec.arq.jboss.CustomSpringEmbeddedCacheManager">
  <constructor-arg>
   <ref bean="nativeCacheManager"/>
  </constructor-arg>
  <property name="infinispanXMLFile" value="classpath:infinispan-business.xml"/>
 </bean>

<cache:annotation-driven />

Note that you need to use a internal infinispan configuration file, in this case infinispan-business.xml, located in the classpath (maven resources directory):
<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="urn:infinispan:config:5.1 http://www.infinispan.org/schemas/infinispan-config-5.1.xsd"
 xmlns="urn:infinispan:config:5.2">

 <namedCache name="someBusiness">
  <eviction strategy="LRU" maxEntries="10000" />
  <expiration lifespan="60000" maxIdle="60000" />
  <clustering mode="replication">
   <sync replTimeout="10000" />
  </clustering>
 </namedCache>

</infinispan>

Dependency config

You need to import the following dependencys:
<dependencies>

 <dependency>
  <groupId>org.springframework</groupId>
  <artifactId>spring-context</artifactId>
  <version>3.2.17.RELEASE</version>
 </dependency>
  
 <dependency>
  <groupId>org.infinispan</groupId>
  <artifactId>infinispan-core</artifactId>
  <version>5.2.15.Final</version>
  <scope>provided</scope>
 </dependency>
 
</dependencies>

Documentation

Available doc:

domingo, 24 de julio de 2016

Jboss - Hibernate - Infinispan cache

This is the basic configuration in order to use Hibernate Second Level Cache (2LC) inside a JBoss Server in a cluster enviroment. The cache provider will be Infinispan, wich is shipped with the server.
Second Level Cache (2LC)
A Second Level Cache (2LC) is a data store which holds persistent information relating to application state. Examples of 2LC consumers are session state replication, Single Sign On (SSO), and Java Persistence API (JPA). JBoss Enterprise Application Platform 6 uses Infinispan to manage its 2LC.
There is also a First Level Cache which is always enabled, but it works at the session level. The Hiberante Session, usually, is created when a transaction is opened and it is bounded to it. If during the life span of the transaction you get the same object two times from database, only in the first one Hibernate hits the database, the second time, the object is recovered from session.
The limitation with First Level Cache is that it only apply during the transaction, and only for the client who started it.

JBoss configuration

You need to use, at least, the HA profile (High Availability) .

Inside infinispan subsystem there should be an hibernate cache container, but it must be configured with the start attribute to EAGER.
/profile=ha/subsystem=infinispan/cache-container=hibernate:write-attribute(name=start, value=EAGER)

The result would be something like this:
<cache-container name="hibernate" start="EAGER" default-cache="local-query" module="org.jboss.as.jpa.hibernate:4">
    <transport lock-timeout="60000"/>
    <local-cache name="local-query">
        <transaction mode="NONE"/>
        <eviction strategy="LRU" max-entries="10000"/>
        <expiration max-idle="100000"/>
    </local-cache>
    <invalidation-cache name="entity" mode="SYNC">
        <transaction mode="NON_XA"/>
        <eviction strategy="LRU" max-entries="10000"/>
        <expiration max-idle="100000"/>
    </invalidation-cache>
    <replicated-cache name="timestamps" mode="ASYNC">
        <transaction mode="NONE"/>
        <eviction strategy="NONE"/>
    </replicated-cache>
</cache-container>

As you can see, there are three declared caches: local-query, entity and timestamps, each one with its own type attending to the clusterization mode. Infinispan allows the following clustering modes:
  • Local mode. Better performance, but no cluster aware.
  • Replicated mode. All the nodes in the cluster have their own local cache copy. When one node modify its local cache, it sends a message (sychronous or asynchronous) to the other nodes so they can update their local copy too.
  • Invalidation mode. No data is shared. When a node modifies some data in its own local cache copy, it sends a message to the other nodes so they can clear this object from their local copy.
  • Replication mode. Each cached object exists only in one (or in a small number) of the nodes in the cluster. Instead of replicate an object in all the nodes, they are informed about who owns it.

When the number of nodes increase (bigger than 10) the best option is the replication mode.

Application configuration

Hibernate config

Configure hibernate session factory. With the hibernateProperties you can enable 2LC and force hibernate to use the JBoss cache container obtained via JNDI.
<bean class="org.springframework.orm.hibernate4.LocalSessionFactoryBean" id="sessionFactory" lazy-init="false">
 (...)
 <property name="hibernateProperties">
 <props>
   (...)
   
     <!-- Cache -->
     <prop key="hibernate.cache.use_second_level_cache">true</prop>
     <prop key="hibernate.cache.use_query_cache">true</prop>            
     <prop key="hibernate.cache.infinispan.statistics">true</prop>   
     <prop key="hibernate.cache.region.factory_class">org.hibernate.cache.infinispan.JndiInfinispanRegionFactory</prop>
     <prop key="hibernate.cache.infinispan.cachemanager">java:jboss/infinispan/container/hibernate</prop> 
                               
 </props>
 </property>  
</bean>

Hibernate will use two of them, plus two other news cache wich will be configured the application startup. As you can see in org.hibernate.cache.infinispan.InfinispanRegionFactory, these caches are:
  • entity
  • collection
  • timestamps
  • query
The default values of thesea caches are well documented in this post: Using Infinispan as JPA/Hibernate Second Level Cache Provider

Hibernate will not share entities, collections or querys among the nodes of the cluster in order to reduce the traffic. For example, each node will have an entity instaced cached in its local store, and when this instance is changed by one of them, the others will be informed to evicts it.

Dependency config

You need to import the following dependencys:
<dependencyManagement>
 <dependencies>
  <dependency>
   <groupId>org.jboss.bom.eap</groupId>
   <artifactId>jboss-javaee-6.0-with-hibernate</artifactId>
   <version>6.2.0.GA</version>
   <type>pom</type>
   <scope>import</scope>
  </dependency>
 </dependencies>
</dependencyManagement>  


<dependencies>
 (...)
 
 <dependency>
  <groupId>org.hibernate</groupId>
  <artifactId>hibernate-core</artifactId>
  <scope>provided</scope>
 </dependency>

 <dependency>
  <groupId>org.hibernate</groupId>
  <artifactId>hibernate-infinispan</artifactId>
  <scope>provided</scope>
 </dependency>
</dependencies>

Note that I'm declaring hibernate dependencies as provided, because I will use the jars shipped inside JBoss, so the WAR does not need to have them in its lib directory.


You have to declare thease dependencies in the META-INF/jboss-deployment-structure.xml file, so JBoss will add them to the classpath during the application boot:
<?xml version="1.0" encoding="UTF-8"?>
<jboss-deployment-structure>
 <deployment>
  <dependencies>
   <module name="org.hibernate" />
   <module name="org.javassist" />
   <module name="org.infinispan" services="import"/>   
   <module name="org.jboss.ironjacamar.jdbcadapters"/>
  </dependencies>

 </deployment>
</jboss-deployment-structure>

Documentation

Available doc:
This doc belongs to wildfly, but it can also be applied to EAP 6:

sábado, 25 de junio de 2016

Apache CXF - Logging interceptor in separate log files

If you have multiple services, each one with its LoggingInInterceptor and LoggingOutInterceptor, probably you will get all messages logged in the same log file.

<bean id="loggingInInterceptor" class="org.apache.cxf.interceptor.LoggingInInterceptor"/>    
<bean id="loggingOutInterceptor" class="org.apache.cxf.interceptor.LoggingIOutInterceptor"/>


<jaxws:endpoint id="service1" implementor="#springService1" address="/service1">
 <jaxws:inInterceptors>          
  <ref bean="logginInInterceptor"/>             
 </jaxws:inInterceptors>
 
 <jaxws:outInterceptors>          
  <ref bean="loggingOutInterceptor"/>             
 </jaxws:inInterceptors>
</jaxws:endpoint>


<jaxws:endpoint id="service2" implementor="#springService2" address="/service2">
 <jaxws:inInterceptors>          
  <ref bean="logginInInterceptor"/>             
 </jaxws:inInterceptors>
 
 <jaxws:outInterceptors>          
  <ref bean="loggingOutInterceptor"/>             
 </jaxws:inInterceptors>
</jaxws:endpoint>

The trick is that CXF prints each message in a specify log category, based in the service name, port name and portTypeName. These parameters are part of the own service configuration, so they will be unique for each web service.

You can see their values in the logging message (the minimum log level required is INFO). For example:
20:54:19,421 INFO  [stdout] (http-localhost/127.0.0.1:8080-2) [2016-06-25 20:54:19,420] (AbstractLoggingInterceptor.java:249) INFO http-localhost/127.0.0.1:8080-2 org.apache.cxf.services.service1.MyService1WebServiceImplPort.MyService1WebService Inbound Message ...

You can handle these categories by using your log configuration, for example, with log4j:
<logger name="org.apache.cxf.services.service1">
    <level value="INFO" />
  <appender-ref ref="FILE1" />
</logger>

<logger name="org.apache.cxf.services.service2">
    <level value="INFO" />
  <appender-ref ref="FILE2" />
</logger>

sábado, 4 de junio de 2016

Apache CXF - Logging interceptor mask sensitive information

Apache CXF has two built-in logging interceptors:

They are used when you define your service client or service provider:
<bean id="loggingOutInterceptor" class="org.apache.cxf.interceptor.LoggingOutInterceptor">
 <property name="prettyLogging" value="false" />
</bean>

<bean id="loggingInInterceptor" class="org.apache.cxf.interceptor.LoggingInInterceptor">
 <property name="prettyLogging" value="false" />
</bean>

<jaxws:client id="myWSClient" 
 name="xxxxx"     
 serviceClass="xxxxx"
 address="xxxxx"> 
   
 <jaxws:outInterceptors>   
  <ref bean="loggingOutInterceptor" />
 </jaxws:outInterceptors>

 <jaxws:inInterceptors>
  <ref bean="loggingInInterceptor" />  
 </jaxws:inInterceptors>   
</jaxws:client>

In this way, you get a Log of the outbound and inbound messages in your application.

The problem is that you are also logging sensitive information, like user passwords, in case they are sent in plain text (often happens). So, I want to transform specific xml tags content (SOAP messages are xml based) in '*'.  For example:
Outbound Message
---------------------------
ID: 4
Address: https://desarr.local/accesosweb/servizos/entrada
Encoding: UTF-8
Http-Method: POST
Content-Type: text/xml
Headers: {Accept=[*/*], Connection=[Keep-Alive], SOAPAction=["http://tempuri.org/ObtenerAplicaciones"]}
Payload: <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Body><ObtenerAplicaciones xmlns="http://tempuri.org/"><usuario><Uid>Sisifo</Uid><password>**********</password></usuario></ObtenerAplicaciones></soap:Body></soap:Envelope>
--------------------------------------

Both CXF logging interceptors extends from AbstractLoggingInterceptor. In this class there is a protected method called transform. As this javadoc states, it is meant to mask sensitive information:
Transform the string before display. The implementation in this class does nothing. Override this method if you wish to change the contents of the logged message before it is delivered to the output. For example, you can use this to mask out sensitive information.
So you only need to extend these interceptors and override this method. You receives the message before it is printed, so you have to extract the sensitive data and to convert then in whatever you want.

I created a class called LoggingTransform, which implements the same signature as the transform method. In the constructor, it receives the list o tags to filter. For example, if you want to mask '<password>myPass</password>' you identify it with 'password'. It does not have xpath query support.


LoggingTransform
import java.util.ArrayList;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;


public class LoggingTransform {
 private final List<Pattern> patterns;

 public LoggingTransform(final List<String> tagsToFilter) {
  patterns = new ArrayList<Pattern>();
  for (final String tagToFilter : tagsToFilter) {
   patterns.add(buildNewPattern(tagToFilter));
  }
 }

 private Pattern buildNewPattern(final String tagName) {
  return Pattern.compile(
    String.format("<%s>(.+?)</%s>", tagName, tagName),
    Pattern.CASE_INSENSITIVE | Pattern.MULTILINE);
 }

 
 public String transform(final String originalLogString) {
  String filtered = originalLogString;

  for (final Pattern pattern : patterns) {
   final Matcher matcher = pattern.matcher(originalLogString);
   while (matcher.find()) {
    filtered = filter(filtered, matcher.start(1), matcher.end(1));
   }
  }
  return filtered;
 }

 private String filter(final String original, final int posIni, final int posFin) {
  return original.substring(0, posIni) + repeat("*", posFin - posIni) + original.substring(posFin);
 }

 private String repeat(final String str, final int times) {
  return new String(new char[times]).replace("\0", str);
 }
}

Then you create the two interceptors, wich extends from their respective CXF interceptors, and override the transform method. In this method you have to calle the LoggingTransform´s transform method.

FilteredLoggingInInterceptor
import java.util.List;

import org.apache.cxf.interceptor.LoggingInInterceptor;

public class FilteredLoggingInInterceptor extends LoggingInInterceptor {
 private final LoggingTransform loggingTransform;

 public FilteredLoggingInInterceptor(final List<String> tagsToFilter) {
  loggingTransform = new LoggingTransform(tagsToFilter);
 }

 @Override
 public String transform(final String originalLogString) {
  return loggingTransform.transform(originalLogString);
 }
}

FilteredLoggingOutInterceptor
import java.util.List;

import org.apache.cxf.interceptor.LoggingOutInterceptor;

public class FilteredLoggingOutInterceptor extends LoggingOutInterceptor {
 private final LoggingTransform loggingTransform;

 public FilteredLoggingOutInterceptor(final List<String> tagsToFilter) {
  loggingTransform = new LoggingTransform(tagsToFilter);
 }

 @Override
 public String transform(final String originalLogString) {
  return loggingTransform.transform(originalLogString);
 }
}

And finally, declare your new interceptors.
<bean id="loggingOutInterceptor" class="es.sisifo.cxf.logging.FilteredLoggingOutInterceptor">
 <constructor-arg name="tagsToFilter">
  <list>
   <value>password</value>
   <value>otherSensitiveTag</value>
  </list>
 </constructor-arg>
 <property name="prettyLogging" value="false" />
</bean>


<bean id="loggingInInterceptor" class="es.sisifo.cxf.logging.FilteredLoggingInInterceptor">
 <constructor-arg name="tagsToFilter">
  <list>
   <value>password</value>
   <value>otherSensitiveTag</value>   
  </list>
 </constructor-arg>
 <property name="prettyLogging" value="false" />
</bean>