How to Programatically Parse a Custom Web Services SoapFault and Map it to an Object Defined in the WSDL's Schema xsd

Sometimes the server side Web Services provider offers a custom soap fault detail implementation. The client side needs to parse it and map it back into a java object generated from the WSDL/xsd.
For example, an soapFault look likes this with the custom :

<s:Body>
 <s:Fault>
  <faultcode>s:Client</faultcode>
  <faultstring xml:lang="en-US">Operation parameter is invalid.</faultstring>
  <detail>
   <ServerCustomizedError>
    <ErrorMessage>Operation parameter is invalid.</ErrorMessage>
    <Id>56a145c6-a7f3-4069-94a4-6ec945d95613</Id>
    <ParameterName>input</ParameterName>
    <ValidationDetails>
     <Message>Why is this wrong?</Message>
     <Target>wrong value</Target>
     <Key>wrong data</Key>
    </ValidationDetails>
   </ServerCustomizedError>
  </detail>
 </s:Fault>
</s:Body>

The corresponding client side java code to parse the above Soap Fault will look like:

Iterator it = soapFaultClientException.getSoapFault().getFaultDetail().getDetailEntries();
while (it.hasNext())
{
 Source errSource = it.next().getSource();
 @SuppressWarnings("unchecked")
 JAXBElement errJaxb = (JAXBElement) springWebServiceTemplate.getUnmarshaller().unmarshal(errSource);
 ServerCustomizedError err = errJaxb.getValue();
 ....
}

Access Web Services over a SSL Layer

I have a Java application to access web services hosted behind a HTTPS transport protocol. The client side java application gave me the error message like:

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

I did the following to resolve the issue:
1. Try to access the web services endpoint from a browser and export its certificate using the 'X.509 certificate format' into a sample.cert file.
2. Locate the JVM's keystore. For example, \dev\Java\jdk1.6.0_34\jre\lib\security
3. Back up its original keystore and run the following command under the above directory:
\dev\Java\jdk1.6.0_34\jre\bin\keytool -import -keystore cacerts -file sample.cer -alias sampleXYZ
The default password is 'changeit'. It's fine if the certificate is a self-signed one. If so, be sure to answer 'Yes' for the question to trust the certificate.
4. After restart of the Java application, another error may show up as:
CertificateException: No subject alternative names present
I tried to edit C:\Windows\System32\drivers\etc\hosts as Administrator and put the ip address matching to the CN host name defined in the cert
To review the CN name, run the following command:
keytool -list -v -keystore .\cacerts -alias sampleXYZ
The dns needs to be refreshed from a command line console to pick up the new definition:
ipconfig /flushdns

Websphere Application Server installation and deployment

As a Java developer, you may need to install IBM Websphere Application Server (WAS) onto machines from time to time. But the process is a little bit intense for most newbies. My post here applies to Websphere Application Server Version 7. Hope it save your valuable time of trouble shoot some non-sense errors or reading through lengthy documentation.

In a very high level, the IBM Websphere Application Server (WAS) installation can be achieved through the following steps:
1. Obtain the license or your IBM Partener world ID
2. Install the latest IBM Installation Manager. Manually add the two links of the repositories if they are missing. Import existing WAS installation if any.
2.1 https://www.ibm.com/software/rational/repositorymanager/repositories/websphere/repository.config
2.2 http://public.dhe.ibm.com/software/websphere/repositories/repository.config
3. Install the right version of WAS onto a directory without space
4. Use the latest IBM Update Installer to install the fix pack. Otherwise you may get some weird exceptions or cannot install the desired feature pack. For example,
The EAR contains both xmi and XML bindings files at the application level which is not supported.
I got my eyes hurt trying to find what's wrong with my ears but simply found out it's a typical IBM joke.
5. Use the IBM Installation Manager to install the desired feature pack and update to its latest fix pack version if needed

After the installation is finished, please use your AppSrv01's Administrative Console to
* define any security
* define the environment variables of db driver path and etc
* define the datasource/jdbc properties
* install your application and start it
* access your web application through the port 9080.

All the configuration will be represented in one of the configuration files in a directory like: IBM\WebSphere\AppServer\profiles\AppSrv01\config\cells\...

Please be sure to run the command console as an Administrator if you try to start/stop the server from the command line on a Windows 7 or above.

Mutual Authentication with Client Certificate over HTTPS/SSL using Java

This blog is about SSL/TLS mutual authentication using Java. I am on the client side with a client certificate signed by an intermediate issuer and finally by Verisign. (aha, a certificate chain is here to make the situation not vanilla already.) The server requests a client certificate and recognizes Verisign as a Certification Authority (CA). You can safely ignore this blog if you have a self-certified certificate.

During the implementation, I got tripped more than a couple of times over error messages like:

403.7 Forbidden: Client Certificate Required.
javax.net.ssl.SSLException: HelloRequest followed by an unexpected  handshake message

To save somebody some time in the future, a step by step instruction is provided below:

1. I assume you have a valid certificate or a chain of certificates, whose root is acceptable by the server. The valid certificate contains its private key. Run the following command to verify:
   

   keytool -list -v -keystore "your certificate file"
   ...
   Entry type: PrivateKeyEntry

2. Import your certificate and intermediate certificates into a browser like IE or Firefox and test out the https URL. This step will validate the certificates and save you a lot of troubles down the road. Java version of the SSL implementation is not as simple/mature as the browsers'. Please make sure all the certificates have not expired.
3. Backup your keystore located at /your_home_directory/.keystore by default and the truststore located at somewhere similar to \Java\jre6\lib\security\cacerts
   
4. Use not-yet-commons-ssl utility to import your certificates into the Java keystore format. Sample command is:
   

   java -cp not-yet-commons-ssl-0.3.9.jar org.apache.commons.ssl.KeyStoreBuilder 

5. Customize the following java code, replace the static final Strings to fit in your needs. Note that this implementation forcefully use a specific alias to present the corresponding certificate/certificate chain to the server. Somehow the default KeyManager simply disqualifies my certificate to be presented to the server.
   

   public class Main {
   
   private static final Logger logger = Logger.getLogger(Main.class.getName());
   private static final String LINE_BREAKER = System.getProperty("line.separator");
   
   private static final String CERTIFACATE_FILE = "your keystore location";
   private static final String CERTIFACATE_PASS = "changeit";
   private static final String CERTIFACATE_ALIAS = "your alias";
   private static final String TARGET_URL = "https://xyz.com";
   
   
   public static void main(String[] args) {
      String targetURL = TARGET_URL;
      URL url;
      HttpsURLConnection connection = null;
      BufferedReader bufferedReader = null;
      InputStream is = null;
   
      try {
          //Create connection
          url = new URL(targetURL);
          //Uncomment this in case server demands some unsafe operations
          //System.setProperty("sun.security.ssl.allowUnsafeRenegotiation", "true");
          connection = (HttpsURLConnection) url.openConnection();
   
          connection.setRequestMethod("POST");
          connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
          connection.setRequestProperty("Content-Language", "en-US");
   
          SSLSocketFactory sslSocketFactory = getFactory(new File(CERTIFACATE_FILE), CERTIFACATE_PASS, CERTIFACATE_ALIAS);
          connection.setSSLSocketFactory(sslSocketFactory);
   
          //Process response
          is = connection.getInputStream();
   
          bufferedReader = new BufferedReader(new InputStreamReader(is));
          String line;
          StringBuffer lines = new StringBuffer();
          while ((line = bufferedReader.readLine()) != null) {
              lines.append(line).append(LINE_BREAKER);
          }
          logger.info("response from " + targetURL + ":" + LINE_BREAKER + lines);
   
      } catch (Exception e) {
        ...
      }
    }
   
       private static SSLSocketFactory getFactory(File pKeyFile, String pKeyPassword, String certAlias) throws Exception {
           KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance("SunX509");
           KeyStore keyStore = KeyStore.getInstance("JKS");
   
           InputStream keyInput = new FileInputStream(pKeyFile);
           keyStore.load(keyInput, pKeyPassword.toCharArray());
           keyInput.close();
           keyManagerFactory.init(keyStore, pKeyPassword.toCharArray());
   
           //Replace the original KeyManagers with the AliasForcingKeyManager
           KeyManager[] kms = keyManagerFactory.getKeyManagers();
           for (int i = 0; i < kms.length; i++) {
               if (kms[i] instanceof X509KeyManager) {
                   kms[i] = new AliasForcingKeyManager((X509KeyManager) kms[i], certAlias);
               }
           }
   
           SSLContext context = SSLContext.getInstance("TLS");
           context.init(kms, null, null);
           return context.getSocketFactory();
       }
   
       /*
        * This wrapper class overwrites the default behavior of a X509KeyManager and
        * always render a specific certificate whose alias matches that provided in the constructor
        */
       private static class AliasForcingKeyManager implements X509KeyManager {
   
           X509KeyManager baseKM = null;
           String alias = null;
   
           public AliasForcingKeyManager(X509KeyManager keyManager, String alias) {
               baseKM = keyManager;
               this.alias = alias;
           }
   
           /*
            * Always render the specific alias provided in the constructor
            */
           public String chooseClientAlias(String[] keyType, Principal[] issuers, Socket socket) {
               return alias;
           }
   
           public String chooseServerAlias(String keyType, Principal[] issuers, Socket socket) {
               return baseKM.chooseServerAlias(keyType, issuers, socket);
           }
   
           public X509Certificate[] getCertificateChain(String alias) {
               return baseKM.getCertificateChain(alias);
           }
   
           public String[] getClientAliases(String keyType, Principal[] issuers) {
               return baseKM.getClientAliases(keyType, issuers);
           }
   
           public PrivateKey getPrivateKey(String alias) {
               return baseKM.getPrivateKey(alias);
           }
   
           public String[] getServerAliases(String keyType, Principal[] issuers) {
               return baseKM.getServerAliases(keyType, issuers);
           }
       }
   }
   

   
   
   
6. Try to set
   

   -Dsun.security.ssl.allowUnsafeRenegotiation=true

   if you get the error message like:
   

   javax.net.ssl.SSLException: HelloRequest followed by an unexpected  handshake message

7. If anything goes wrong, turn on -Djavax.net.debug=all to debug. Verify the keystore and truststore locations. Verify the presence of the certificates. Here is a sample log file with successful connection: http://java.sun.com/javase/6/docs/technotes/guides/security/jsse/ReadDebug.html

Grails Environment Specific Logging to a File

I need to log my Grails application across different environments. For example, a Grails embeded Jetty server in DEV environment and a Tomcat server in TEST environment. Different servers have different log directory locations. How to dynamically pick up the right location? The trick is to define the file location (a relative path in my case) as a global variable inside Config.groovy, customized it in the environment blocks, and use the variable location inside log4j closure.

The following is the Config.groovy:

def logDirectory = '.'
environments {
 development {
 }
 test {
     logDirectory = "../logs"    
 }
}

// log4j configuration
log4j = {
 appenders {
     console name:'stdout'
     rollingFile  name:'file', file: logDirectory + '/app.log', threshold: org.apache.log4j.Level.INFO, maxFileSize:"1MB", maxBackupIndex: 10, 'append':true    
 }

 error  'org.codehaus.groovy.grails.web.servlet',  //  controllers
        'org.codehaus.groovy.grails.web.pages', //  GSP
        'org.codehaus.groovy.grails.web.sitemesh', //  layouts
        'org.codehaus.groovy.grails.web.mapping.filter', // URL mapping
        'org.codehaus.groovy.grails.web.mapping', // URL mapping
        'org.codehaus.groovy.grails.commons', // core / classloading
        'org.codehaus.groovy.grails.plugins', // plugins
        'org.codehaus.groovy.grails.orm.hibernate', // hibernate integration
        'org.springframework',
        'org.hibernate',
        'org.codehaus.groovy.grails.scaffolding.view.ScaffoldingViewResolver'

 warn   'org.mortbay.log'

 root {
     info 'stdout', 'file'
     additivity = true
 }
}

It has been tested to work on Grails version is 1.1.1 with Tomcat 6.0.

Grails and Query Into Database View and Multiple Datasources

I want to bypass the Grails stack to query a database directly. It sounds silly. Anyway the very first reason why I chose Grails is to avoid that hassle. However I have a special case that it demands read-only queries into:

1. multiple databases
2. a database view
3. Netezza, a database with no hibernate dialect
   

In my opinion, the difficulties of 2) and 3) keep any Object Relational Mapping (ORM) solutions out of the game. Otherwise, the Grails plugin, datasources, will be a good candidate to resolve issue 1).

The following solution is a Groovy-style DAO query:

1. define the second datasource in grails-app/conf/spring/resources.groovy:

   
   import org.apache.commons.dbcp.BasicDataSource
   beans = {
   dataSourceNetezza(BasicDataSource) {
    driverClassName = "org.netezza.Driver"
    url = "jdbc:netezza://server:5480/database_name"
    username = "xyz"
    password = "abc"
   }
   }
   

   
   


2. Use the dataSourceNetezza defined above in a controller where the Spring application context is available. Note that the dataSourceNetezza is auto-wired by Spring and the names have to match.

import groovy.sql.Sql

class TablesController {

def dataSourceNetezza

def list = {
 if (servletContext.dbList == null)
  servletContext.dbList = new Sql(dataSourceNetezza).rows("SELECT distinct DATABASE_NAME FROM REPOSITORY")
 print servletContext.dbList
}
 ...
}

This approach looks extremely simple with less than 15 lines of code. Keep in mind that, there is no ORM.

Grails and Legacy Database

Recently I played around Grails with a legacy database. Here are some tips I gathered around the journey:
1. I chose to use SpringSource Tool Suite, an IDE based on Eclipse. I love its feature to bring up Grails command prompt with a convenient short cut of 'ctrl-alt-g'.
2. Google 'GRAG' and use its latest version to reverse engineer your legacy database. GRAG certainly has room for improvement but it does a decent job to create the domain objects and save you a lot of typing. BTW, it's totally fine if you don't want to use GRAG and prefer to manually generate the domain classes.
3. Refine the domain definition. Note that Grails/Hibernate uses 'id' as the default field name for primary keys. You may not wish to change this convention or define another field as existingId. Unfortunately that's just the default behavior of GRAG 1.1.

static mapping = {
 table 'Legacy_Table'
 version false
 id column:'Existing_ID'
} ...

4. If the existing primary key is in the type of a String the domain definition needs additional tweak:

static mapping = {
 table 'Legacy_Table'
 version false
 id column:'String_ID', generator:'assigned'
}
String id

An exception of 'Fail to convert to internal representation' will be thrown if you fail to declare the id as a String

5. If you are careful enough you may find that the generator is defined as 'assigned' instead of the default value. This means the application will generate the primary key. Accordingly the save() operation has to explicitly set the String 'id'. Controller's save operation will look like:

   def save = {
       def legacyTable = new legacyTable(params)
legacyTable.id = params.id
       if(!legacyTable.hasErrors() && legacyTable.save())
       ...
    }

Please let me know if it works. I will be surprised if not.

One of the limitations Grails have is that, the 'version false' statement basically turns off the versioning from Gorm and its underline Hibernate implementation. If two users update the same database entry simultaneously, the system's behavior may not be predictable. In another word, we are counting on the optimistic lock on the data integrity.