Sunday, May 13, 2012

Solving a Producer-Consumer Problem in Java

The producer-consumer problem is one of the most frequently encountered problems when we attempt multi threaded programming. While not as challenging as some of the other problems in multi-threaded programming, an incorrect implementation of this problem can create a mess of your application. Produced items will be left unconsumed, starting items will be skipped, consumption depends on whether the production began earlier or later than consumption attempts etc. To add to this you might notice the anomalies long after it has actually happened and most importantly like almost all multi-threaded programs, this one is hard to debug and reproduce too.

So in this post I thought I would attempt to solve this problem in Java with the help of Java' awesome java.util.concurrent package and its classes.

First of all, let us see the characteristics of the producer-consumer problem:
  • Producer(s) produce items.
  • Consumer(s) consume the items produced by the producer(s).
  • Producer(s) finish production and let the consumers know that they are done.
Note that in this producer-consumer problem the producer is running on a different thread than the ones on consumer. This setup makes sense in two cases:
  • The steps to do the consumption of the item produced in independent and not dependent on other items.
  • The time to process the items is larger that the time to produce them.
The term "larger" in the second point is used a bit loosely. Consider the case where producer reads a line from a file and the "consumption and processing" is just to log the line in a special format back to a file then the use of a producer consumer problem solution can be considered a case of over-engineering a solution. However if for each of those lines the "consumption and processing" step is to make a HTTP GET/POST request to a web-server and then dump the result somewhere then we should opt for a producer-consumer solution. In this case I am assuming that all the data to do a GET/POST is available in the line (item) itself and we are not dependent on previous/next lines.

So let us first take a look at the characteristics of the producer-consumer problem solution that I have posted below:
  • There can be multiple producer.
  • There will be multiple consumers.
  • Once the production of new items is done the producer(s) will let the consumers know so that the consumer will exit after the last item is consumed and processed.
It is interesting to note that to solve this problem at a generic level we can address only the consumer side and not the producer side. This is because the production of items might be done at any time and there is very little that we can do in a generic way to control the production of items. We can, however control the consumer's behaviour while accepting items from producer(s). Having laid out the rules let us take a look at the consumer contract:


package com.maximus.producerconsumer;

public interface Consumer
{
 public boolean consume(Item j);
 
 public void finishConsumption();
} 

Here the consumer can be shared between multiple producers of similar items; by similar items I mean producer that produces objects of type "Item". The definition if Item is as follows:

package com.maximus.consumer;

public interface Item
{
 public void process();
}

Now we take a look at an implementation of the Consumer interface:

package com.maximus.consumer;

import java.util.LinkedList;
import java.util.List;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.LinkedBlockingQueue;

public class ConsumerImpl implements Consumer
{
 private BlockingQueue< Item > itemQueue = 
  new LinkedBlockingQueue< Item >();
 
 private ExecutorService executorService = 
  Executors.newCachedThreadPool();
 
 private List< ItemProcessor > jobList = 
  new LinkedList< ItemProcessor >();
 
 private volatile boolean shutdownCalled = false;
  
 public ConsumerImpl(int poolSize)
 {
  for(int i = 0; i < poolSize; i++)
  {
   ItemProcessor jobThread = 
    new ItemProcessor(itemQueue);
   
   jobList.add(jobThread);
   executorService.submit(jobThread);
  }
 }
 
 public boolean consume(Item j)
 {
  if(!shutdownCalled)
  {
   try
   {
    itemQueue.put(j);
   }
   catch(InterruptedException ie)
   {
    Thread.currentThread().interrupt();
    return false;
   }
   return true;
  }
  else
  {
   return false;
  }
 }
 
 public void finishConsumption()
 {
  shutdownCalled = true;
  
  for(ItemProcessor j : jobList)
  {
   j.cancelExecution();
  }
  
  executorService.shutdown();
 }
}

Now the only point of interest is the ItemProcessor that the consumer internally uses to process the incoming items. ItemProcessor is coded as follows:

package com.maximus.consumer;

import java.util.concurrent.BlockingQueue;
import java.util.concurrent.TimeUnit;

public class ItemProcessor implements Runnable
{
 private BlockingQueue< Item> jobQueue;
 
 private volatile boolean keepProcessing;
  
 public ItemProcessor(BlockingQueue< Item > queue)
 {
  jobQueue = queue;
  keepProcessing = true;
 }
 
 public void run()
 {
  while(keepProcessing || !jobQueue.isEmpty())
  {
   try
   {
    Item j = jobQueue.poll(10, TimeUnit.SECONDS);
    
    if(j != null)
    {
     j.process();
    }
   }
   catch(InterruptedException ie)
   {
    Thread.currentThread().interrupt();
    return;
   }
  }
 }
 
 public void cancelExecution()
 {
  this.keepProcessing = false;
 }
} 
The only challenge above is the condition in the while loop. The while loop is so written to support the continuation of the consumption of items even after the producer(s) have finished production and has notified the consumer that production is finished. The above while loop ensures that consumption of all the items is done before the threads exit.This will be the case when producers run faster that consumers.

The above consumer is thread-safe and can be shared multiple producers such that each producer may concurrently call consumer.consume() without bothering about synchronization and other multi-threading caveats. Producers just need to submit an implementation of the Item interface whose process() method will contain the logic of how the consumption will be done.

As a bonus for reading the post I put forward a test program that demonstrates how to use the above classes:


package com.maximus.consumer;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStreamReader;

public class Test
{
 public static void main(String[] args) throws Exception
        {
         Consumer consumer = new ConsumerImpl(10);
         
         BufferedReader br = 
          new BufferedReader(
          new InputStreamReader(
          new FileInputStream(
          new File(args[0]))));
         
         String line = "";
         
         while((line = br.readLine()) != null)
         {
          System.out.println(
           "Producer producing: " + line);
          consumer.consume(new PrintJob(line));
         }
         
         consumer.finishConsumption();
        }
}

class PrintJob implements Item
{
 private String line;
 
 public PrintJob(String s)
 {
  line = s;
 }
 
 public void process()
 {
  System.out.println(
   Thread.currentThread().getName() + 
   " consuming :" + line);
 }
}
The above consumer can be tweaked in a host of different ways to make it more flexible. We can define what the consumer will do when production is done. It may be tweaked to allow batch processing but I leave that to the user. Feel free to use it and twist it in whatever way you want.

Happy coding!

Thursday, October 20, 2011

A Generic and Concurrent Object Pool

In this post we will take a look at how we can create an object pool in Java. In recent years, the performance of the JVM has multiplied manifold that object pooling for better performance has been made almost redundant for most type of objects. In essence, creation of objects are no longer considered as expensive as it was done before.

However there are some kind of objects that certainly proves costly on creation. Objects such as Threads, database connection objects etc are not lightweight objects and are slightly more expensive to create. In any application we require the use of multiple objects of the above kind. So it would be great if there was a very way easy to create and mantain an object pool of that type so that objects can be dynamically used and reused, without the client code being bothered about the live cycle of the objects.

Before actually writing the code for an object pool, let us first identify the main requirements that any object pool must answer.
  • The pool must let clients use an object if any is available.
  • It must reuse the objects once they are returned to the pool by a client.
  • If required, it must be able to create more objects to satisfy growing demands of the client.
  • It must provide a proper shutdown mechanism, such that on shutdown no memory leaks occur.
Needless to say, the above points will form the basis of the interface that we will expose to our clients.

So our interface declaration will be as follows:

package com.test.pool;


/**
 * Represents a cached pool of objects.
 * 
 * @author Swaranga
 *
 * @param < T > the type of object to pool.
 */
public interface Pool< T >
{
 /**
  * Returns an instance from the pool. 
  * The call may be a blocking one or a non-blocking one 
  * and that is determined by the internal implementation.
  * 
  * If the call is a blocking call, 
  * the call returns immediately with a valid object 
  * if available, else the thread is made to wait 
  * until an object becomes available.
  * In case of a blocking call, 
  * it is advised that clients react 
  * to {@link InterruptedException} which might be thrown
  * when the thread waits for an object to become available.
  * 
  * If the call is a non-blocking one, 
  * the call returns immediately irrespective of 
  * whether an object is available or not.
  * If any object is available the call returns it 
  * else the call returns < code >null< /code >.
  * 
  * The validity of the objects are determined using the
  * {@link Validator} interface, such that 
  * an object < code >o< /code > is valid if 
  * < code > Validator.isValid(o) == true < /code >.
  * 
  * @return T one of the pooled objects.
  */
 T get();
 
 /**
  * Releases the object and puts it back to the pool.
  * 
  * The mechanism of putting the object back to the pool is
  * generally asynchronous, 
  * however future implementations might differ.
  * 
  * @param t the object to return to the pool
  */
  
 void release(T t);
 
 /**
  * Shuts down the pool. In essence this call will not 
  * accept any more requests 
  * and will release all resources.
  * Releasing resources are done 
  * via the < code >invalidate()< /code >
  * method of the {@link Validator} interface.
  */
 
 void shutdown();
}

The above interface is intentionally made very simple and generic to support any type of objects. It  provides methods to  get/return an object from/to the pool. It also provides a shutdown mechanism to dispose of the objects. 

Now we try to create an implementation of the above interface. But before doing that it is important to note that an ideal release() method will first try to check if the object returned by the client is still reusable. If yes then it will return it to the pool else the object has to be discarded. We want every implementation of the Pool interface to follow this rule. So  before creating a concrete implementation, we create an abstract implementation hat imposes this restriction on subsequent implementations. Our abstract implementation will be called, surprise, AbstractPool and its definition will be as follows:
package com.test.pool;

/**
 * Represents an abstract pool, that defines the procedure
 * of returning an object to the pool.
 * 
 * @author Swaranga
 *
 * @param < T > the type of pooled objects.
 */
abstract class AbstractPool < T > implements Pool < T >
{
 /**
  * Returns the object to the pool. 
  * The method first validates the object if it is
  * re-usable and then puts returns it to the pool.
  * 
  * If the object validation fails, 
  * some implementations
  * will try to create a new one 
  * and put it into the pool; however 
  * this behaviour is subject to change 
  * from implementation to implementation
  * 
  */
 @Override
 public final void release(T t)
 {
  if(isValid(t))
  {
   returnToPool(t);
  }
  else
  {
   handleInvalidReturn(t);
  }
 }
 
 protected abstract void handleInvalidReturn(T t);
 
 protected abstract void returnToPool(T t);
 
 protected abstract boolean isValid(T t);
}

In the above class, we have made it mandatory for object pools to validate an object before returning it to the pool. To customize the behaviour of their pools the implementations are free to chose the way they implement the three abstract methods. They will decide using their own logic, how to check if an object is valid for reuse [the validate() method], what to do if the object returned by a client is not valid [the handleInvalidReturn() method] and the actual logic to return a valid object to the pool [the returnToPool() method].

Now having the above set of classes we are almost ready for a concrete implementation. But the catch is that since the above classes are designed to support generic object pools, hence a generic implementation of the above classes will not know how to validate an object [since the objects will be generic :-)]. Hence we need something else that will help us in this.

What we actually need is a common way to validate an object so that the concrete Pool implementations will not have to bother about the type of objects being validated. So we introduce a new interface, Validator, that defines methods to validate an object. Our definition of the Validator interface will be as follows:

package com.test.pool;

 /**
  * Represents the functionality to 
  * validate an object of the pool
  * and to subsequently perform cleanup activities.
  * 
  * @author Swaranga
  *
  * @param < T > the type of objects to validate and cleanup.
  */
 public static interface Validator < T >
 {
  /**
   * Checks whether the object is valid.
   * 
   * @param t the object to check.
   * 
   * @return true 
   * if the object is valid else false.
   */
  public boolean isValid(T t);
  
  /**
   * Performs any cleanup activities 
   * before discarding the object.
   * For example before discarding 
   * database connection objects,
   * the pool will want to close the connections. 
   * This is done via the 
   * invalidate() method.
   * 
   * @param t the object to cleanup
   */
  
  public void invalidate(T t);
 }
The above interface defines methods to check if an object is valid and also a method to invalidate and object. The invalidate method should be used when we want to discard an object and clear up any memory used by that instance. Note that this interface has little significance by itself and makes sense only when used in context of an object pool. So we define this interface inside the top level Pool interface. This is analogous to the Map and Map.Entry interfaces in the Java Collections Library. Hence our Pool interface becomes as follows:

package com.test.pool;


/**
 * Represents a cached pool of objects.
 * 
 * @author Swaranga
 *
 * @param < T > the type of object to pool.
 */
public interface Pool< T >
{
 /**
  * Returns an instance from the pool. 
  * The call may be a blocking one or a non-blocking one 
  * and that is determined by the internal implementation.
  * 
  * If the call is a blocking call, 
  * the call returns immediately with a valid object 
  * if available, else the thread is made to wait 
  * until an object becomes available.
  * In case of a blocking call, 
  * it is advised that clients react 
  * to {@link InterruptedException} which might be thrown
  * when the thread waits for an object to become available.
  * 
  * If the call is a non-blocking one, 
  * the call returns immediately irrespective of 
  * whether an object is available or not.
  * If any object is available the call returns it 
  * else the call returns < code >null< /code >.
  * 
  * The validity of the objects are determined using the
  * {@link Validator} interface, such that 
  * an object < code >o< /code > is valid if 
  * < code > Validator.isValid(o) == true < /code >.
  * 
  * @return T one of the pooled objects.
  */
 T get();
 
 /**
  * Releases the object and puts it back to the pool.
  * 
  * The mechanism of putting the object back to the pool is
  * generally asynchronous, 
  * however future implementations might differ.
  * 
  * @param t the object to return to the pool
  */
  
 void release(T t);
 
 /**
  * Shuts down the pool. In essence this call will not 
  * accept any more requests 
  * and will release all resources.
  * Releasing resources are done 
  * via the < code >invalidate()< /code >
  * method of the {@link Validator} interface.
  */
 
 void shutdown();

 /**
  * Represents the functionality to 
  * validate an object of the pool
  * and to subsequently perform cleanup activities.
  * 
  * @author Swaranga
  *
  * @param < T > the type of objects to validate and cleanup.
  */
 public static interface Validator < T >
 {
  /**
   * Checks whether the object is valid.
   * 
   * @param t the object to check.
   * 
   * @return true 
   * if the object is valid else false.
   */
  public boolean isValid(T t);
  
  /**
   * Performs any cleanup activities 
   * before discarding the object.
   * For example before discarding 
   * database connection objects,
   * the pool will want to close the connections. 
   * This is done via the 
   * invalidate() method.
   * 
   * @param t the object to cleanup
   */
  
  public void invalidate(T t);
 }
}

We are almost ready for a concrete implementation. But before that we need one final weapon, which is actually the most important weapon of an object pool. It is called "the ability to create new objects".c Sine our object pools will be generic, they must have knowledge of how to create new objects to populate its pool. This functionality must also not depend on the type of the object pool and must be a common way to create new objects. The way to do this will be an interface, called ObjectFactory that defines just one method, which is "how to create a new object". Our ObjectFactory interface is as follows:
package com.test.pool;

/**
 * Represents the mechanism to create 
 * new objects to be used in an object pool.
 * 
 * @author Swaranga
 *
 * @param < T > the type of object to create. 
 */
public interface ObjectFactory < T >
{
 /**
  * Returns a new instance of an object of type T.
  * 
  * @return T an new instance of the object of type T
  */
 public abstract T createNew();
}

We are finally done with our helper classes and now we will create a concrete implementation of the Pool interface. Since we want a pool that can be used in concurrent applications, we will create a blocking pool that blocks the  client if no objects are available in the pool. The blocking mechanism will block indefinitely until an objects becomes available. This kind of implementation begets that another method be there which will block only for a given time-out period, if any object becomes available before the time out that object is returned otherwise after the timeout instead of waiting for ever, a null object is returned. This implementation is analogous to a LinkedBlockingQueue implementation of the Java Concurrency API and thus before implementing the actual class we expose another implementation, BlockingPool, which is analogous to the BlockingQueue interface of the Java Concurrency API.

Hence the Blockingpool interface declaration is as follows:

package com.test.pool;

import java.util.concurrent.TimeUnit;

/**
 * Represents a pool of objects that makes the 
 * requesting threads wait if no object is available.
 * 
 * @author Swaranga
 *
 * @param < T > the type of objects to pool.
 */
public interface BlockingPool < T > extends Pool < T >
{
 /**
  * Returns an instance of type T from the pool.
  * 
  * The call is a blocking call, 
  * and client threads are made to wait
  * indefinitely until an object is available. 
  * The call implements a fairness algorithm 
  * that ensures that a FCFS service is implemented.
  * 
  * Clients are advised to react to InterruptedException. 
  * If the thread is interrupted while waiting 
  * for an object to become available,
  * the current implementations 
  * sets the interrupted state of the thread 
  * to true and returns null. 
  * However this is subject to change 
  * from implementation to implementation.
  * 
  * @return T an instance of the Object 
  * of type T from the pool.
  */
 T get();
 
 /**
  * Returns an instance of type T from the pool, 
  * waiting up to the
  * specified wait time if necessary 
  * for an object to become available..
  * 
  * The call is a blocking call, 
  * and client threads are made to wait
  * for time until an object is available 
  * or until the timeout occurs. 
  * The call implements a fairness algorithm 
  * that ensures that a FCFS service is implemented.
  * 
  * Clients are advised to react to InterruptedException. 
  * If the thread is interrupted while waiting 
  * for an object to become available,
  * the current implementations 
  * set the interrupted state of the thread 
  * to true and returns null. 
  * However this is subject to change 
  * from implementation to implementation.
  *  
  * 
  * @param time amount of time to wait before giving up, 
  *   in units of unit
  * @param unit a TimeUnit determining 
  *   how to interpret the
  *        timeout parameter
  *        
  * @return T an instance of the Object 
  * of type T from the pool.
  *        
  * @throws InterruptedException 
  * if interrupted while waiting
  */
 
 T get(long time, TimeUnit unit) throws InterruptedException;
}

And our BoundedBlockingPool implementation will be as follows:
package com.test.pool;

import java.util.concurrent.BlockingQueue;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.TimeUnit;

public final class BoundedBlockingPool < T > 
 extends AbstractPool < T >
 implements BlockingPool < T >
{
 private int size;
 
 private BlockingQueue < T > objects;
 
 private Validator < T > validator;
 private ObjectFactory < T > objectFactory;
 
 private ExecutorService executor = 
  Executors.newCachedThreadPool();
  
 private volatile boolean shutdownCalled;
 
 public BoundedBlockingPool(
   int size, 
   Validator < T > validator, 
   ObjectFactory < T > objectFactory)
 {
  super();
  
  this.objectFactory = objectFactory;
  this.size = size;
  this.validator = validator;
  
  objects = new LinkedBlockingQueue < T >(size);
  
  initializeObjects();
  
  shutdownCalled = false;
 }
 
 public T get(long timeOut, TimeUnit unit)
 {
  if(!shutdownCalled)
  {
   T t = null;
   
   try
   {
    t = objects.poll(timeOut, unit);
    
    return t;
   }
   catch(InterruptedException ie)
   {
    Thread.currentThread().interrupt();
   }
   
   return t;
  }
  
  throw new IllegalStateException(
   "Object pool is already shutdown");
 }
 
 public T get()
 {
  if(!shutdownCalled)
  {
   T t = null;
   
   try
   {
    t = objects.take();
   }
   catch(InterruptedException ie)
   {
    Thread.currentThread().interrupt();
   }
   
   return t;
  }
  
  throw new IllegalStateException(
   "Object pool is already shutdown");
 }
 
 public void shutdown()
 {
  shutdownCalled = true;
  
  executor.shutdownNow();
  
  clearResources();
 }
 
 private void clearResources()
 {
  for(T t : objects)
  {
   validator.invalidate(t);
  }
 }
 
 @Override
 protected void returnToPool(T t)
 {
  if(validator.isValid(t))
  {
   executor.submit(new ObjectReturner(objects, t));
  }
 }
 
 @Override
 protected void handleInvalidReturn(T t)
 {
  
 }
 
 @Override
 protected boolean isValid(T t)
 {
  return validator.isValid(t);
 }
 
 private void initializeObjects()
 {
  for(int i = 0; i < size; i++)
  {
   objects.add(objectFactory.createNew());
  }
 }
 
 private class ObjectReturner < E > 
            implements Callable < Void >
 {
  private BlockingQueue < E > queue;
  private E e;
  
  public ObjectReturner(BlockingQueue < E > queue, E e)
  {
   this.queue = queue;
   this.e = e;
  }
  
  public Void call()
  {
   while(true)
   {
    try
    {
     queue.put(e);
     break;
    }
    catch(InterruptedException ie)
    {
     Thread.currentThread().interrupt();
    }
   }
   
   return null;
  }
 }
}

The above is a very basic object pool backed internally by a LinkedBlockingQueue. The only method of interest is the returnToPool() method. Since the internal storage is a blocking pool, if we tried to put the returned element directly into the LinkedBlockingPool, it might block he client if the queue is full. But we do not want a client of an object pool to block just for a mundane task like returning an object to the pool. So we have made the actual task of inserting the object into the LinkedBlockingQueue as an asynchronous task and submit it to an Executor instance so that the client thread can return immediately.

Now we will use the above object pool into our code. We will use the object pool to pool some database connection objects. Hence we will need a Validator to validate our database connection objects.

Our JDBCConnectionValidator will look like this:

package com.test;

import java.sql.Connection;
import java.sql.SQLException;

import com.test.pool.Pool.Validator;

public final class JDBCConnectionValidator 
    implements Validator < Connection >
{
 public boolean isValid(Connection con)
 { 
  if(con == null)
  {
   return false;
  }
  
  try
  {
   return !con.isClosed();
  }
  catch(SQLException se)
  {
   return false;
  }
 }
 
 public void invalidate(Connection con)
 {
  try
  {
   con.close();
  }
  catch(SQLException se)
  {
   
  }
 }
}

And our JDBCObjectFactory, that will enable the object pool to create new objects will be as follows:

package com.test;

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;

import com.test.pool.ObjectFactory;

public class JDBCConnectionFactory 
 implements ObjectFactory < Connection >
{
 private String connectionURL;
 private String userName;
 private String password;
  
 public JDBCConnectionFactory(
  String driver, 
  String connectionURL, 
  String userName, 
  String password)
        {
         super();
         
         try
         {
          Class.forName(driver);
         }
         catch(ClassNotFoundException ce)
         {
          throw new IllegalArgumentException(
           "Unable to find driver in classpath", ce);
         }
         
         this.connectionURL = connectionURL;
         this.userName = userName;
         this.password = password;
        }
 
 public Connection createNew()
 { 
  try
  {
   return 
       DriverManager.getConnection(
    connectionURL, 
    userName, 
    password);
  }
  catch(SQLException se)
  {
   throw new IllegalArgumentException(
    "Unable to create new connection", se);
  }
 }
}

Now we create a JDBC object pool using the above Validator and ObjectFactory:

package com.test;
import java.sql.Connection;

import com.test.pool.Pool;
import com.test.pool.PoolFactory;


public class Main
{
 public static void main(String[] args)
    {
  Pool < Connection > pool = 
   new BoundedBlockingPool < Connection > (
    10, 
    new JDBCConnectionValidator(),
    new JDBCConnectionFactory("", "", "", "")
    );
  
  //do whatever you like
    }
}

As a bonus for reading the entire post. I will provide another implementation of the Pool interface that is essentially a non blocking object pool. The only difference of this implementation for the previous one is that this implementation does not block the client if an element is unavailable, rather return null. Here it goes:

package com.test.pool;

import java.util.LinkedList;
import java.util.Queue;
import java.util.concurrent.Semaphore;

public class BoundedPool < T > 
 extends AbstractPool < T >
{
 private int size;
 
 private Queue < T > objects;
 
 private Validator < T > validator;
 private ObjectFactory < T > objectFactory;
 
 private Semaphore permits;
  
 private volatile boolean shutdownCalled;
 
 public BoundedPool(
  int size, 
  Validator < T > validator, 
  ObjectFactory < T > objectFactory)
 {
  super();
  
  this.objectFactory = objectFactory;
  this.size = size;
  this.validator = validator;
  
  objects = new LinkedList < T >();
  
  initializeObjects();
  
  shutdownCalled = false;
 }
 
 
 @Override
 public T get()
 {
  T t = null;
  
  if(!shutdownCalled)
  {
   if(permits.tryAcquire())
   {
    t = objects.poll();
   }
  }
  else
  {
   throw new IllegalStateException(
    "Object pool already shutdown");
  }
  
  return t;
 }

 @Override
 public void shutdown()
 {
  shutdownCalled = true;
  
  clearResources();
 }
 
 private void clearResources()
 {
  for(T t : objects)
  {
   validator.invalidate(t);
  }
 }

 @Override
 protected void returnToPool(T t)
 {
  boolean added = objects.add(t);
  
  if(added)
  {
   permits.release();
  }
 }
 
 @Override
 protected void handleInvalidReturn(T t)
 {
  
 }

 @Override
 protected boolean isValid(T t)
 {
  return validator.isValid(t);
 }
 
 private void initializeObjects()
 {
  for(int i = 0; i < size; i++)
  {
   objects.add(objectFactory.createNew());
  }
 }
}

Considering we are now two implementations strong, it is better to let users create our pools via factory with meaningful names. Here is the factory:

package com.test.pool;

import com.test.pool.Pool.Validator;

/**
 * Factory and utility methods for 
 * {@link Pool} and {@link BlockingPool} classes 
 * defined in this package. 
 * This class supports the following kinds of methods:
 *
 * 
    *
  • Method that creates and returns a default non-blocking * implementation of the {@link Pool} interface. *
  • * *
  • Method that creates and returns a * default implementation of * the {@link BlockingPool} interface. *
  • *
* * @author Swaranga */ public final class PoolFactory { private PoolFactory() { } /** * Creates a and returns a new object pool, * that is an implementation of the {@link BlockingPool}, * whose size is limited by * the size parameter. * * @param size the number of objects in the pool. * @param factory the factory to create new objects. * @param validator the validator to * validate the re-usability of returned objects. * * @return a blocking object pool * bounded by size */ public static < T > Pool < T > newBoundedBlockingPool( int size, ObjectFactory < T > factory, Validator < T > validator) { return new BoundedBlockingPool < T > ( size, validator, factory); } /** * Creates a and returns a new object pool, * that is an implementation of the {@link Pool} * whose size is limited * by the size parameter. * * @param size the number of objects in the pool. * @param factory the factory to create new objects. * @param validator the validator to validate * the re-usability of returned objects. * * @return an object pool bounded by size */ public static < T > Pool < T > newBoundedNonBlockingPool( int size, ObjectFactory < T > factory, Validator < T > validator) { return new BoundedPool < T >(size, validator, factory); } }

Thus our clients now can create object pools in a more readable manner:

package com.test;
import java.sql.Connection;

import com.test.pool.Pool;
import com.test.pool.PoolFactory;


public class Main
{
 public static void main(String[] args)
    {
  Pool < Connection > pool = 
   PoolFactory.newBoundedBlockingPool(
    10, 
    new JDBCConnectionFactory("", "", "", ""), 
    new JDBCConnectionValidator());
  
  //do whatever you like
    }
}

And so ends our long post. This one was long overdue. Feel free to use it, change it, add more implementations.

Happy coding!

Wednesday, September 14, 2011

Customizing Java Serialization [Part 2]

In our last post we looked at how to customize the process of serialization. Today we look a bit further and see how we can manage versioning of our classes in serialization and deserialization. Basically we will cover the following reasons on why we should look to customize the serialization mechanism:

  • Persist only meaningful data. 
  • Manage serialization between different versions of your class. 
  • Avoid exposing the serialization mechanism to client API.

Persist only sensible data: This one can be subtle sometimes and hence I will take the example from one of the standard classes. Consider the following skeletal declaration of the LinkedList<E> class in java.util:
class LinkedList < E >
{
	private Entry < E > header;
	int size = 0;

	private static class Entry < E >
	{
		E element;

		Entry < E > next;
		Entry < E > previous;

		// other implementations
	}

	// other implementations
}
I have removed and edited out parts of the above class that does not pertain to our example. The above class represents a basic linked list where each node is internally represented by an instance of the Entry class. Note that each instances of Entry has two references next and previous to simulate to simulate a doubly linked list.

Now consider that we have to make our LinkedList class serializable. If we accept the default serializable policy [after making the Entry class implement Serializable] we get a working solution, provided the element E is also Serializable. Hence the class declaration could become:
class LinkedList < E > implements Serializable
{
	private Entry < E > header;
	int size = 0;

	private static class Entry < E > implements Serializable
	{
		E element;
		Entry < E > next;
		Entry < E > previous;

		// other implementations
	}

	// other implementations
}
However consider the way Java resolves object serialization by creating an object graph. In this case, for each node in the list the JVM will have to visit the next and previous node too for serialization. But in effect, the previous node will always have been marked for serialization so effectively 50% of the object resolution is done with no additional effect. We can do much better if we go for a custom serialization. The idea is to make the header instance transient and not implement Serializable for the Entry class. Now we define a writeObject and readObject  hook that instead serializes/de-serializes all the elements into/from the stream. Hence our class definition becomes:
class LinkedList < E > implements Serializable
{
	private transient Entry < E > header;
	int size = 0;

	private static class Entry < E > implements Serializable
	{
		E element;
		Entry < E > next;
		Entry < E > previous;

		// other implementations
	}

	private Object readObject(ObjectInputStream o)
		throws IOException, ClassNotFoundException
	{
		int size = o.readInt();
		// Initialize header

		header = new Entry < E >(null, null, null);
		header.next = header.previous = header;

		// Read in all elements in the proper order.
		for (int i = 0; i  <  size; i++)
		{
			addBefore((E) s.readObject(), header);
		}
	}

	private void writeObject(java.io.ObjectOutputStream s)
		throws java.io.IOException
	{
		// Write out size
		s.writeInt(size);
		
		// Write out all elements in the proper order.
		for (Entry e = header.next; e != header; e = e.next)
		{
			s.writeObject(e.element);
		}
	}
}
In the above case the JVM is relieved of the expensive and partly redundant object graph traversal/creation because we chose to do away with the default serialization process.
Manage Class Versions: Whenever we implement a class as a Serializable one, the JVM looks for a field in the class called serialVersionUID. This is a long value and is basically used to associate version numbers to Serializable classes. The field is used during deserialization to verify that the sender and receiver of a serialized object have loaded classes for that object that are compatible with respect to serialization. If the receiver has loaded a class for the object that has a different serialVersionUID than that of the corresponding sender's class, then deserialization will result in an InvalidClassException. A serializable class can declare its own serialVersionUID explicitly by declaring a field named "serialVersionUID" that must be static, final, and of type long:
ANY-ACCESS-MODIFIER static final long serialVersionUID = 42L;
If a serializable class does not explicitly declare a serialVersionUID, then the serialization runtime will calculate a default serialVersionUID value for that class based on various aspects of the class, as described in the Java(TM) Object Serialization Specification. However, it is strongly recommended that all serializable classes explicitly declare serialVersionUID values, since the default serialVersionUID computation is highly sensitive to class details that may vary depending on compiler implementations, and can thus result in unexpected InvalidClassExceptions during deserialization. Therefore, to guarantee a consistent serialVersionUID value across different java compiler implementations, a serializable class must declare an explicit serialVersionUID value. It is also strongly advised that explicit serialVersionUID declarations use the private modifier where possible, since such declarations apply only to the immediately declaring class--serialVersionUID fields are not useful as inherited members.

To illustrate the above, let us go through an example:

Suppose we have a class that represents a version of our product and want to serialize the class:
class Version implements Serializable
{
	private String majorVersion;
	private String minorversion;
	private String subMinorVersion;

	public Version(String majorVersion, 
		           String minorversion, 
		           String subMinorVersion)
	{
		this.majorVersion = majorVersion;
		this.minorversion = minorversion;
		this.subMinorVersion = subMinorVersion;
	}

	public String getVersion()
	{
		return  majorVersion + "." + 
			    minorversion + "." + 
			    subMinorVersion;
	}
}
Note that we have not declared any serialVersionUID in our class, hence the compiler will insert its own serialVersionUID into our class.
Now we use the following snippet to serialize an instance of the above class:

public final class VersionTest
{
	public static void main(String[] args) throws Exception
	{
		Version v1 = new Version("5", "01", "001");

		ObjectOutputStream out = new ObjectOutputStream(
						new FileOutputStream(
							new File("VersionTest.dat")));

		out.writeObject(v1);

		ObjectInputStream in = new ObjectInputStream(
						new FileInputStream(
							new File("VersionTest.dat")));

		Version u = (Version) in.readObject();

		in.close();

		System.out.println(u.getVersion());
	}
}
The above code prints “5.01.001”. So far so good.
Now suppose we decide to insert a “service pack” details to our version class. The class declaration becomes:

class Version implements Serializable
{
	private String majorVersion;
	private String minorversion;
	private String subMinorVersion;
	private String servicePackName;
	private String servicePackNumber;

	public Version(String majorVersion, 
			       String minorversion, 
			       String subMinorVersion, 
			       String servicePackName, 
			       String servicePackNumber)
	{
		this.majorVersion = majorVersion;
		this.minorversion = minorversion;
		this.subMinorVersion = subMinorVersion;
		this.servicePackName = servicePackName;
		this.servicePackNumber = servicePackNumber;
	}

	public String getVersion()
	{
		return majorVersion + "." + 
			   minorversion + "." + 
			   subMinorVersion + ", " + 
			   servicePackName + " " + 
			   servicePackNumber;
	}
}
Now we try to read the previously serialized object from the Stream using the following code:
public final class VersionTest
{
	public static void main(String[] args) throws Exception
	{
		ObjectInputStream in = new ObjectInputStream(
						new FileInputStream(
							new File("VersionTest.dat")));

		Version u = (Version) in.readObject();

		in.close();

		System.out.println(u.getVersion());
	}
}
On running the above code, the program throws a java.io.InvalidClassException. The reason is that while serialization, since we did not specify a serialVersionUID member in our class, the JVM inserted the serialVersionUID itself, which happened to be 7356172634211253087L. This calculation was done using the various field declarations of the class. Now when we modify our class to add two new fields, the generated serialVersionUID will be different [8734138131922723649 to be exact]. So when deserializing, the JVM sees that the serialVersionUID of the stream class and that of the local class is different, hence it saw the conflict as an attempt to deserialize an incompatible version of the local class. So it threw the exception.

However we know that the two versions are not completely incompatible. In fact the newer version of the class is just an extension of its previous version. So is there a way we can avoid the exception and still manage the deserialization? You bet we can!

The solution is to introduce our own serialVersionUID field in our class definition. This value can be any arbitrary long value. Now our previous class definition gets a serialVersionUID of 1L.

class Version implements Serializable
{
	private static final long serialVersionUID = 1L;

	private String majorVersion;
	private String minorversion;
	private String subMinorVersion;

	public Version(String majorVersion, 
		           String minorversion, 
		           String subMinorVersion)
	{
		this.majorVersion = majorVersion;
		this.minorversion = minorversion;
		this.subMinorVersion = subMinorVersion;
	}

	public String getVersion()
	{
		return  majorVersion + "." + 
			    minorversion + "." + 
			    subMinorVersion;
	}
}
Now we serialize an instance of the above class using the same code as shown above. Later we change our class definition to this:

class Version implements Serializable
{
	private static final long serialVersionUID = 1L;
	
	private String majorVersion;
	private String minorversion;
	private String subMinorVersion;
	private String servicePackName;
	private String servicePackNumber;

	public Version(String majorVersion, 
			       String minorversion, 
			       String subMinorVersion, 
			       String servicePackName, 
			       String servicePackNumber)
	{
		this.majorVersion = majorVersion;
		this.minorversion = minorversion;
		this.subMinorVersion = subMinorVersion;
		this.servicePackName = servicePackName;
		this.servicePackNumber = servicePackNumber;
	}

	public String getVersion()
	{
		return majorVersion + "." + 
			   minorversion + "." + 
			   subMinorVersion + ", " + 
			   servicePackName + " " + 
			   servicePackNumber;
	}
}
Note that we have not changed our serialVersionUID value. If we deserialize our stream into this version of the class the code prints “5.01.001, null null”. Which means the classes were found to be compatible and the unknown fields were given their default values. We can perfect our class so that the null, null is not printed if the servicePackName and the servicePackNumber is null.

class Version implements Serializable
{
	private static final long serialVersionUID = 1L;
        
	private String majorVersion;
	private String minorversion;
	private String subMinorVersion;
	private String servicePackName;
	private String servicePackNumber;
	
	
	public Version(String majorVersion, 
			String minorversion, 
			String subMinorVersion, 
			String servicePackName, 
			String servicePackNumber)
	{
	        this.majorVersion = majorVersion;
	        this.minorversion = minorversion;
	        this.subMinorVersion = subMinorVersion;
	        this.servicePackName = servicePackName;
	        this.servicePackNumber = servicePackNumber;
	}
	
	public String getVersion()
	{
		String ret = majorVersion +"." + 
			     minorversion + "." + 
			     subMinorVersion;
		
		if(servicePackName != null)
		{
			ret = ret + ", " + servicePackName;
		}
		
		if(servicePackNumber != null)
		{
			ret = ret + " " + servicePackNumber;
		}
		
		return ret;
	}
}
Avoid exposing the serialization mechanism to client API: The last reason to customize serialization is perhaps the most important when it comes to designing your classes. The problem with implementing the default serialization is that in doing so we expose the internals of our class to the client. This is never a good idea. As such if we do a default serialization, clients might code their logic depending on our default serialization. Later when we think that we should serialize our class in a different way we cannot do it without potentially breaking our client’s code.

However if we customize our serialization, then we prevent the serialization mechanism from being a part of our public API. We can, whenever we want, change the way we do our serialization without breaking our client’s code. 

That's all for now.

In this and the previous article, we covered a bit of the Serializable interface and customizing serialization in Java. For readers who are interested in the full details of the Java serialization specification, I would redirect them to the official serialization specification:



Wednesday, September 7, 2011

Customizing Java Serializarion [Part 1]

In our last post we looked at some of the basics of Java Serialization. We also took a look at how object resolving is done by the JVM and the effects of serialization of the same object on different/same ObjectOutputStreams.

In this post we will look at how we can customize the process of serialization.

When we write a class and implement the Serializable interface, Java gives us a default serialization process. But sometimes we may want to customize the serialization process. There are several reasons why we might go for a customized serialization. Following is a not-an-exhaustive list:

  • Protect sensitive data.
  • Protect your invariants.
  • Control your instances.
  • Persist only meaningful data.
  • Manage serialization between different versions of your class.
  • Avoid exposing the serialization mechanism to client API.

In this part of the post we will look into the first three items.

Disclaimer: Some of the reasons for each of the above situations may seem argumentative. So it is best to view the below examples merely as illustrative purposes only and not as a reference to good programming practice. Basically the examples below depict the “how” of customizing serialization rather than the “when” of customizing serialization.

Protect Sensitive Data: Suppose we have a class UserAccount class that stores a username and password for an account. The class could be as follows:
public class UserAccount
{
         private String username;
         private String password;

         //other implementations
}
Now suppose we want to store all the details of the UserAccount instance except the password; then simply implement Serializable in the class will serialize the password as well. Hence we need to prevent the serialization of the password field. The way to do this is to introduce a method of the exact following signature into the class and do the serialization ourselves.
private void writeObject(ObjectOutputStream o) 
                       throws IOException, ClassNotFoundException
{
    //do serialization of required fields here
}

What is this method? Why is it private?

Well, this method is one of the several methods provided in the Java Serialization Specification that lets us customize the process of object serialization. If we put this method onto our class the JVM will call this method to do the serialization of the current object. Hence, to prevent the serialization of the password field our class will look like this:
class UserAccount implements Serializable
{
	public String username;
	public String password;   

	public UserAccount()
	{
       		username = "defaultUsername";
       		password = "defaultPassword";
	}   

	public UserAccount(String u, String p)
	{
       		username = u;
      		password = p;
	}   

	private void readObject(ObjectInputStream o) 
		throws IOException, ClassNotFoundException
	{
       		username = (String)o.readObject();
	}

	private void writeObject(ObjectOutputStream o) 
		throws IOException
	{
       		o.writeObject(username);
	}   

	public String toString()
	{
       		return username + ", " + password;
	}
}
In the above class we are skipping the serialization of the password field and just persisting the userName field. In addition, while de-serializing the object, we use a similar hook, readObject(), to read only the userName field. Note that trying to read the password field would have resulted in a java.io.OptionalDataException.

Protect your invariants: Suppose we have a class that represents a coordinate in the first quadrant. The class might look like this:
class Coordinate
{
	private int x;
	private int y;

	public Coordinate(int x, int y)
	{
		validateInvariants();       

		this.x = x;
		this.y = y;
	}

	public int getX()
	{
		return x;
	}

	public int getY()
	{
		return y;
	}  

	private void validateInvariants()
	{
		if(x < 0 || y < 0)
		{
           	throw new IllegalArgumentException();
		}
  	 }
}
Here the method validateInvariants throws an IllegalArgumentException if either of the coordinates are negative. Now if we want our Coordinate class to be serializable we should protect our invariants when an instance of the class is de-serialized. Hence after implementing Serializable, this is how we might do that:
class Coordinate implements Serializable
{
	private int x;
	private int y;
	
	public Coordinate(int x, int y)
	{
		validateInvariants();
		
		this.x = x;
		this.y = y;
	}

	public int getX()
	{
		return x;
	}

	public int getY()
	{
		return y;
	}
	
	private void readObject(ObjectInputStream o) 
		throws IOException, ClassNotFoundException
	{
		x = o.readInt();
		y = o.readInt();
		
		validateInvariants();
	}
	
	private void writeObject(ObjectOutputStream o) 
		throws IOException
	{
		o.writeInt(x);
		o.writeInt(y);
	}
	
	private void validateInvariants()
	{
		if(x < 0 || y < 0)
		{
			throw new IllegalArgumentException();
		}
	}
}
Here while de-serializing our instance we againg check for our invariants. If someone had modified the byte representation on the serialized form to introduce a negative value of the co-ordinates we would not have found it withoud the above readObject() method.

Control your instances: Suppose we have a class for which we want only a single instance of that class. Such instance are called Singletons. Singletons do not have any public constructors and clients get the reference to the instance via a static method normally named as getInstance. Consider the following class that is a singleton:

class Singleton
{
	private static final Singleton INSTANCE = new Singleton();

	private Singleton()
	{

	}

	public static Singleton getInstance()
	{
		return INSTANCE;
	}

	// other methods
}
Now suppose we want the above Singleton class to be serializable. We can implement the Serializable interface for the above class and be done with it. But in that case we won’t be able to protect the singleton nature of the instance, such that after de-serialization there will be more than one instance of the class. This can be proved as follows:
class Singleton implements Serializable
{
	private static final Singleton INSTANCE = new Singleton();

	private Singleton()
	{

	}

	public static Singleton getInstance()
	{
		return INSTANCE;
	}

	// other methods
}
Now I use the following snippet to prove that there will be multiple instances:
public final class ControlInstances
{
	public static void main(String[] args) throws Exception
	{
		ObjectOutputStream out = new ObjectOutputStream(
						new FileOutputStream(new File("out3.dat")));

		out.writeObject(Singleton.getInstance());
		out.close();

		ObjectInputStream in = new ObjectInputStream(
						new FileInputStream(new File("out3.dat")));

		Singleton u = (Singleton) in.readObject();
		in.close();

		System.out.println(Singleton.getInstance() == u);
	}
}
The above code prints false. This means that now after serialization and a subsequent de-serialization there are multiple instances of our supposedly singleton class. The way to avoid this is using another hook, the readResolve() method. The readResolve method is called when the  ObjectInputStream has read an object from the stream and is preparing to return it to the caller. ObjectInputStream checks whether the class of the object defines the readResolve method. If the method is defined, the readResolve method is called to allow any changes in the object before it is returned.

Hence our class definition will be:
class Singleton implements Serializable
{
	private static final Singleton INSTANCE = new Singleton();

	private Singleton()
	{

	}

	public static Singleton getInstance()
	{
		return INSTANCE;
	}

	private Object readResolve() throws ObjectStreamException
	{
		return getInstance();
	}

	// other methods
}
Now the test code will print true which means that even after de-serialization, we still have only one instance of the class in our JVM.

That is all for now. In part 2 of this post, we will look into the rest three items [Persist only meaningful data, Manage serialization between different versions of your class, Avoid exposing the serialization mechanism to client API] and some best practices that should be followed when we use the Serialization API in our applications.

Happy Coding! 

Tuesday, August 23, 2011

Serialization in Java

Java provides its developers with a rich set of APIs for object serialization. In this article we will look into some of the intricacies of object serialization in Java.

We shall start with the most basic example of how to serialize an object in Java; here are the high level steps:

  • Step 1: Get the object to serialize [can be null]. Let the object be o.
  • Step 2: Get a valid ObectOutputStream instance. Let the ObjectOutputStream instance be out.
  • Step 3: out.writeObject(o); will serialize the object to the stream.

Let us see the above steps in terms of a code snippet:
String s = new String(); //the object that we will serialize
ObjectOutputStream out = new ObjectOutputStream(
        new FileOutputStream(“out.dat”));
out.writeObject(s); //serialize the object
Now let us look at some of the basic guidelines that should be followed for Object serialization:

1. Any class that wants its objects to be serialized must implement the Serializable interface. The Serializable interface does not have any methods. It merely serves to flag an object as serializable to an ObjectOutputStream.

2. If there any members in a Serializable class, then the following guidelines apply:

     i. If they are primitives, they are automatically serializable.

    ii. If they are non-primitive objects, they must implement Serializable. If we try to serialize an object that contains reference to an object that does not implement Serializable then while serializing the object, we get a Runtime Exception.

    iii. If we have a reference to a non-serializable object in our class, then we have to mark the reference with the keyword transient. The transient keyword on a reference means that when the parent object is serialized then the object whose reference is marked as transient will not be serialized.

Let us take a look at a simple code snippet that illustrates the above guidelines:
// a non-serializable class
public class Model
{
    private Integer modelID;
    private String modelName;
    //rest of the implementations
}

// a serializable class
public class Engine implements Serializable
{
    private int engineID;
    private String engineName;
    // other implementations
}

public class Car implements Serializable
{
    //a primitive, hence serializable
    private int carID; 

    //a non-serializable object, hence transient
    private transient Model carModel; 

    //a serializable object, hence no transient
    private Engine carEngine; 
    // other implementations
}
Now when we try to serialize an instance of a Car object, there will be no exceptions thrown because all the members of the car object are either primitives, or implement Serializable or are marked with the keyword transient.

Note that in our Car declaration if we had not marked our Model object as transient, we would have got a RuntimeException while trying to serialize the Car object.

The above example was a very basic one to show how to serialize an object in Java. Now let us look under the hood as to how Java resolves objects during serialization. To state the problem, consider the following class declarations:

public class GearType implements Serializable
{
    private int ID;
    private String gearName;
    //other implementations;
}

public class Car implements Serializable
{
    private int ID;
    private GearType gearType
    public Car(int i, GearType g)
    {
        this.ID = i;
        this.gearType = g;
    }
}

Now let us serialize two Car objects:

GearType g = new GearType();
Car c1 = new Car(1, g);
Car c2 = new Car(2, g);
ObjectOutputStream out = new ObjectOutputStream(
        FileOutputStream(“out.dat”));
out.writeObject(c1);
out.writeObject(c2);
out.close();

In the above code snippet, we serialized two Car objects [note the usage of the same GearType object to construct two different Car objects]. The interesting question is: how many GearType objects were serialized?

There was only one GearType object that both the Car objects were sharing. Hence it should only serialize one GearType object. The answer indeed is one. This can be proved by checking if the serialized GearType object is the same for both the Car objects. Let us look into that:
ObjectInputStream in = new ObjectInputStream(
        new FileInputStream(“out.dat”));
Car first = (Car) in.readObject();
Car second = (Car) in.readObject();
System.out.println(first.getGearType() == second.getGearType());

The above code does print true. Note that here we are testing for object identity of the GearType object instead of logical equality [which is done via equals() method]. The identity test is required because we want to verify whether both the GearType objects are actually the same.

While performing serialization of objects, Java forms a data structure similar to an Object Graph to determine which objects need to be serialized. It starts from the main object to serialize, and recursively traverses all the objects reachable from the main object. For each object that it encounters, which needs serialization, it associates an identifier that marks the object as already been serialized to the given ObjectOutputStream instance. So when Java encounters the same object that has already been marked as serialized to the ObjectOutputStream, it does nor serialize the object again, rather a handle to the same object is serialized. This is how Java avoids having to re-serialize an already serialized object. The seemingly complex problem was solved by the simple method of assigning IDs to objects. Beautiful!

One important thing to note is that if we used different ObjectOutputStream instances to serialize the two Car objects, then Java would have serialized the same GearType object twice albeit in the different streams. This is because the first time Java marks the GearType object with an ID, that ID will associate the object to the first ObjectOutputStream and the next time when Java encounters the same object for serialization it sees that this object has not been serialized to the current ObjectOutputStream and hence it will serialize it again to the new stream.

Let us illustrate this via a code snippet:
GearType g = new GearType();
Car c1 = new Car(1, g);
Car c2 = new Car(2, g);
ObjectOutputStream out1 = new ObjectOutputStream(
        FileOutputStream(“out1.dat”));
ObjectOutputStream out2 = new ObjectOutputStream(
        FileOutputStream(“out2.dat”));

out1.writeObject(c1);
out2.writeObject(c2);
out1.close();
out2.close();
Now to prove that the GearType object is indeed serialized twice, we read the streams back:

ObjectInputStream in1 = new ObjectInputStream(
        new FileInputStream(“out1.dat”));
ObjectInputStream in2 = new ObjectInputStream(
        new FileInputStream(“out2.dat”));
Car first = (Car) in1.readObject();
Car second = (Car) in2.readObject();
System.out.println(first.getGearType() == second.getGearType());

Now the above code prints false, which proves that the GearType object was serialized twice.

That is all for now. We have barely touched the surface of serialization in Java. In the next article [hopefully soon] we will take a look at other aspects of serialization like, customizing serialization, protecting classes from serialization [specifically protecting your singletons], class versioning problems etc.

Happy coding!