Tuesday, June 21, 2011

Test driven development with SmartGWT and JPA

SmartGWT is a collection of GWT widgets that can easily be connected to database tables or some other data on the server.

SmartGWT Pro supports Hibernate/JPA on the server, but there are some limitations, and unit testing it is tricky.

This article shows how to implement and test server logic with SmartGWT and JPA. Specifically, it will show how to:
  • Configure server.properties for unit testing.
  • Handle JPA relations with SmartGWT.
  • Implement an EMF provider so the unit tests and the data sources share the same EntityManager.
  • Use special JPA queries such as <= and string matching in SmartGWT.
  • Build complete server logic with pagination and sorting for JPA.

Examples

Before we get into the practicalities of SmartGWT, we need to design good tests. Many who start with test driven development find this very hard. My approach is to test examples of application behavior. If you discuss these examples with the customer first, you can implement automatic tests that are directly based on the requirements.

Let's say I wanted to build an online auction application. I would start by asking the client for some examples of what the application should do. The client might answer that sellers may place items for auction, bidders place bids, and the seller then sells to the highest bidder. This is not what I mean with examples, these are abstract requirements. I want concrete examples with realistic input and output values. I mean something like this:

Jane wants to sell an electric guitar. She enters the following information:


Bob wants to buy an electric guitar. He searches for auctions, clicks on Electric guitar, and places a bid.





Jane can then see the bids in her auction:



The advantages of using examples are that they are detailed, they reveal a lot of misunderstandings and they can be tested.

Objects

Another thing that many struggle with is to design a good domain model, be it a class model or a database schema. The approach shown here makes this easy. Instead of starting with a class diagram, I start by looking for objects in the examples. Then I derive the class model from the objects.

An example connected to objects is called an object case.

Jane wants to sell an electric guitar

For each example, I need to determine how to handle the input from the user and how to calculate the output. In this example, I will create a User object and an Auction object. The arrows show that all the input from the user is stored in these objects.

Bob searches for auctions

In this example, the Free_text and Max_price input is used to search for Auctions, but the Category is not used. This means that something is missing in the object model.

I decide to introduce a new object that will be used to search for category. Now, all input values are used and the output in the table is calculated from the Auction objects, so everything is covered.

Bob clicks on Electric guitar

All the output values can be calculated from an Auction object. We should define another example that shows an actual bid.


Bob places a bid

The input values are stored in two new objects: User and Bid. What happens if the user already exists? We should add another example to deal with this.


Classes

All input and output values are now connected to objects. This is a simple and powerful method to design an object model. Based on these objects, I can create the following class diagram:


Find more functionality

We can use the class diagram to find missing functionality. The diagram below shows how objects are created and read by the examples defined so far.


What is missing?
  • Create Category objects. Should an adminstrator to this or should it happen automatically? This needs to be discussed with the client.
  • Set link between Category and Auction. This should be done by Create auction.
  • Read User objects. What do we need the email for?
  • Read Bid objects. This should be part of Auction details. We should define an example to show this.
  • Update attributes. This needs to be discussed with the client.
  • Delete objects. We need to discuss with the client if this should be possible.

Based on this analysis, I add a Category parameter to the first example and use that parameter to find an existing Category object. I create a link between this Category and the new Auction:

Implement JPA classes

It's straight forward to implement the classes as JPA entities.
@Entity @Table(name="tb_auction")
public class Auction {
    @Id @GeneratedValue(strategy=GenerationType.AUTO)
    private Long id;

    private String title;
    private float startingPrice;
    private String description;
 
    @ManyToOne(optional=false)
    private Category category;

    @ManyToOne(optional=false)
    private User seller;

    @OneToMany(mappedBy="auction")
    private Set<Bid> bids = new HashSet<Bid>();
}

@Entity @Table(name="tb_bid")
public class Bid {
    @Id @GeneratedValue(strategy=GenerationType.AUTO)
    private Long id;

    private float bid;

    @ManyToOne(optional=false)
    private Auction auction;
 
    @ManyToOne(optional=false)
    private User bidder;
}

@Entity @Table(name="tb_category")
public class Category {
    @Id @GeneratedValue(strategy=GenerationType.AUTO)
    private Long id;

    private String name;
    
    @OneToMany(mappedBy="category")
    private Set<Auction> auctions = new HashSet<Auction>();
}

@Entity @Table(name="tb_user")
public class User {
    @Id @GeneratedValue (strategy=GenerationType.AUTO)
    private Long id;

    private String email;
    
    @OneToMany(mappedBy="seller")
    private Set<Auction> auctions = new HashSet<Auction>();

    @OneToMany(mappedBy="bidder")
    private Set<Bid> bids = new HashSet<Bid>();
}
(Getters and setters removed for clarity.)

Implement tests

Each example is a test. I divide each test into 4 parts:
  1. Initial objects
  2. Input
  3. Output
  4. Result objects

With SmartGWT and JPA this becomes:
  1. Initial objects: Create objects in the database. The best way to do this in a unit test is to use a HSQLDB in-memory database, which can create the database schema on the fly.
  2. Input: Call a SmartGWT datasource with the specified parameters.
  3. Output: Check the output from the datasource.
  4. Result objects: Check the resulting objects in the database.

Let's start with testing the first example: Jane wants to sell an electric guitar.


public class AuctionTest extends TestCase {
 private DataSource auctionDS;
 private DataSource bidDS;
 
 private HashMap<String,Object> input = new HashMap<String,Object>();

 @Override
 protected void setUp() throws Exception {
  super.setUp();
  auctionDS = DataSourceManager.get("Auction");
  bidDS = DataSourceManager.get("Bid");
 }

 public void testJaneWantsToSellAnElectricGuitar() throws Exception {
  // Initial objects:
  category1 = new Category();
  category1.setName("Musical instruments");
  persist(category1); 

  // Input:
  input.put("title",       "Electric guitar for beginners");
  input.put("startingPrice",  150f);
  input.put("description", "Nice electric guitar with amplifier.");
  input.put("email",       "jane99@abc123.com");
  input.put("categoryId",  category1.getId());
  auctionDS.add(input);

  // Output: None.

  // Result objects:
  List users = getResultObjects(User.class);
  assertEquals(1, users.size());
  assertEquals("jane99@abc123.com", users.get(0).getEmail());

  List auctions = getResultObjects(Auction.class);
  assertEquals(1, auctions.size());
  Auction auction = auctions.get(0);
  assertEquals("Electric guitar for beginners",        auction.getTitle());
  assertEquals(150f,                                   auction.getStartingPrice());
  assertEquals("Nice electric guitar with amplifier.", auction.getDescription());
  assertEquals(category1.getId().intValue(),           auction.getCategoryId());
 }

 private void persist(Object... objects) throws Exception {
  EntityManager em = EMF.getEntityManager();
  Object tx = EMF.getTransaction(em);
  for(Object obj : objects) {
   em.persist(obj);
  }
  EMF.commitTransaction(tx);
 }

 private <T> List getResultObjects(Class<T> clazz) throws Exception {
  EntityManager em = EMF.getEntityManager();
  Object tx = EMF.getTransaction(em);
  List<T> result = em.createQuery("from " + clazz.getName()).getResultList();
  EMF.commitTransaction(tx);
  return result;
 }
}

Here is the second example: Bob wants to buy an electric guitar. This is more complicated with some initial objects and several steps.




public void testBobWantsToBuyAnElectricGuitar() throws Exception {
  initialObjects2Auctions();
  
  // Search for auctions.
  input.put("categoryId", category1.getId());
  input.put("title",      "guitar");
  input.put("maxPrice",   250f);
  
  List<Auction> output = auctionDS.fetch(input);
  assertEquals(2, output.size());
  assertEquals("Guitar",          output.get(0).getTitle());
  assertEquals(200f,              output.get(0).getStartingPrice());
  assertEquals("Electric guitar", output.get(1).getTitle());
  assertEquals(150f,              output.get(1).getStartingPrice());

  // Auction details.
  input.clear();
  input.put("id", auction1.getId());
  
  output = auctionDS.fetch(input);
  assertEquals(1, output.size());
  Auction auction = output.get(0);
  assertEquals("Electric guitar for beginners",        auction.getTitle());
  assertEquals("Nice electric guitar with amplifier.", auction.getDescription());
  assertEquals(150f,                                   auction.getStartingPrice());
  assertEquals("No bids.",                             auction.getHighestBid());

  // Place bid.
  input.clear();
  input.put("auctionId",   auction1.getId());
  input.put("bidderEmail", "bob@abc999.com");
  input.put("bid",         150);
  ErrorReport errors = bidDS.validate(input, true);
  assertEquals(null, errors);
  bidDS.add(input);

  // Result objects:
  List<User> users = getResultObjects(User.class);
  assertEquals(1, users.size());
  User user = users.get(0);
  assertEquals("bob@abc999.com", user.getEmail());
  List<Bid> bids = getResultObjects(Bid.class);
  assertEquals(1, bids.size());
  Bid bid = bids.get(0);
  assertEquals(150,              bid.getBid());
  assertEquals(user.getId(),     bid.getBidder().getId());
  assertEquals(auction1.getId(), bid.getAuction().getId());
 }
 
 private void initialObjects2Auctions() throws Exception {
  category1 = new Category();
  category1.setName("Musical instruments");
  persist(category1);

  auction1 = initialAuction(category1, "Electric guitar for beginners", "Nice electric guitar with amplifier.", 150f);
  auction2 = initialAuction(category1, "Guitar", "Used guitar.", 250f);
  persist(auction1, auction2);
 }

 private Auction initialAuction(Category category, String title, String description, float startingPrice) {
  Auction object = new Auction();
  object.setTitle(title);
  object.setDescription(description);
  object.setStartingPrice(startingPrice);
  category.addAuction(object);  
  return object;
 }

Let's add some more tests while we're at it:

public void testNoSearchCriteria() throws Exception {
  initialObjects2Auctions();
  assertEquals(2, auctionDS.fetch(input).size());
 }

 public void testSearchForUnknownCategory() throws Exception {
  initialObjects2Auctions();
  input.put("categoryId", category1.getId() + 1);
  assertEquals(0, auctionDS.fetch(input).size());
 }

 public void testSearchForMaxPrice150() throws Exception {
  initialObjects2Auctions();
  input.put("maxPrice", 150f);
  assertEquals(1, auctionDS.fetch(input).size());
 }

 public void testSearchForMaxPrice250() throws Exception {
  initialObjects2Auctions();
  input.put("maxPrice", 250f);
  assertEquals(2, auctionDS.fetch(input).size());
 }

 public void testSearchForTitlePiano() throws Exception {
  initialObjects2Auctions();
  input.put("title", "piano");
  assertEquals(0, auctionDS.fetch(input).size());
 }

 public void testSearchForTitleElectricGuitar() throws Exception {
  initialObjects2Auctions();
  input.put("title", "electric guitar");
  assertEquals(1, auctionDS.fetch(input).size());
 }

These tests will actually fail, but that's ok. When I fix them, I will learn much about the behavior.

The first thing I notice, is a compilation error because of a missing method, so let's add it:
public class Auction {
 public String getHighestBid() {
  Float highestBid = null;
  for(Bid bid : getBids()) {
   if(highestBid == null || bid.getBid() > highestBid) {
    highestBid = bid.getBid();
   }
  }
  return (highestBid != null ? highestBid.toString() : "No bids.");
 }
 ...

This can instead be handled by a database query in the "serverObject" that will be implemented later in this article.

To use HSQLDB for unit testing, add hsqldb.jar to the project and create a persistence.xml file in the test/META-INF directory with the following properties:
<property name="hibernate.connection.url" value="jdbc:hsqldb:mem:auction"/>
<property name="hibernate.hbm2ddl.auto" value="create-drop"/>

Implement datasources

The tests use the datasources "Bid" and "Auction" that I haven't defined yet. Here is a starting point:
<DataSource ID="Auction"
    serverConstructor="com.isomorphic.jpa.JPADataSource"
    beanClassName="com.objectgeneration.auctions.Auction">
    <fields>
        <field name="id"          type="sequence" hidden="true"       primaryKey="true" />
        <field name="title"       type="text"     title="Title"       required="true"   />
        <field name="description" type="text"     title="Description" required="true" />
        <field name="categoryId"  type="integer"  title="Category"    canEdit="false" />
    </fields>
</DataSource>

<DataSource ID="Bid"
    serverConstructor="com.isomorphic.jpa.JPADataSource"
    beanClassName="com.objectgeneration.auctions.Bid">
    <fields>
        <field name="id"   type="sequence" hidden="true" primaryKey="true"/>
        <field name="bid"  type="number"   title="Bid"   required="true"/>
    </fields>
</DataSource>

Using SmartGWT and JPA from JUnit

I am getting eager to implement the behavior, but unfortunately I have to fix some issues to get SmartGWT to work with JPA in JUnit first. Let's start:


That didn't go so well. Here is a stack trace:
java.lang.NullPointerException
 at com.isomorphic.io.ISCFile.<init>(ISCFile.java:145)
 ...
 at com.isomorphic.datasource.DataSourceManager.get(DataSourceManager.java:68)
 at com.objectgeneration.auctions.AuctionTest.setUp(AuctionTest.java:31)

The problem here is that SmartGWT thinks it is running as a servlet, but it does not find its web root. This can be configured in server.properties. Create a server.properties file for unit testing in the test folder. The content should be similar to the production version, except for the following:
# This is user specific. Use Ant filtering to generate this.
webRoot: /Users/lars/temp/auctions/war/

jpa.emfProvider: com.isomorphic.jpa.EMFProviderLMT

# Name of the datasource (from persistence.xml)
jpa.persistenceUnitName: ds
Also make sure the test source has a separate output folder (see Eclipse: Project > Properties > Java Build Path > Source). Otherwise the server.properties from src and test will overwrite each other in the output directory with random results.

Running JUnit again, we get the next problem:
java.lang.ClassCastException: java.util.ArrayList cannot be cast to java.util.Map
 at com.isomorphic.datasource.BasicDataSource.buildFieldData(BasicDataSource.java:426)
 ...
 at com.isomorphic.datasource.DataSourceManager.get(DataSourceManager.java:68)
 at com.objectgeneration.auctions.AuctionTest.setUp(AuctionTest.java:31)
This can be fixed by another setting in test/server.properties:
isomorphicPathRootRelative: Auctions_js/sc
(It should point to the sub-directory of war/ where initsc.js is created.)

JPA relations and SmartGWT


Trying again I get this stack trace:
javax.persistence.PersistenceException: org.hibernate.PropertyValueException: not-null property references a null or transient value: com.smartgwt.sample.server.Auction.category
 at org.hibernate.ejb.AbstractEntityManagerImpl.throwPersistenceException(AbstractEntityManagerImpl.java:637)
 at org.hibernate.ejb.AbstractEntityManagerImpl.persist(AbstractEntityManagerImpl.java:226)
 at com.isomorphic.jpa.JPADataSource.executeAdd(JPADataSource.java:452)
 at com.isomorphic.datasource.DataSource.execute(DataSource.java:1050)
 at com.isomorphic.jpa.JPADataSource.execute(JPADataSource.java:218)
 …
 at com.isomorphic.datasource.DataSource.add(DataSource.java:1948)
 at com.objectgeneration.auctions.AuctionTest.testJaneWantsToSellAnElectricGuitar(AuctionTest.java:45)
Caused by: org.hibernate.PropertyValueException: not-null property references a null or transient value: com.smartgwt.sample.server.Auction.category
SmartGWT does not handle JPA relations like "Category category" in class Auction, it only handles foreign keys like "long categoryId". But I want to use JPA relations. The workaround is to have both and define only the foreign key in the datasource as above. The Java code is as follows:
public class Auction {
 @ManyToOne(optional=false)
 @JoinColumn(name="category", nullable=false, insertable=false, updatable=false)
 private Category category;
 
 @Column(name="category")
 private long categoryId;
...
}
The category column is mapped to 2 Java variables, but only one of them can be writable. Both of them need to be set when creating the initial objects:
public class Category {
 public void addAuction(Auction auction) {
  auction.setCategory(this);
  getAuctions().add(auction);
  assert getId() != null;
  auction.setCategoryId(getId());
 }
...
}
Trying again:
javax.persistence.EntityExistsException: org.hibernate.exception.ConstraintViolationException: could not insert: [com.smartgwt.sample.server.Auction]
 …
 at com.isomorphic.datasource.DataSource.add(DataSource.java:1948)
 at com.objectgeneration.auctions.AuctionTest.testJaneWantsToSellAnElectricGuitar(AuctionTest.java:45)
Caused by: org.hibernate.exception.ConstraintViolationException: could not insert: [com.smartgwt.sample.server.Auction]
 ...
Caused by: java.sql.SQLException: Integrity constraint violation - no parent FKD896457230F353B7 table: TB_USER in statement [insert into tb_auction (id, category, description, seller, startingPrice, title) values (null, ?, ?, ?, ?, ?)]
Now we are finally getting to the business logic. I cannot create an Auction without a User.

I decide to create a new User datasource and add a User before creating the Auction:
public void testJaneWantsToSellAnElectricGuitar() throws Exception {
  // Initial objects:
  category1 = new Category();
  category1.setName("Musical instruments");
  persist(category1);

  // Input:
  long userId = createUser("jane99@abc123.com");
  input.put("title",       "Electric guitar for beginners");
  input.put("startingPrice",  150f);
  input.put("description", "Nice electric guitar with amplifier.");
  input.put("categoryId",  category1.getId());
  input.put("sellerId",    userId);
  auctionDS.add(input);

  ...
 }

 private long createUser(String email) throws Exception {
  HashMap<String,object> input = new HashMap<String,object>();
  input.put("email", email);
  ErrorReport errors = userDS.validate(input, true);
  assertEquals(null, errors);
  DSRequest dsRequest = new DSRequest(userDS.getName(), DataSource.OP_ADD);
  dsRequest.setValues(input);
  DSResponse dsResponse = dsRequest.execute();
  return ((User)dsResponse.getData()).getId();
 }
I also connect the initial Auction objects in the other unit tests to a User. This makes some of the tests pass:


Many of these failures are because the tests are not isolated. If I run each test separately, many of them will pass. The reason for this is that the database is created when the first test is run, and then the next test uses the same database with the objects that are left. I need to empty the database between each test. This is easy to do in JPA; just call EntityManagerFactory.close(). But I don't have access to the EntityManagerFactory that SmartGWT uses. To fix this, I need to implement my own "EMF provider" and point to it in test/server.properties:
jpa.emfProvider: com.objectgeneration.auctions.server.MyEMFProvider

And here is the implementation:
import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.EntityTransaction;
import javax.persistence.Persistence;

import com.isomorphic.jpa.EMFProviderInterface;

public class MyEMFProvider implements EMFProviderInterface {
 private static EntityManagerFactory factory;
 
 // Support multi-threaded tests.
 private static ThreadLocal<Entitymanager> entityManager = new ThreadLocal<Entitymanager>();
 
 public MyEMFProvider() {
 }
 
 /** Close the factory after each unit test so it starts with an empty database next time. */
 public static void close() {
  if(entityManager.get() != null) {
   entityManager.get().close();
   entityManager.set(null);
  }
  if(factory != null) {
   factory.close();
   factory = null;
  }
 }

 protected EntityManagerFactory createFactory() {
  return Persistence.createEntityManagerFactory("ds");
 }

 public EntityManagerFactory get() {
  if(factory == null) {
   factory = createFactory();
  }
  return factory;
 }

 public EntityManager getEntityManager() {
  if(entityManager.get() == null) {
   entityManager.set(get().createEntityManager());
  }
  return entityManager.get();
 }

 public void returnEntityManager(EntityManager em) {
  // Do nothing.
 }
 
 public Object getTransaction(EntityManager em) {
  EntityTransaction tx = em.getTransaction();
  if(!tx.isActive()) {
   tx.begin();
  }
  return tx;
 }

 public void commitTransaction(Object obj) {
  EntityTransaction tx = (EntityTransaction) obj;
  if(tx.isActive()) {
   tx.commit();
  }
 }
 
 public void rollbackTransaction(Object obj) {
  EntityTransaction tx = (EntityTransaction) obj;
  if(tx.isActive()) {
   tx.rollback();
  }
 }
}
Add the following to the test class to start with an empty database in each unit test:
protected void tearDown() throws Exception {
  MyEMFProvider.close();
  super.tearDown();
}
This takes us much closer to our goal, and we can continue with the business logic:


Implement datasource

There is no wonder that maxPrice is not working, the Auction class does not even have this field. I want to find all Auctions where Auction.startingPrice <= the maxPrice parameter. To implement this, I need to implement special handling for this data source in a "serverObject":
<DataSource ID="Auction"
    serverConstructor="com.isomorphic.jpa.JPADataSource"
    beanClassName="com.smartgwt.sample.server.Auction">
    <serverObject className="com.objectgeneration.auctions.AuctionRequestHandler"/>
    <fields>
        <field name="id"          type="sequence" hidden="true"   primaryKey="true" />
        <field name="title"       type="text"     title="Title"       required="true"   />
        <field name="description" type="text"     title="Description" required="true" />
        <field name="categoryId"  type="integer"  title="Category"    canEdit="false" />
        
        <!-- This field does not exist in the Auction class. It is implemented in AuctionRequestHandler. -->
        <field name="maxPrice"    type="number"   title="Max price" hidden="true" />
    </fields>
</DataSource>

The serverObject in the XML above refers to a class that will be called instead of the default SmartGWT handler, and there I can handle maxPrice:
import java.util.List;
import java.util.Map;

import javax.persistence.EntityManager;
import javax.persistence.Query;

import org.apache.log4j.Logger;

import com.isomorphic.datasource.DSRequest;
import com.isomorphic.datasource.DSResponse;
import com.isomorphic.jpa.EMF;
import com.smartgwt.sample.server.Auction;

public class AuctionRequestHandler {
 private static final Logger log = Logger.getLogger(AuctionRequestHandler.class);
 
 public DSResponse fetch(DSRequest dsRequest) throws Exception {
  Map<String,Object> criteria = dsRequest.getCriteria();
  log.info("fetch(" + criteria + ")");
  
  String queryString = "from Auction";
  String separator = " where ";
  for(String paramName : criteria.keySet()) {
   queryString += separator;
   if(paramName.equals("maxPrice")) {
    queryString += "startingPrice <= :" + paramName;
   } else {
    queryString += paramName + " = :" + paramName;
   }
   separator = " and ";
  }
  
  EntityManager em = EMF.getEntityManager();
  Query query = em.createQuery(queryString);
  for(String paramName : criteria.keySet()) {
   Object paramValue = criteria.get(paramName);
   query.setParameter(paramName, paramValue);
  }
  List<Auction> result = query.getResultList();
  return new DSResponse(result, DSResponse.STATUS_SUCCESS);
 }
}

I want to search for title with substrings and ignore case. Substrings is built in to SmartGWT, but not ignore case. Here is an updated version with special handling for the title field:
public DSResponse fetch(DSRequest dsRequest) throws Exception {
  Map<String,Object> criteria = dsRequest.getCriteria();
  log.info("fetch(" + criteria + ")");
  
  String queryString = "from Auction";
  String separator = " where ";
  for(String paramName : criteria.keySet()) {
   Object paramValue = criteria.get(paramName);
   queryString += separator;
   if(paramName.equals("maxPrice")) {
    queryString += "startingPrice <= :" + paramName;
   } else if(paramName.equals("title")) {
    queryString += "UPPER(" + paramName + ") LIKE '%" + ((String)paramValue).toUpperCase() + "%'";
   } else {
    queryString += paramName + " = :" + paramName;
   }
   separator = " and ";
  }
  
  EntityManager em = EMF.getEntityManager();
  Query query = em.createQuery(queryString);
  for(String paramName : criteria.keySet()) {
   if(!paramName.equals("title")) {
    Object paramValue = criteria.get(paramName);
    query.setParameter(paramName, paramValue);
   }
  }
  List result = query.getResultList();
  return new DSResponse(result, DSResponse.STATUS_SUCCESS);
 }

Success!

Wrapping it up

One improvement could be to move the getHighestBid() logic to a database query. The AuctionRequestHandler could call this query and store the value in a @Transient field in the Auction class to send it to the client, or it could send HashMaps instead of Auction objects.

We also need to handle the things that SmartGWT does by default: Transactions, pagination and sorting. (I believe that if you don't implement pagination and sorting in the server, the client will handle it instead. This may be acceptable if you don't have thousands of rows in that database table. Please refer to the SmartGWT documentation.)

A complete solution with a general superclass and special handling in a subclass looks like this:
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;

import javax.persistence.EntityManager;
import javax.persistence.EntityTransaction;
import javax.persistence.Query;

import org.apache.log4j.Logger;

import com.isomorphic.datasource.DSRequest;
import com.isomorphic.datasource.DSResponse;
import com.isomorphic.jpa.EMF;

public class RequestHandler {
 private static final Logger log = Logger.getLogger(RequestHandler.class);

 public DSResponse fetch(DSRequest dsRequest) throws Exception {
  return handleRequest(dsRequest);
 }
 
 public DSResponse add(DSRequest dsRequest) throws Exception {
  return handleRequest(dsRequest);
 }

 public DSResponse remove(DSRequest dsRequest) throws Exception {
  return handleRequest(dsRequest);
 }
 
 public DSResponse update(DSRequest dsRequest) throws Exception {
  return handleRequest(dsRequest);
 }

 protected DSResponse handleRequest(DSRequest dsRequest) throws Exception {
  EntityManager em = EMF.getEntityManager();
  EntityTransaction tx = (EntityTransaction) EMF.getTransaction(em);
  try {
   DSResponse dsResponse = handleInsideTransaction(dsRequest);
   tx.commit();
   return dsResponse;
  } catch(Exception e) {
   MyEMFProvider.rollback(tx);
   throw e;
  } finally {
   MyEMFProvider.finish();
  }
 }

 protected DSResponse handleInsideTransaction(DSRequest dsRequest) throws Exception {
  return dsRequest.execute();
 }

 @SuppressWarnings("unchecked")
 protected <T> DSResponse query(EntityManager em, DSRequest dsRequest, Class<T> clazz) {
  String queryString = "from " + clazz.getName();

  LinkedHashMap<String, Object> parameters = new LinkedHashMap<String, Object>();
  queryString = addCriteria(queryString, dsRequest.getCriteria(), parameters);
  
  List<T> result = selectObjects(em, dsRequest, queryString, parameters);
  long totalRows = countObjects(em, dsRequest, queryString, parameters, result);

  DSResponse dsResponse = new DSResponse(result, DSResponse.STATUS_SUCCESS);
  dsResponse.setStartRow(dsRequest.getStartRow());
  dsResponse.setEndRow(dsRequest.getStartRow() + result.size());
  dsResponse.setTotalRows(totalRows);
  return dsResponse;
 }

 private String addCriteria(String queryString, Map<String, Object> criteria, Map<String, Object> parameters) {
  String separator = " where ";
  for(String paramName : criteria.keySet()) {
   Object paramValue = criteria.get(paramName);
   queryString += separator + addCriterium(paramName, paramValue, parameters);
   separator = " and ";
  }
  return queryString;
 }

 /** Override this method for special handling of query parameters. */
 protected String addCriterium(String paramName, Object paramValue, Map<String, Object> parameters) {
  parameters.put(paramName, paramValue);
  return paramName + " = :" + paramName;
 }

 private <T> List<T> selectObjects(EntityManager em, DSRequest dsRequest, String queryString, Map<String, Object> parameters) {
  queryString = addOrderBy(queryString, dsRequest.getSortBy());
  Query query = em.createQuery(queryString);
  addPagination(dsRequest, query);
  setQueryParameters(query, parameters);
  return query.getResultList();
 }

 private String addOrderBy(String queryString, String sortBy) {
  if(sortBy != null) {
   if(sortBy.startsWith("-")) {
    queryString += " ORDER BY " + sortBy.substring(1) + " DESC";
   } else {
    queryString += " ORDER BY " + sortBy + " ASC";
   }
  }
  return queryString;
 }

 private void addPagination(DSRequest dsRequest, Query query) {
  long startRow = dsRequest.getStartRow();
  if(startRow >= 0) {
   query.setFirstResult((int)startRow);
  }
  
  long endRow = dsRequest.getEndRow();
  if(endRow > startRow) {
   query.setMaxResults((int)endRow - (int)startRow);
  }
 }

 private long countObjects(EntityManager em, DSRequest dsRequest, String queryString, Map<String, Object> parameters, List result) {
  long totalRows;
  long startRow = dsRequest.getStartRow();
  long endRow = dsRequest.getEndRow();
  if(startRow < 0 || endRow <= startRow) {
   totalRows = result.size();
   log.debug("no pagination, totalRows=" + totalRows);
  } else if(result.size() < endRow - startRow) {
   totalRows = result.size();
   log.debug("all rows received, totalRows=" + totalRows);
  } else {
   queryString = "select count(*) " + queryString;
   log.debug("countObjects: queryString=" + queryString + ", parameters=" + parameters);
   Query countQuery = em.createQuery(queryString);
   setQueryParameters(countQuery, parameters);
   Object countResult = countQuery.getSingleResult();
   log.debug("countObjects: result=" + countResult);
   totalRows = (Long) countResult;
  }
  return totalRows;
 }

 private void setQueryParameters(Query query, Map<String,Object> parameters) {
  for(String paramName : parameters.keySet()) {
   Object paramValue = parameters.get(paramName);
   query.setParameter(paramName, paramValue);
  }
 }
}


import java.util.Map;

import com.isomorphic.datasource.DSRequest;
import com.isomorphic.datasource.DSResponse;
import com.isomorphic.datasource.DataSource;

public class AuctionRequestHandler extends RequestHandler {
 @Override
 protected DSResponse handleInsideTransaction(DSRequest dsRequest) throws Exception {
  if(dsRequest.getOperationType().equals(DataSource.OP_FETCH)) {
   return query(dsRequest, Auction.class);
  } else {
   return dsRequest.execute();
  }
 }
 
 @Override
 protected String addCriterium(String paramName, Object paramValue, Map<String, Object> parameters) {
  if(paramName.equals("maxPrice")) {
   parameters.put(paramName, (Float) paramValue);
   return "startingPrice <= :" + paramName;
  } else if(paramName.equals("title")) {
   return "UPPER(" + paramName + ") LIKE '%" + ((String)paramValue).toUpperCase() + "%'";
  } else {
   return super.addCriterium(paramName, paramValue, parameters);
  }
 }
}

Implement user interface

When the server logic works, it is relatively straight forward to implement the user interface. The code looks something like this:
    public void onModuleLoad() {
        final DataSource auctionDS = DataSource.get("Auction");
        final DataSource bidDS = DataSource.get("Bid");

        final ListGrid auctionGrid = new ListGrid();
        auctionGrid.setDataSource(auctionDS);
        auctionGrid.setAutoFetchData(true);

        final ListGrid bidGrid = new ListGrid();
        bidGrid.setDataSource(bidDS);
        bidGrid.setAutoFetchData(false);
        ...
    }
That's all SmartGWT needs to fetch data from the server when needed.

Conclusion

Is it really worth all this effort? I would say yes. It is a lot less work than it would be to implement your own server logic, and in many cases you can just use the default SmartGWT data sources. If you like SQL better than JPA, you can use SmartGWT SQL datasources, wich are more flexible than the JPA, but then you cannot use this unit test framework.

This testing methodology has saved us lots of work. It is far cheaper to test the logic in unit tests than to test it manually, fix problems, and restart the application to test again. With this kind of unit tests, most of the server logic just works. The remaining work is to implement and fine-tune the user interface. And of course, the unit tests keep our code working.

It is actually not just a testing methodology but a requirements methodology also. Using examples and objects makes the communication between clients/analysts and developers crystal clear. It reveals misunderstanding and missing details so that the developers get all the information they need to test and implement new functionality.

Object cases make it easy to design tests, and the tests makes it easy to focus development and figure out what to do next.

Thursday, June 2, 2011

Back to the agile values

In recent years the term agile has become overused. Many seem to think that if they have have unit tests, standup meetings and burn-down charts, they're agile. All these practices are good, but they don't necessarily make you agile. Even iterations or some kind of certified master don't necessarily make you agile.

So what is agile? If I were to sum it up with one word it would be communication. Communication is everywhere in the agile manifesto:

[We value] individuals and interactions over processes and tools.
I see this as a reaction against processes like RUP that felt like a software development factory where developers were replacable cog wheels. Agile recognizes that it's individuals with intelligence, creativity and drive that make a project succeed.

But individuals are not working in isolation, they need to interact with others. Interactions means communication. Not one-way communication but interactive dialogues. Misunderstandings are inevitable in communication. When you say or write something, it is almost certain that the receiver will misunderstand something. You can't just send someone a document and think they will understand what you mean. You need to verify what they understood, and the best way to do this is in a face to face conversation.

A key part of agile is to have close communication within the team and between the team and the customers.

[We value] working software over comprehensive documentation.
Documentation is a form of communication. Some teams stop writing documents "because that's not agile", but that's a huge misunderstanding. Agile does value documentation, but it values working software more. Working software demonstrates progress better than completing a number of documents, and it demonstrates the team's understanding of the requirements better than a requirements document. But documentation may be useful to explain what you were thinking when you developed the software.

I said that iterations don't necessarily make you agile, but iterations are definitely needed to be agile. Iterations is not a goal in itself, the purpose of iterations is to improve communication with the customers by getting feedback often. It is inevitable that we misunderstand what the customers need. Iterations help us to discover these misunderstandings early, before they get too expensive to fix.

[We value] customer collaboration over contract negotiation.
Collaboration certainly means communication. The development team needs a positive dialogue with the customers, and not just communicate with formal documents.

[We value] responding to change over following a plan.
This may not seem like to be about communication, but actually it is. Where do the changes come from? From the customers. The customers and the team should communicate often, not just up front.

These values are the basis for practices like on-site customer, iterative development and pair programming. It's the values that make you agile, not various practices. The practices vary depending on the size and complexity of the project.