Monday, February 15, 2010

Step 4.5 – Spring DM Extender Logging

I’m going to sneak in a bonus blog post in the Enterprise OSGi series as I think some people may have had issues in getting the last step to work and will be relatively stuck without a little help in debugging what the heck is going on with Spring DM Extender. Enabling logging on a bundle that you did not write may seem tricky. Some may even attempt to download the source, tweak it and re-bundle the extender. A cool feature of OSGi is the ability to extend bundles that you’ve not written by developing a Fragment. Fragments allow bundles to be closed for modification, but open for extension. They are typically used to customize web bundles with separate UI skins (as we’ll see later), internationalization, separate OS installations, and a few other niche cases. In our case, we’ll leverage a fragment to customize the log level of the dm extender so that we can figure out what is going on when say, our bundle doesn’t start and there are absolutely zero messages explaining why. Let’s get started by creating another bundle

Screen shot 2010-02-15 at 8.28.05 PM

I passed another couple of flags to maven on this command to ensure that it doesn’t create any internal or interface classes that we’ll have to delete. In our simple case of enabling some logging, we really only need to add a single log4j.properties file to get the magic to happen. I placed it in a src/main/resources/ to keep in line with a typical maven project and added some simple verbose logging configuration.

log4j.rootLogger=info, A


log4j.appender.A=org.apache.log4j.ConsoleAppender


log4j.appender.A.layout=org.apache.log4j.PatternLayout


log4j.appender.A.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n


The last order of business is to adjust the manifest to attach this fragment to a host bundle. As usual, we accomplish this task by adding the following line to the BND file



Fragment-Host: com.springsource.org.apache.log4j


Compiling the project and running pax-provision should provide a slew of information regarding the process spring uses to look for bundles and resolve dependencies



Screen shot 2010-02-15 at 8.58.21 PM



If you’ve had issues, this will hopefully provide some clues as to why the dm extender was not able to find your dependencies. My most frequent mistake is to misspell the directory containing the context files. In the next post, we’ll resume our regularly scheduled programming by providing a database backed persistence layer in our application.



Examples, as always available on github.

Thursday, February 11, 2010

Step 4 – Spring Dynamic Modules

Our previous installment finally began to show the benefits of OSGi. Service production, consumption and registry were all accomplished programmatically with the APIs of the OSGi specification. While this does provide the benefit of reduced coupling and improved cohesion in an enterprise application, the end result was an invasive and difficult to test implementation. Surely there has to be a better way?

The good people at SpringSource asked the same question and they came up with a solution that feels very natural to Spring developers: Spring Dynamic Modules. Spring DM provides OSGi developers with a declarative means of wiring bundles together in much the same manner that the Spring framework allows developers to wire together Java classes. Getting up and running requires a couple of new repositories and an import from the root project level.

Up until now we’ve been able to download the few OSGi bundles that we’ve needed from the typical maven repositories. When we begin to use the Spring extender for OSGi, a number of other bundles will come into the picture. Downloading these bundles from the maven repositories can be hit or miss, so instead let’s use the SpringSource Enterprise Bundle Repository as our main repository for OSGi bundles. EBR is basically a collection of commonly used libraries for Java in valid OSGi bundle form.  Add the two repository locations with pax construct using the following commands from the root directory:

pax-add-repository \
-i com.springsource.repository.bundles.release \
-u http://repository.springsource.com/maven/bundles/release
pax-add-repository \
-i com.springsource.repository.bundles.external \
-u http://repository.springsource.com/maven/bundles/external


Then we can slightly adjust the pax import command to pull in the Spring DM extender and all of it’s dependencies in one fell swoop.



Screen shot 2010-02-11 at 7.34.37 AM



This command is a little different from prior imports in that we have added a couple of properties to the maven command. The importTransitive property instructs pax-construct to not only pull in the spring-osgi-extender, but also all of the other bundles that it depends on. The widenScope property instructs it to import not only those compile time bundles, but also the runtime dependencies as well. This should result in a slew of downloads occurring in your main project directory.



Using Spring DM extender will remove the OSGi programmatic approach to wiring and activating bundles, in favor of autowiring bundles together in a Spring context. By default the extender will look in META-INF/spring for any .xml files that it can sink it’s teeth into. I follow the pattern of using an OSGi specific context and an other file for all the rest. Change to the directory of the service bundle and from there create a src/main/resources/META-INF/spring directory. Within that directory, we’ll create two application context files. The first, service-context.xml will just declare the service implementation as a bean:



<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/beans
  http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">
  
  <bean id="raffleService"
    class="com.pillartech.raffle.service.internal.RaffleServiceImpl" />
</beans>


The osgi-context.xml is fairly straightforward as well:



<beans:beans xmlns="http://www.springframework.org/schema/osgi"
  xmlns:beans="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/osgi
  http://www.springframework.org/schema/osgi/spring-osgi.xsd
  http://www.springframework.org/schema/beans
  http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">
  <service ref="raffleService" 
    interface="com.pillartech.raffle.service.RaffleService" />
</beans:beans>


This context decalres the osgi namespace as the default and can then simply publish the service to OSGi via the <service> tag. You’ll note it uses a reference to the raffleService bean in the other context, make sure these match. With these two files in place the activator for this bundle is no longer necessary. Remove that entire file and you should be able to compile and deploy.



OSGi also provides a means for consuming services. Let’s change the Rigged bundle to make use of it. Start by creating the same src/main/resources/META-INF/spring structure so that the dm extender will find all of the contexts. The OSGi context for a consumer is very similar to the producer:



<beans:beans xmlns="http://www.springframework.org/schema/osgi"
  xmlns:beans="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/osgi
  http://www.springframework.org/schema/osgi/spring-osgi.xsd
  http://www.springframework.org/schema/beans
  http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">
  <reference id="raffleService" 
    interface="com.pillartech.raffle.service.RaffleService" />
  
</beans:beans>


The major change is that instead of using a service tag to produce a service, we use a reference tag to consume one. The rigged activator class will need changed to consume this service via dependency injection. Let’s tweak that class so that all OSGi based lookups are removed and replace it with a simple setter. While we’re in here, let’s also remove the APIs for bundle activation as well. The end result is a ServiceBasedRiggedRaffleActivator that looks like this



package com.pillartech.raffle.rigged.internal;
import java.util.Set;
import com.pillartech.raffle.service.RaffleService;
public final class ServiceBasedRiggedRaffleActivator {
  private RaffleService service;
  
  public RaffleService getRaffleService() {
    return service;
  }
  
  public void setRaffleService(RaffleService svc) {
    service = svc;
  }
  
  public void start() throws Exception {
    if (service != null) {
      addEntrants();
    }
    else {
      System.out.println("Unable to rig the raffle, cannot get a handle on the service");
    }
  }
  private void addEntrants() {
    System.out.println("Adding entrants");
    final int COUNT = 10;
    for (int i = 0; i < COUNT; i++) {
      service.addEntry("Todd("+i+")", "toddkaufman@gmail.com");
    }
    System.out.println(COUNT + " entries added to the raffle.");
  }
  public void stop() throws Exception {
    System.out.println("And the winner of the raffle is ...");
    Set<String> winners = service.pickWinners(1);
    for (String winner : winners) {
      System.out.println(winner + " won!");
    }
  }
}


You can tell by the imports that this code now has zero reliance on OSGi APIs so it is easier to understand and maintain, with the benefit of also being able to be tested in isolation. The final step in getting this to work is to create a context file that wires up the rigged class with it’s dependency on the external service bundle. The rigged-context.xml is



<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/beans
  http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">
  
  <bean id="riggedActivator"
    class="com.pillartech.raffle.rigged.internal.ServiceBasedRiggedRaffleActivator"
    init-method="start" destroy-method="stop">
    <property name="raffleService" ref="raffleService" />
  </bean>
</beans>


Here we make use of the init-method and destroy-method properties of a spring bean to automatically kick off the raffle and shut it down, just as we did with the bundle’s OSGi start/stop lifecycles. One last item of cleanup is to tweak the BND files for both bundles to reflect the removal of a BundleActivator. While we’re in there I would add the following line so that Spring does not expose the bundle’s context to other services:



Spring-Context: META-INF/spring/*.xml;publish-context:=false


Executing a mvn clean install and pax-provision command should provide you with a more verbose lifecycle to these bundles, but the main functionality remains the same. Starting and stopping the rigged service will create a raffle and pick a winner as you would hope.  We’ve deleted a slew of code and added just a handful of configuration to the application to get a simpler and easier to test set of bundles. In the next installment we’ll build upon the use of Spring DM from this point on by adding some database interaction as 90% of the apps we write would.



Source is available as always on githhub.

Monday, February 1, 2010

Step 3 – Service Consumption

OSGi has often been referred to as SOA inside a JVM and I think that’s a fair analogy. The beauty of OSGi is that you do not have to deal with much of the <ceremony /> involved in traditional web services approaches. Comparing OSGi to traditional SOA implementations shows that the complexity is lower, performance is better, and all of the power is still there with OSGi. The one exception to this rule is when your application needs to connect to non-JVM based dependency, you’ll still need to use web services.

In the last few posts, we’ve managed to create a domain bundle and create a rigged raffle bundle that makes use of it. Let’s change this implementation to have a domain agnostic service layer that can leverage the domain bundle under the hood (or a database, web service provider, or something else in the future). Attentive readers will know the drill by now. Start out by creating a service bundle using pax-create-bundle from the project directory.

Screen shot 2010-02-01 at 2.15.40 PM

Creating the service is pretty straightforward. We’ll provide a publicly accessible interface in the com.pillartech.raffle.service package.

package com.pillartech.raffle.service;
import java.util.Set;
public interface RaffleService {
  public void addEntry(String name, String email);
  public Set<String> pickWinners(int numOfWinners);
}


This service allows a consumer to add an entry, and pick  a certain number of winners which is all our rigged consumer really needs at this point. A common practice in OSGi is to create an internal package underneath this main package to store all of non-public classes. Only packages explicitly exposed in the manifest will be made public for other bundles, not directories underneath, so we’ll store our service implementation and activators in this internal directory. Create a com.pillartech.raffle.service.internal package and define a class within this package that implements the service interface. My implementation is fairly straightforward:



package com.pillartech.raffle.service.internal;
import java.util.Date;
import java.util.HashSet;
import java.util.Set;
import com.pillartech.raffle.domain.Entry;
import com.pillartech.raffle.domain.Raffle;
import com.pillartech.raffle.service.RaffleService;
public class RaffleServiceImpl implements RaffleService {
  private Raffle raffle = null;
  public RaffleServiceImpl() {
    raffle = new Raffle();
  }
  public void addEntry(String name, String email) {
    Entry e = new Entry();
    e.setName(name);
    e.setEmail(email);
    e.setCreated(new Date());
    raffle.addEntry(e);
  }
  public Set<String> pickWinners(int numOfWinners) {
    raffle.setNumberOfWinners(numOfWinners);
    Set<Entry> winners = raffle.pickWinners();
    
    Set<String> winnerNames = new HashSet<String>();
    for (Entry entry : winners) {
      winnerNames.add(entry.getName());
    }
    return winnerNames;
  }
}


As you can see, this trivial example just stores the domain objects internally and translates the simplistic arguments passed in to domain objects within the raffle. This may seem like a zero sum gain compared to our last implementation and in all honesty it is. It does pave the way for our service bundle to do things like leveraging a database, and managing transactions across separate DAOs which we’ll accomplish in due time.  Now, we are one activator away from having a viable service published in our container. In the internal directory create an activator class and plumb in some code like this:



package com.pillartech.raffle.service.internal;
import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;
import com.pillartech.raffle.service.RaffleService;
public class RaffleServiceActivator implements BundleActivator {
  public void start(BundleContext bc) throws Exception {
    bc.registerService(RaffleService.class.getName(),
        new RaffleServiceImpl(), null);
  }
  public void stop(BundleContext bc) throws Exception {
  }
}

This activator leverages the BundleContext passed into the start method to register the service. We’re using the common practice of publishing the service using the fully qualified class name of the interface. At this time we could provide a file location, database connection, or other such dependency into the constructor of the implementation, but we’ll get to that soon enough. If you do provide such dependencies, make sure that they are cleaned up and destroyed in the stop method of the activator as well. All that is left is to update the BND file to the name of your activator and you should have a deployable service bundle. Feel free to run mvn install and pax-provision to test the waters.

Publishing the service is the easy part, we still need to consume this service from the rigged raffle bundle. From the rigged bundle directory, run a pax-import-bundle, providing the information used when creating the service.



Screen shot 2010-02-01 at 2.40.00 PM 



Now, we need to re-implement the activator in the rigged bundle to take advantage of the beautiful service waiting out there for us. My updated copy is this:



ackage com.pillartech.raffle.rigged.internal;
import java.util.Set;
import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;
import org.osgi.util.tracker.ServiceTracker;
import com.pillartech.raffle.service.RaffleService;
public final class ServiceBasedRiggedRaffleActivator implements BundleActivator {
  private ServiceTracker tracker;
  private RaffleService service;
  
  public void start(BundleContext bc) throws Exception {
    System.out.println("Grabbing a handle on the raffle service");
    tracker = new ServiceTracker(bc, RaffleService.class.getName(), null);
    tracker.open();
    
    service = (RaffleService) tracker.getService();
    if (service != null) {
      addEntrants();
    }
    else {
      System.out.println("Unable to rig the raffle, cannot get a handle on the service");
    }
  }
  private void addEntrants() {
    System.out.println("Adding entrants");
    final int COUNT = 10;
    for (int i = 0; i < COUNT; i++) {
      service.addEntry("Todd("+i+")", "toddkaufman@gmail.com");
    }
    System.out.println(COUNT + " entries added to the raffle.");
  }
  public void stop(BundleContext bc) throws Exception {
    System.out.println("And the winner of the raffle is ...");
    Set<String> winners = service.pickWinners(1);
    for (String winner : winners) {
      System.out.println(winner + " won!");
    }
    
    System.out.println("Releasing handle on the raffle service.");
    tracker.close();
  }
}


You’ll notice in this example we now have to make use of a ServiceTracker to access the published service. In OSGi land, services can come and go at any time. ServiceTracker’s can be used to grab a handle to a service and optionally wait if one is not available.  Leveraging the service tracker we then access the published service via the interface and interact with it as any normal java class would. Also, take note that we need to close the service tracker in the stop method to ensure that the integration of the two bundles is gracefully decoupled should the rigged raffle go away. If we would prefer to wait for the service should it not be available, we would call tracker.waitForService instead of getService and provide a time in ms to wait should the service not be available immediately. Craig Walls warns against waiting for services in the activator in his Modular Java book though as it may cause a traffic jam of bundles waiting to activate on startup. You should be able to execute a mvn install on the entire project and a pax-provision from the main directory to see everything up and running.



Screen shot 2010-02-01 at 2.55.18 PM



Beauty! We have a working consumer interacting with a service all through the OSGi container.  We’re starting to see how modular bundles can interact within an OSGi container, but if you are like me you are probably feeling pretty dirty about the amount of OSGi APIs that are leaking into your code. Testing these bundles in isolation would require a significant amount of mocking now and we had to manually construct a service implementation using the new keyword. Fear not, there are better, less invasive ways of accomplishing this and we’ll go through them in the next installment.



Examples as always available on github.

Sunday, January 24, 2010

Step 2 – Importing Bundles

The previous post in this series covered a lot of ground on setting up a project with pax-construct, creating a domain bundle, and then embedding another jar file as part of that bundle. At the end of the day we had a simple bundle deployed to our OSGi container, but it didn’t have any external dependencies or an activator so we’ve only developed a traditional jar file and deployed it somewhere new. Let’s change that by creating another bundle that relies on the domain.

From the main directory of the project, run pax-create-bundle as we did last time. If you would like to omit the generated code or interfaces, you can pass flags directly to maven. In this example I’ve advised maven to not bother creating a dummy activator class.

Screen shot 2010-01-24 at 9.42.14 PM

I’ve titled this bundle “rigged” as we’re going to rig our raffle to consistently pick one winner… me. The code to rig the raffle is pretty straightforward as it just makes use of the business logic in the domain classes we created last time. Unfortunately, we have no references to those classes yet, so we need to import that bundle. From the rigged sub-directory run the pax-import-bundle command, passing in the group, artifact, and version information used when creating the domain bundle.

Screen shot 2010-01-24 at 9.48.08 PM

Pax will update the poms appropriately so that the dependency points to the domain bundle within the same project. Now we can code up a simple activator that will add some raffle entries on startup and pick a winner on shutdown.

package com.pillartech.raffle.rigged.internal;
import java.util.Set;
import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;
import com.pillartech.raffle.domain.Entry;
import com.pillartech.raffle.domain.Raffle;
public final class SimpleRiggedRaffleActivator implements BundleActivator {
  private Raffle raffle = null;
  public void start(BundleContext bc) throws Exception {
    System.out.println("Adding entrants");
    raffle = new Raffle();
    raffle.setNumberOfWinners(1);
    for (int i = 0; i < 10; i++) {
      Entry e = new Entry();
      e.setName("The Todd(" + i + ")");
      e.setEmail("toddkaufman@gmail.com");
      raffle.addEntry(e);
    }
    System.out.println(raffle.getEntries().size()
        + " entries added to the raffle.");
  }
  public void stop(BundleContext bc) throws Exception {
    System.out.println("And the winner of the raffle is ...");
    Set<Entry> winners = raffle.pickWinners();
    for (Entry winner : winners) {
      System.out.println(winner.getName());
    }
  }
}


The code to implement an activator is not complex. Implementing the BundleActivator interface necessitates a start and stop method, each taking a BundleContext as a parameter. We don’t need to use the BundleContext just yet so we’ll just get by with some simple rigging of the raffle and system.out.println calls. The only last thing to update is the osgi.bnd file so that the manifest generated correctly points to the activator. The bnd file can make use of properties set by maven so it remains relatively simple.



#-----------------------------------------------------------------
# Use this file to add customized Bnd instructions for the bundle
#-----------------------------------------------------------------
Bundle-Activator: ${bundle.namespace}.internal.SimpleRiggedRaffleActivator


Running mvn clean install and pax-provision at the project layer should get you a success message and something that looks like this



Screen shot 2010-01-24 at 10.20.20 PM



You can see that the activator kicked off on the rigged bundle and it added some entries into the raffle. You can look at how the bundles are viewed differently by the container by running bundle and headers commands on the two. Domain will show relatively little, while the rigged bundle will display the import of the domain bundle and the class name specified as the activator. Stopping the raffle will correctly award the raffle prize to me.



So to this point we’ve created a couple of bundles, imported one to the other, and leveraged the activator to execute some logic when the bundle is started or stopped. Importing other bundles is obviously useful, but to really show the power of OSGi, we need to get bundles talking together in a service / consumer fashion. Stay tuned.



Examples as always, available via github.

Wednesday, January 20, 2010

Step 1 - Creating your Domain

The most common question I've heard regarding OSGi is how do I carve my application into bundles? There is no clear cut answer, but I typically look at functional subsystems within the enterprise (shipping, fulfillment, billing, etc...) and then at the horizontal slices within those systems (presentation, services and persistence). This will tend to provide a good balance for dealing with change and running efforts in parallel without introducing too much complexity. In order to share business objects across separate bundles within the system, we'll need to carve out a domain bundle that others can use.

I'll use pax-construct in these examples as it provides bundle management on top of maven in a relatively intuitive and easy to use fashion. The steps are relatively short to get up and running with a domain bundle.

Start out by creating the project using pax-create-project

Screen shot 2010-01-20 at 9.32.42 PM

This command will setup a maven project for this OSGi system using an OSGi archetype. From here you may want to update the main pom.xml to generate eclipse project and classpath files, update the compiler version used by Maven, and switch it to use equinox over felix as the OSGi container. The main pom.xml will then resemble this

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.pillartech.raffle</groupId>
  <artifactId>raffle</artifactId>
  <version>1.0-SNAPSHOT</version>
  <name>com.pillartech.raffle (OSGi project)</name>
  <description>Generated using Pax-Construct</description>
  <properties>
    <org.osgi.service.http.port>8080</org.osgi.service.http.port>
<org.osgi.service.http.port.secure>8443</org.osgi.service.http.port.secure>
  </properties>
  <packaging>pom</packaging>
  <modules>
    <module>poms</module>
    <module>provision</module>
    <module>domain</module>
  </modules>
  <build>
    <plugins>
      <plugin>
        <artifactId>maven-compiler-plugin</artifactId>
        <configuration>
          <source>1.6</source>
          <target>1.6</target>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.ops4j</groupId>
        <artifactId>maven-pax-plugin</artifactId>
        <version>1.4</version>
        <configuration>
          <provision>
            <param>--platform=equinox</param>
          </provision>
        </configuration>
        <executions>
          <execution>
            <id>ide-support</id>
            <goals>
              <goal>eclipse</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>


After the main project is created pax-runner will be able to start an osgi container from this folder and automatically deploy any homemade or imported bundles automatically. Before we do that, let’s give it something to deploy. Run pax-create-bundle from the main project directory, keeping the group id the same as the one you used when creating the project.



Screen shot 2010-01-20 at 9.47.36 PM 



The pax-create-bundle command will create a familiar project structure for anyone who has used maven. From here you can explore the sample code, delete it when ready and then create your domain classes. My simple example just has two domain classes an Entry and a Raffle. If you are creating domain objects that are not anemic, you’ll probably have a good chance of needing some third party libraries. In my code I planned on using the commons-lang libraries for their equals and hashcode builders. When presented with the need for an external jar in an OSGi environment, you have to ask yourself if this should be embedded into your own bundle as a dependency or installed as a separate bundle within the container. Both have pros and cons, but the decision boils down to whether you feel this jar will be used by other bundles within the system. The commons-lang jars are only useful right now in our domain bundle so we’ll embed. Just update your pom.xml file with the dependency and maven will take care of bringing the external jar into a repository and ensuring that it’s compiled into your bundle. Now is also a good time to add a test dependency on TestNG or JUnit and write some unit tests around the logic in your domain classes.



One other thing to note before deploying this bundle. Pax Construct will make use of BND to generate the manifest file. BND scans the source files, generates an import list, and provides sensible defaults for the manifest so that you don’t have to roll it by hand each time. Since we are just using our domain bundle as a standalone bundle to be consumed by others and it doesn’t have a service or activator, you can simply nuke the generated line in the osgi.bnd file. If all has gone according to plan so far, the last thing is to fire it up. From the main project directory you can run mvn install to compile the domain and then run pax-provision to start up the container and deploy your bundle. Hopefully you get a slew of nifty ascii art indicating that pax-runner has successfully started up and you are greeted with an OSGi prompt. Issuing a short status command if you are using equinox should result in 2 happily active bundles:



Screen shot 2010-01-20 at 10.26.15 PM



So far we’ve created a project, learned how to use pax-construct, embedded a third party jar, and created a bundle that is relatively useless by itself. In the next post, we’ll see how to make use of that bundle from another.



Code samples are available on github.