The Todd on Tech

Show your Debt

2011-09-01T09:00:00.000-04:00

I had an interesting twitter debate a couple weeks ago with a bunch of people I really respect and admire. A good number of them totally disagreed with what I was saying though, so in order to maintain my large ego, I’m going to chalk it up to 140 character limitations and try to expand on my thoughts here.

The crux of the discussion is this. Our team had recently imposed a limit on the number of refactoring or technical debt cards in process at any given time in our system. The reason for this was that we had a couple of iterations where we accomplished some significant changes to the code base which will result in longer term savings and easier maintenance, but the product owner and sales organizations that we work with didn’t really get to see much visible progress on the software for a couple of weeks. We felt that we should spread out the tech debt cards so that the product owner and stakeholders could still get to see progress every week, yet still empower our team to play the tech debt cards as they see fit. So my open question on twitter was:

The consistent push back against my comment was some form of “Why do you have refactoring items on your board?”, as my colleagues mention here:

A lot of people felt that we should just have completed the refactoring as we finished the card. I am not saying that we are not doing refactoring as we are delivering stories and I’m not saying we have an extensive list of refactoring items. We have a solid team with many members who teach and coach others on TDD and the red, green, refactor cycle. The refactoring items or technical debt that we’re tracking are typically orthogonal to a user story we’re currently working on. Consider a fictional example to illustrate the point.

Say you are building a website that sells index cards. The website may have some domain objects like a user, product, address, etc… You have decided to use validation framework X to validate the data that users enter for these objects. This works well until you run into the account domain object. Here there is much more extensive acceptance criteria around the validation of credit card information which isn’t easily accomplished using validation framework X. The team completes a spike and determines that validation framework Y is the right approach so the account creation user story is completed with a new validation framework Y. The team collectively decides that having two validation frameworks is confusing and since Y is the superset of X, that they should standardize on using Y throughout the system. What do you do?

I think many of my peers seem to be suggesting that we should just complete the validation framework swap while working on the account creation user story. I don’t like this for a number of reasons. First, the completed account creation user story is delayed from moving through the rest of the system. I’d rather my product owner or exploratory tester start hammering on the new story (and validation framework) before I start expanding this to the rest of the system. We would be able to see any deficiencies in the new framework prior to swapping everything over and avoid a potentially painful conversion back. If the validation framework swap was a day or more worth of work, then we’d also run the risk of not demoing a completed user story to the rest of the team when really that work is done.

I also think the team would be better served to prioritize this work when they feel it is most important. Collectively. Completing large refactoring or technical work in under the guise of a user story is fairly disingenuous to the product owner. Maybe we’re just lucky because we have a product owner who really sees the potential maintenance costs of not doing these refactorings, but we’ve never felt like we needed to be anything but open and honest about the work that lays ahead. Added bonus here is that if a tech debt card, like swapping out your entire validation framework needs to be played, it can be planned in a way that may not cause significant merge pain for the rest of the team.

I think the discussion on twitter may have been largely a semantic argument about what is technical debt versus refactoring, but I want to make sure my points here are clear. Whether calling technical cards a user story, refactoring card, or tech debt card, make sure that they are planned, prioritized, and completed in the same visible manner as other cards in the system. The board should be a mirror of the work, the work shouldn’t have to be forced to fit into the board.

Why we went Kanban

2011-08-11T08:30:00.001-04:00

I don’t think any single process is the best fit for all teams. Teams and projects using Scrum, Kanban, Waterfall, or any other custom process you can think of have been successful in the past. In my opinion, the most important thing to keep in mind is that the process should be constantly evolving, similar to the software that you are building.

We recently started a new project off with 1 week iterations as they seemed to be a good balance between a client who was coming from a waterfall background and our team, who was experienced in agile development eager to get show off completed functionality as soon as possible. A few retrospectives ago, we found a number of us were openly questioning why we weren’t using Kanban. So as any adaptive, continually improving team would do, we switched. There are a number of reasons for moving to a pull based model, but here were the top ones for our team.

Estimates were WASTE

We performed our demo, retrospective, and planning on Friday of each week. This was usually accomplished in 4 hours or less but by far, the most significant portion of that time was spent discussing and estimating the stories for the next week. We continually struggled to balance how much discussion was needed to properly understand the solution without totally solving the problem within the discussion. With a team of 4 or more developers, the time spent determining whether or not a card was a 3 pointer or a 5 pointer was not trivial. It was however WASTE, and it leads me to my next point.

Track output, not arbitrary numbers

We had been tracking overall burn down based on the number of high level stories identified anyway so the specific number of points that we accomplished in any given week were just a number. Across projects that I’ve been a part of recently, the points and further, velocity as a weekly total of those points, generally winds up becoming at best an arbitrary number and at worst a point of contention or undo pressure. Clients wind up believing that the team is gaming the number of points and the team has to listen to the endless sighs and groans when a card is estimated at 5 points. Screw the points. Get rid of them and instead track either how many cards are being accomplished over time, or better yet the cycle time of cards.

Planning sessions are no substitute for conversations

Another bad habit that we had gotten into was trying to provide cursory information on a card or in TFS work items, discussing that during the planning session, and then just kicking the feature over to the developers to work on during the week. Cards need to be a promise for a future conversation. Without some level of collaboration between product owner and developer, you really don’t get all of the detail flushed out. Further, doing that conversation in a long meeting a few days prior to starting the work inevitably results in details being lost or having to be rehashed in the form of defects or new cards. Since moving to Kanban, we began to execute conversations about the cards in real time much more often. This has inevitably led to less defects or missing requirements coming back into the system.

What’s going to work?

Teamwork! Our project had a couple iterations where our product owner and QA were frustrated by the lack of time they had to test the application before demoing to end users. Ultimately the team is to blame and should’ve realized that having 8 features partially tested and completed is less valuable than having 4 features DONE. When developers are prohibited from pulling another feature forward until stalled cards in QA are moved to done, they immediately begin helping in areas they might otherwise thumb their nose at. It also facilitates in focusing product owners and BA types on keeping a nice stack of prioritized stories ready for development.

Better Stories

Kanban doesn’t write your user stories or even provide better visibility into your user stories. It does however, eliminate some boundaries that are counter productive. When a team feels that a user story must be decomposed into chunks that fit nicely into iterations they may begin to create cards that have many interdependencies. If a project has a bunch of interdependencies within user stories, it will also have significant issues with stories not being testable until other stories are done. Is it worse to have a story that spans an arbitrary segment of time or to have 4 developers all stepping on each other trying to complete small portions of a single large story? In my opinion the former is much better and Kanban doesn’t prohibit it.

I don’t believe everyone should change their process to Kanban. I do believe everyone should evolve their process, though. Kanban definitely seems to be the better fit for our group, but I’m confident it will continue to change before our project is done. In the last couple of weeks using it compared to the prior months of doing weekly iterations, we are communicating more, resolving defects in real time, and collaborating with our product owner much more tightly. This works for us, but as always, use your noodle to determine what works for you. Most importantly, don’t be afraid to change it up!

Numbers photo courtesy of c a r a m e l on flickr. User Stories photo courtesy of jakuza on flickr.

Thank You Coimbatore

2011-07-29T01:24:00.001-04:00

It’s difficult to put into words what a eye opening experience it’s been coaching our development team in Coimbatore, India over the last three weeks. As with anyone, my view of people and the world is largely limited by lack of experience. An interaction here or there with an India based development team has really been all that I have had to base my experiences on and with this limited experience I’ve come to realize I’ve developed some unrealistic beliefs about IT in India. I’m extremely thankful for the opportunity to dispel many of the myths that I’ve had about this gigantic and growing population of developers in India. Here are the top ones:

Myth #1: Indian Developers are Subpar

I had assumed that the caliber of development with the team over here would be pretty substandard. I’ve worked with a couple of different offshoring teams and interacted with a number of people who have done the same. The common lessons learned across those interactions have been, don’t give the offshore team any challenging work and relentlessly review the code when it comes back in. Many people I know attribute this to a lack of technical knowledge with the offshore developers. People believe they are only a year or so out of school and that they don’t have anywhere close to the amount of programming time that their counterparts in the US have.

Maybe this holds true for bigger segments of developers, but we were extremely fortunate to have six really talented developers on our team. Additionally, within the sixty developer company where they work they are more the norm than the exception. Each team member was extremely comfortable in C#, .NET, SQL, and a number of other technical skills. When technologies were new to the team, they were able to quickly and effectively pick up working knowledge of the tools with a little help from us. We quickly changed our plan from trying to teach technical skill to teaching craft and improving process within our first couple days here. In my opinion the difference between the developers I work with back home and the ones over here is mainly just experience with test driving development.

Myth #2: Communication Will be Easy

Everyone speaks English right? Well at least a country that was colonized by England speaks English, right? Uh, no. The difference between the Indians who have frequently interacted with Americans and those who haven’t was astounding. Communicating with our taxi driver was largely primitive grunting and pointing. There were two members of our group were fluent in English and had little trouble understanding us when we slowed our pace of speech down to a reasonable level. The other five members of the team had difficulty understanding us unless we were gesturing, speaking very slowly and enunciating very well. Neither Tim nor I were able to master speaking to this group effectively.

We did find however that everyone on our team was able to read English and all of their code is written in English so we were able to use that as a common denominator. With a little bit of translation help then we were off and running. We’d favor walking through code on a projector as a team and showing by example the practices that we were trying to teach. Ideas and concepts that we felt were extremely important were captured as reference in our wiki on GitHub. It’s a difficult problem to solve when so much of our success relies on frequent, open and honest communication. As we move forward with 8000 miles between us, we’ll leverage IM, shared living documents, and timely feedback with daily standups, weekly retrospectives and demos.

Myth #3: Cultural Differences are not that Wide

I knew that there would be cultural differences but I really had no idea how different the people are here from back home. I had brushed up on the differences with a great book, Speaking of India by Craig Storti, but to understand the impact they have they really must be experienced. The most challenging one to overcome was the rigid hierarchy in the workplace. When the owner of the company was in the room, no team member spoke except for the team lead and he would only reply with a “Yes” or “Yes, sir”. When the team was asked for feedback they would defer to the team lead every time. If he was out of the room, they would defer to the “secondary team lead”.

This rigid hierarchy was stifling communication which as I mentioned before must be open, direct and honest. We didn’t really try to overcome cultural standards that have been ingrained in this group since the beginning of their lives. Instead we picked our opportunities to have direct conversation with individuals and to solicit feedback from the group as much as possible when they were their most honest (bosses out of the room).

Myth #4: Little Effect can be had in Three Weeks

I’ve worked with a number of really good trainers, coaches, and teachers of Agile techniques and everyone routinely agrees that to get somewhat proficient at Test Driving, Refactoring, and Emergent Design takes months. I have no intention of being in India, away from my family for months, so the question of how much change we could enact in three weeks was really concerning. I’m not foolish or egotistical enough to think that the developers we’ve worked with are now proficient at TDD, Refactoring, or anything else that we’ve tried to teach them. Further, I know that they will start to regress once we’ve returned to the states. I do believe, however, that these guys already had an inherent thirst for knowledge. We were able to show them the benefits of every practice we talked about and you could see in their eyes and hear in their conversations that they were excited and eager to continue the voyage. Will they become proficient in the same time as if we were with them? Probably not. Will they continue down the path? Definitely. These guys already had the spark of interest going, the most important thing that we were able to accomplish over here was to just fan the flame.

In all, this trip has taught me to question all of my preconceived notions about people. I am hopefully returning from India, humbled, grateful, and much more respectful of the great country and it’s people. Despite conditions that many Americans would find below standard, the people of India are as welcoming and hospitable as any I have ever met. I am thankful for the opportunity to learn more about their culture and I look forward to continuing to build some great software with this team.

Senthil, Antony, Chenthil, Perumal, Sathish, Govind, and Priya, you were all extremely gracious and patient with the boisterous Americans. You made us feel very much at home when we were in need and we will forever be thankful!

தங்க யு கோயம்புத்தூர்

Hire Craft over Technology

2011-07-21T09:14:00.001-04:00

Consider this blog post an open plea with hiring managers and recruiters. The vast majority of you focus on the wrong thing when trying to hire developers. Really good developers are exponentially more productive than mediocre ones, but you can’t tell the difference because you are looking at the wrong thing. If you disagree, please ask yourself: “What is the first thing you assess in a candidate when considering them for a project?” I’m sure a few will answer differently but by and large most will say that they are trying to make sure that someone has adequate experience with the technologies in use on the project. I’ve seen hundreds of good, bad and mediocre developers at work and I’m here to tell you that the focus on technical skill should be secondary. Focus instead on hiring developers who embrace quality, take pride in their work, and consistently practice their skill. These developers are craftsmen, not programmers.

Taken at a glance, it doesn’t seem to be a bad thing to focus on technology. Technical skill is mandatory for most development positions. Most hiring managers and recruiters couldn’t fathom bringing someone onto a project written in C# if they professed a novice or rudimentary knowledge of C#. The problem with this mentality however is that there are a number of C# developers who are proficient with the language, yet build very costly systems due to the quality of their code. A developer may know Java inside and out, but it doesn’t mean they know how to build software.

Put yourself in a different mindset, that of a person hiring potential carpenters to construct your new home. If you were interviewing the carpenters would you quiz them continually on their knowledge of a power nailer? No, you’d ask them more appropriately about the differences in constructing load bearing walls from non-load bearing. How to frame a doorway. What procedures they use for ensuring a square corner and flat walls. These questions are all striving to assess what level of quality the carpenter builds into their final product. Why do we not assess the same thing with developers?

I believe it namely boils down to either misperception or laziness.

Misperception

The main misperception in hiring managers is that technical skill equals speed. It may in fact equate to short term speed, but without an additional focus on quality the legacy codebase quickly becomes a giant time sink that is difficult to change. Studies have shown that the vast percentage of our project costs are in the maintenance phase, not the construction phase, yet how many dollars have been lost by companies emphasize saving time on the 20% at the expense of the 80%? Unless you are constructing throwaway code or something that will be extremely short lived, do not focus on the speed to build. Focus instead on the quality and your overall speed will increase. Focus on your speed and your overall quality will decrease.

Laziness

I actually think the misperception referenced above is being debunked more and more by hiring managers every day. Sadly though, many recruiters and hiring managers I know are in a rut where they still screen development candidates exactly as they did many years ago. They build simple quizzes to screen for technical knowledge, followed by phone interviews that screen for technical skill and the good ones even follow this up with a programming interview that again determines technical knowledge. Whether it’s ignorance, apathy, or laziness I’m not sure, but if these groups want to see an increase in overall quality (and therefore speed), they need to start asking the difficult questions focused on craft, not technology. I still do a decent number of pairing interviews and I never dictate that someone use a specific language in the interview process. I think my most memorable candidate actually used a language and editor that he wasn’t that familiar with and still you could tell that he had intimate knowledge of TDD and emergent design. I immediately wanted to hire him for any project that we had available and I knew he would be productive and successful regardless of the environment. Can you say the same for the C# developer who has 15 years of experience which are really 1 year projects repeated 15 times?

Craft is not something that is easily learned. K. Anders Ericsson noted that it takes 10,000 hours of targeted practice to reach mastery. If you happen to get one of those passionate developers who have 10,000 hours under their belt, then consider yourself lucky. If you can’t find those people then do the next best thing and hire someone who truly gives a shit and is practicing their craft. You’ll know you’ve found them when they talk much more about quality than technology.

Hammer photo courtesy of thefixer on flickr

Things I Have Learned About Driving in India

2011-07-12T22:06:00.001-04:00

In our first few days over here in India, we’ve been privy to some of the most exhilarating car rides I’ve ever been a part of. By exhilarating, I mean that we have actually almost had 3 head on collisions and we’ve stopped in front of oncoming traffic to take pictures while we were passing an ox drawn cart. We really believe that our driver could have some success back in the states, so we’re actively recruiting him for NASCAR. Here are a few of the tidbits we’ve learned from this expert:

Lanes are merely a suggestion and should rarely be followed. Instead, find the best opening in the road to position you for massive future acceleration.
If you are behind a bus with 230 people in it, pass carefully. They always have the right away.
If you are behind an elephant in the road, pass carefully. They always have the right away. Also, do not tailgate.
Horns are by far the best means to communicate that you are on someone’s left, on someone’s right, are about to hit someone, are going to pass someone, are going to ram someone if they don’t let you pass, or that you are sorry for accidentally running into someone.
If horns are not conveying your intent properly, resort to flipping your headlights at them.
Motorcycles are by far the easiest way to get around. And they can safely transport a family of four. And they can transport 18 month old children.
If you are not simultaneously pressing the gas and the break at some point during your drive, then you are doing it wrong.
When encountering a traffic light which counts down until you can go, do not be a loser and wait for 0. Go at 5 at a minimum, 10 if you are feeling lucky.

Wish us luck for the next few weeks. I’ll have the unbridled joy of this little guy for every ride…

Trials and Tribulations of getting Jenkins on Windows pulling from GitHub

2011-04-27T09:22:00.001-04:00

I spent the better part of the last two days getting a Jenkins build which runs on a Windows 2008 server to pull from GitHub. You might think, “Wow, Todd really must be an (idiot|rube|dolt|moron|manager) to need 2 days to setup a CI server”. Well, you might be right, but even a simpleton like myself can google the shit out of a problem and come up with some type of frankensteined solution in a day or less. Apparently though, I am one of 5 people on this planet to actually use the cluster of Jenkins/Windows/Github. In case there are more of you out there, I thought I would try to spell out exactly what I did so that you don’t have to suffer (and so that when I forget how I got this thing to work and the server needs to be rebuilt in a year I can do it).

Step 0: I’m assuming you already have a Windows server with Jenkins up and running on it. I’m also going to assume that you installed it as a service. if you aren’t there yet, follow the instructions here: https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins+as+a+Windows+service

Step 1: Install yourself some msysgit from here: http://code.google.com/p/msysgit/

Step 2: Head to Manage Plugins from the main Jenkins menu, select the Git plugin from the list of available plugins, and install that puppy. You’ll need to restart the service for this to take effect.

Step 3: I had to explicitly configure Jenkins to use the git.cmd instead of git.exe. I believe this properly sets the HOME variable so that your ssh and netrc files can be located when Jenkins fires off a build. From the Manage Jenkins link at the top level, you should have the option of adding a git installation. Provide the full path to the <GIT_HOME>/cmd/git.cmd file in this line as shown here:

Step 4: Generate and upload an ssh key for your github account. Make sure you are logged in as the same user that the Jenkins service runs under. Good instructions here: http://help.github.com/win-set-up-git/

Step 5: Configure a _netrc file and store it in the home directory of the user Jenkins runs under. My _netrc file looks like this:

machine github.com
login pillarci
password notmypassword

Step 6: Set an environment variable for HTTPS_PROXY if you have a firewall.

Step 7: Test that you can pull down from the command line. Given your repository name, you should be able to go to the command line and run something like

git clone https://github.com/SomeOrg/SomeRepo.git

Step 8: Configure your build on Jenkins. Please note that the repository name should not have the username in it and it should be using the https url, not the git url. Something like this:

Step 9: Kick that build off and pray. At this step I had a number of obtuse error messages and hung builds. If either of those instances occur with you, proceed to steps 10-47.

Possible Step 10 – 47: Check for a .git folder in your workspace without a sibling directory containing your repo. Nuke it if so and retry the build.

That should be all you need to get up and running. Again, I had some issues, many of which were the fault of the user, many due to the lack of info out on the internets. Luckily I work with a really smart guy named Justin Searls and he had already laid a bunch of this pavement for me. Happy GitHubbing!

Step 4.5 – Spring DM Extender Logging

2010-02-15T21:10:00.001-05:00

I’m going to sneak in a bonus blog post in the Enterprise OSGi series as I think some people may have had issues in getting the last step to work and will be relatively stuck without a little help in debugging what the heck is going on with Spring DM Extender. Enabling logging on a bundle that you did not write may seem tricky. Some may even attempt to download the source, tweak it and re-bundle the extender. A cool feature of OSGi is the ability to extend bundles that you’ve not written by developing a Fragment. Fragments allow bundles to be closed for modification, but open for extension. They are typically used to customize web bundles with separate UI skins (as we’ll see later), internationalization, separate OS installations, and a few other niche cases. In our case, we’ll leverage a fragment to customize the log level of the dm extender so that we can figure out what is going on when say, our bundle doesn’t start and there are absolutely zero messages explaining why. Let’s get started by creating another bundle

I passed another couple of flags to maven on this command to ensure that it doesn’t create any internal or interface classes that we’ll have to delete. In our simple case of enabling some logging, we really only need to add a single log4j.properties file to get the magic to happen. I placed it in a src/main/resources/ to keep in line with a typical maven project and added some simple verbose logging configuration.

log4j.rootLogger=info, A

log4j.appender.A=org.apache.log4j.ConsoleAppender

log4j.appender.A.layout=org.apache.log4j.PatternLayout

log4j.appender.A.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n

The last order of business is to adjust the manifest to attach this fragment to a host bundle. As usual, we accomplish this task by adding the following line to the BND file

Fragment-Host: com.springsource.org.apache.log4j

Compiling the project and running pax-provision should provide a slew of information regarding the process spring uses to look for bundles and resolve dependencies

If you’ve had issues, this will hopefully provide some clues as to why the dm extender was not able to find your dependencies. My most frequent mistake is to misspell the directory containing the context files. In the next post, we’ll resume our regularly scheduled programming by providing a database backed persistence layer in our application.

Examples, as always available on github.

Step 4 – Spring Dynamic Modules

2010-02-11T09:29:00.001-05:00

Our previous installment finally began to show the benefits of OSGi. Service production, consumption and registry were all accomplished programmatically with the APIs of the OSGi specification. While this does provide the benefit of reduced coupling and improved cohesion in an enterprise application, the end result was an invasive and difficult to test implementation. Surely there has to be a better way?

The good people at SpringSource asked the same question and they came up with a solution that feels very natural to Spring developers: Spring Dynamic Modules. Spring DM provides OSGi developers with a declarative means of wiring bundles together in much the same manner that the Spring framework allows developers to wire together Java classes. Getting up and running requires a couple of new repositories and an import from the root project level.

Up until now we’ve been able to download the few OSGi bundles that we’ve needed from the typical maven repositories. When we begin to use the Spring extender for OSGi, a number of other bundles will come into the picture. Downloading these bundles from the maven repositories can be hit or miss, so instead let’s use the SpringSource Enterprise Bundle Repository as our main repository for OSGi bundles. EBR is basically a collection of commonly used libraries for Java in valid OSGi bundle form. Add the two repository locations with pax construct using the following commands from the root directory:

pax-add-repository \
-i com.springsource.repository.bundles.release \
-u http://repository.springsource.com/maven/bundles/release
pax-add-repository \
-i com.springsource.repository.bundles.external \
-u http://repository.springsource.com/maven/bundles/external

Then we can slightly adjust the pax import command to pull in the Spring DM extender and all of it’s dependencies in one fell swoop.

This command is a little different from prior imports in that we have added a couple of properties to the maven command. The importTransitive property instructs pax-construct to not only pull in the spring-osgi-extender, but also all of the other bundles that it depends on. The widenScope property instructs it to import not only those compile time bundles, but also the runtime dependencies as well. This should result in a slew of downloads occurring in your main project directory.

Using Spring DM extender will remove the OSGi programmatic approach to wiring and activating bundles, in favor of autowiring bundles together in a Spring context. By default the extender will look in META-INF/spring for any .xml files that it can sink it’s teeth into. I follow the pattern of using an OSGi specific context and an other file for all the rest. Change to the directory of the service bundle and from there create a src/main/resources/META-INF/spring directory. Within that directory, we’ll create two application context files. The first, service-context.xml will just declare the service implementation as a bean:

<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/beans
  http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">
  
  <bean id="raffleService"
    class="com.pillartech.raffle.service.internal.RaffleServiceImpl" />
</beans>

The osgi-context.xml is fairly straightforward as well:

<beans:beans xmlns="http://www.springframework.org/schema/osgi"
  xmlns:beans="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/osgi
  http://www.springframework.org/schema/osgi/spring-osgi.xsd
  http://www.springframework.org/schema/beans
  http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">
  <service ref="raffleService" 
    interface="com.pillartech.raffle.service.RaffleService" />
</beans:beans>

This context decalres the osgi namespace as the default and can then simply publish the service to OSGi via the <service> tag. You’ll note it uses a reference to the raffleService bean in the other context, make sure these match. With these two files in place the activator for this bundle is no longer necessary. Remove that entire file and you should be able to compile and deploy.

OSGi also provides a means for consuming services. Let’s change the Rigged bundle to make use of it. Start by creating the same src/main/resources/META-INF/spring structure so that the dm extender will find all of the contexts. The OSGi context for a consumer is very similar to the producer:

<beans:beans xmlns="http://www.springframework.org/schema/osgi"
  xmlns:beans="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/osgi
  http://www.springframework.org/schema/osgi/spring-osgi.xsd
  http://www.springframework.org/schema/beans
  http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">
  <reference id="raffleService" 
    interface="com.pillartech.raffle.service.RaffleService" />
  
</beans:beans>

The major change is that instead of using a service tag to produce a service, we use a reference tag to consume one. The rigged activator class will need changed to consume this service via dependency injection. Let’s tweak that class so that all OSGi based lookups are removed and replace it with a simple setter. While we’re in here, let’s also remove the APIs for bundle activation as well. The end result is a ServiceBasedRiggedRaffleActivator that looks like this

package com.pillartech.raffle.rigged.internal;
import java.util.Set;
import com.pillartech.raffle.service.RaffleService;
public final class ServiceBasedRiggedRaffleActivator {
  private RaffleService service;
  
  public RaffleService getRaffleService() {
    return service;
  }
  
  public void setRaffleService(RaffleService svc) {
    service = svc;
  }
  
  public void start() throws Exception {
    if (service != null) {
      addEntrants();
    }
    else {
      System.out.println("Unable to rig the raffle, cannot get a handle on the service");
    }
  }
  private void addEntrants() {
    System.out.println("Adding entrants");
    final int COUNT = 10;
    for (int i = 0; i < COUNT; i++) {
      service.addEntry("Todd("+i+")", "toddkaufman@gmail.com");
    }
    System.out.println(COUNT + " entries added to the raffle.");
  }
  public void stop() throws Exception {
    System.out.println("And the winner of the raffle is ...");
    Set<String> winners = service.pickWinners(1);
    for (String winner : winners) {
      System.out.println(winner + " won!");
    }
  }
}

You can tell by the imports that this code now has zero reliance on OSGi APIs so it is easier to understand and maintain, with the benefit of also being able to be tested in isolation. The final step in getting this to work is to create a context file that wires up the rigged class with it’s dependency on the external service bundle. The rigged-context.xml is

<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/beans
  http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">
  
  <bean id="riggedActivator"
    class="com.pillartech.raffle.rigged.internal.ServiceBasedRiggedRaffleActivator"
    init-method="start" destroy-method="stop">
    <property name="raffleService" ref="raffleService" />
  </bean>
</beans>

Here we make use of the init-method and destroy-method properties of a spring bean to automatically kick off the raffle and shut it down, just as we did with the bundle’s OSGi start/stop lifecycles. One last item of cleanup is to tweak the BND files for both bundles to reflect the removal of a BundleActivator. While we’re in there I would add the following line so that Spring does not expose the bundle’s context to other services:

Spring-Context: META-INF/spring/*.xml;publish-context:=false

Executing a mvn clean install and pax-provision command should provide you with a more verbose lifecycle to these bundles, but the main functionality remains the same. Starting and stopping the rigged service will create a raffle and pick a winner as you would hope. We’ve deleted a slew of code and added just a handful of configuration to the application to get a simpler and easier to test set of bundles. In the next installment we’ll build upon the use of Spring DM from this point on by adding some database interaction as 90% of the apps we write would.

Source is available as always on githhub.

Step 3 – Service Consumption

2010-02-01T15:05:00.001-05:00

OSGi has often been referred to as SOA inside a JVM and I think that’s a fair analogy. The beauty of OSGi is that you do not have to deal with much of the <ceremony /> involved in traditional web services approaches. Comparing OSGi to traditional SOA implementations shows that the complexity is lower, performance is better, and all of the power is still there with OSGi. The one exception to this rule is when your application needs to connect to non-JVM based dependency, you’ll still need to use web services.

In the last few posts, we’ve managed to create a domain bundle and create a rigged raffle bundle that makes use of it. Let’s change this implementation to have a domain agnostic service layer that can leverage the domain bundle under the hood (or a database, web service provider, or something else in the future). Attentive readers will know the drill by now. Start out by creating a service bundle using pax-create-bundle from the project directory.

Creating the service is pretty straightforward. We’ll provide a publicly accessible interface in the com.pillartech.raffle.service package.

package com.pillartech.raffle.service;
import java.util.Set;
public interface RaffleService {
  public void addEntry(String name, String email);
  public Set<String> pickWinners(int numOfWinners);
}

This service allows a consumer to add an entry, and pick a certain number of winners which is all our rigged consumer really needs at this point. A common practice in OSGi is to create an internal package underneath this main package to store all of non-public classes. Only packages explicitly exposed in the manifest will be made public for other bundles, not directories underneath, so we’ll store our service implementation and activators in this internal directory. Create a com.pillartech.raffle.service.internal package and define a class within this package that implements the service interface. My implementation is fairly straightforward:

package com.pillartech.raffle.service.internal;
import java.util.Date;
import java.util.HashSet;
import java.util.Set;
import com.pillartech.raffle.domain.Entry;
import com.pillartech.raffle.domain.Raffle;
import com.pillartech.raffle.service.RaffleService;
public class RaffleServiceImpl implements RaffleService {
  private Raffle raffle = null;
  public RaffleServiceImpl() {
    raffle = new Raffle();
  }
  public void addEntry(String name, String email) {
    Entry e = new Entry();
    e.setName(name);
    e.setEmail(email);
    e.setCreated(new Date());
    raffle.addEntry(e);
  }
  public Set<String> pickWinners(int numOfWinners) {
    raffle.setNumberOfWinners(numOfWinners);
    Set<Entry> winners = raffle.pickWinners();
    
    Set<String> winnerNames = new HashSet<String>();
    for (Entry entry : winners) {
      winnerNames.add(entry.getName());
    }
    return winnerNames;
  }
}

As you can see, this trivial example just stores the domain objects internally and translates the simplistic arguments passed in to domain objects within the raffle. This may seem like a zero sum gain compared to our last implementation and in all honesty it is. It does pave the way for our service bundle to do things like leveraging a database, and managing transactions across separate DAOs which we’ll accomplish in due time. Now, we are one activator away from having a viable service published in our container. In the internal directory create an activator class and plumb in some code like this:

package com.pillartech.raffle.service.internal;
import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;
import com.pillartech.raffle.service.RaffleService;
public class RaffleServiceActivator implements BundleActivator {
  public void start(BundleContext bc) throws Exception {
    bc.registerService(RaffleService.class.getName(),
        new RaffleServiceImpl(), null);
  }
  public void stop(BundleContext bc) throws Exception {
  }
}

This activator leverages the BundleContext passed into the start method to register the service. We’re using the common practice of publishing the service using the fully qualified class name of the interface. At this time we could provide a file location, database connection, or other such dependency into the constructor of the implementation, but we’ll get to that soon enough. If you do provide such dependencies, make sure that they are cleaned up and destroyed in the stop method of the activator as well. All that is left is to update the BND file to the name of your activator and you should have a deployable service bundle. Feel free to run mvn install and pax-provision to test the waters.

Publishing the service is the easy part, we still need to consume this service from the rigged raffle bundle. From the rigged bundle directory, run a pax-import-bundle, providing the information used when creating the service.

Now, we need to re-implement the activator in the rigged bundle to take advantage of the beautiful service waiting out there for us. My updated copy is this:

ackage com.pillartech.raffle.rigged.internal;
import java.util.Set;
import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;
import org.osgi.util.tracker.ServiceTracker;
import com.pillartech.raffle.service.RaffleService;
public final class ServiceBasedRiggedRaffleActivator implements BundleActivator {
  private ServiceTracker tracker;
  private RaffleService service;
  
  public void start(BundleContext bc) throws Exception {
    System.out.println("Grabbing a handle on the raffle service");
    tracker = new ServiceTracker(bc, RaffleService.class.getName(), null);
    tracker.open();
    
    service = (RaffleService) tracker.getService();
    if (service != null) {
      addEntrants();
    }
    else {
      System.out.println("Unable to rig the raffle, cannot get a handle on the service");
    }
  }
  private void addEntrants() {
    System.out.println("Adding entrants");
    final int COUNT = 10;
    for (int i = 0; i < COUNT; i++) {
      service.addEntry("Todd("+i+")", "toddkaufman@gmail.com");
    }
    System.out.println(COUNT + " entries added to the raffle.");
  }
  public void stop(BundleContext bc) throws Exception {
    System.out.println("And the winner of the raffle is ...");
    Set<String> winners = service.pickWinners(1);
    for (String winner : winners) {
      System.out.println(winner + " won!");
    }
    
    System.out.println("Releasing handle on the raffle service.");
    tracker.close();
  }
}

You’ll notice in this example we now have to make use of a ServiceTracker to access the published service. In OSGi land, services can come and go at any time. ServiceTracker’s can be used to grab a handle to a service and optionally wait if one is not available. Leveraging the service tracker we then access the published service via the interface and interact with it as any normal java class would. Also, take note that we need to close the service tracker in the stop method to ensure that the integration of the two bundles is gracefully decoupled should the rigged raffle go away. If we would prefer to wait for the service should it not be available, we would call tracker.waitForService instead of getService and provide a time in ms to wait should the service not be available immediately. Craig Walls warns against waiting for services in the activator in his Modular Java book though as it may cause a traffic jam of bundles waiting to activate on startup. You should be able to execute a mvn install on the entire project and a pax-provision from the main directory to see everything up and running.

Beauty! We have a working consumer interacting with a service all through the OSGi container. We’re starting to see how modular bundles can interact within an OSGi container, but if you are like me you are probably feeling pretty dirty about the amount of OSGi APIs that are leaking into your code. Testing these bundles in isolation would require a significant amount of mocking now and we had to manually construct a service implementation using the new keyword. Fear not, there are better, less invasive ways of accomplishing this and we’ll go through them in the next installment.

Examples as always available on github.

Step 2 – Importing Bundles

2010-01-24T22:32:00.001-05:00

The previous post in this series covered a lot of ground on setting up a project with pax-construct, creating a domain bundle, and then embedding another jar file as part of that bundle. At the end of the day we had a simple bundle deployed to our OSGi container, but it didn’t have any external dependencies or an activator so we’ve only developed a traditional jar file and deployed it somewhere new. Let’s change that by creating another bundle that relies on the domain.

From the main directory of the project, run pax-create-bundle as we did last time. If you would like to omit the generated code or interfaces, you can pass flags directly to maven. In this example I’ve advised maven to not bother creating a dummy activator class.

I’ve titled this bundle “rigged” as we’re going to rig our raffle to consistently pick one winner… me. The code to rig the raffle is pretty straightforward as it just makes use of the business logic in the domain classes we created last time. Unfortunately, we have no references to those classes yet, so we need to import that bundle. From the rigged sub-directory run the pax-import-bundle command, passing in the group, artifact, and version information used when creating the domain bundle.

Pax will update the poms appropriately so that the dependency points to the domain bundle within the same project. Now we can code up a simple activator that will add some raffle entries on startup and pick a winner on shutdown.

package com.pillartech.raffle.rigged.internal;
import java.util.Set;
import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;
import com.pillartech.raffle.domain.Entry;
import com.pillartech.raffle.domain.Raffle;
public final class SimpleRiggedRaffleActivator implements BundleActivator {
  private Raffle raffle = null;
  public void start(BundleContext bc) throws Exception {
    System.out.println("Adding entrants");
    raffle = new Raffle();
    raffle.setNumberOfWinners(1);
    for (int i = 0; i < 10; i++) {
      Entry e = new Entry();
      e.setName("The Todd(" + i + ")");
      e.setEmail("toddkaufman@gmail.com");
      raffle.addEntry(e);
    }
    System.out.println(raffle.getEntries().size()
        + " entries added to the raffle.");
  }
  public void stop(BundleContext bc) throws Exception {
    System.out.println("And the winner of the raffle is ...");
    Set<Entry> winners = raffle.pickWinners();
    for (Entry winner : winners) {
      System.out.println(winner.getName());
    }
  }
}

The code to implement an activator is not complex. Implementing the BundleActivator interface necessitates a start and stop method, each taking a BundleContext as a parameter. We don’t need to use the BundleContext just yet so we’ll just get by with some simple rigging of the raffle and system.out.println calls. The only last thing to update is the osgi.bnd file so that the manifest generated correctly points to the activator. The bnd file can make use of properties set by maven so it remains relatively simple.

#-----------------------------------------------------------------
# Use this file to add customized Bnd instructions for the bundle
#-----------------------------------------------------------------
Bundle-Activator: ${bundle.namespace}.internal.SimpleRiggedRaffleActivator

Running mvn clean install and pax-provision at the project layer should get you a success message and something that looks like this

You can see that the activator kicked off on the rigged bundle and it added some entries into the raffle. You can look at how the bundles are viewed differently by the container by running bundle and headers commands on the two. Domain will show relatively little, while the rigged bundle will display the import of the domain bundle and the class name specified as the activator. Stopping the raffle will correctly award the raffle prize to me.

So to this point we’ve created a couple of bundles, imported one to the other, and leveraged the activator to execute some logic when the bundle is started or stopped. Importing other bundles is obviously useful, but to really show the power of OSGi, we need to get bundles talking together in a service / consumer fashion. Stay tuned.

Examples as always, available via github.

Step 1 - Creating your Domain

2010-01-20T22:59:00.001-05:00

The most common question I've heard regarding OSGi is how do I carve my application into bundles? There is no clear cut answer, but I typically look at functional subsystems within the enterprise (shipping, fulfillment, billing, etc...) and then at the horizontal slices within those systems (presentation, services and persistence). This will tend to provide a good balance for dealing with change and running efforts in parallel without introducing too much complexity. In order to share business objects across separate bundles within the system, we'll need to carve out a domain bundle that others can use.

I'll use pax-construct in these examples as it provides bundle management on top of maven in a relatively intuitive and easy to use fashion. The steps are relatively short to get up and running with a domain bundle.

Start out by creating the project using pax-create-project

This command will setup a maven project for this OSGi system using an OSGi archetype. From here you may want to update the main pom.xml to generate eclipse project and classpath files, update the compiler version used by Maven, and switch it to use equinox over felix as the OSGi container. The main pom.xml will then resemble this

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.pillartech.raffle</groupId>
  <artifactId>raffle</artifactId>
  <version>1.0-SNAPSHOT</version>
  <name>com.pillartech.raffle (OSGi project)</name>
  <description>Generated using Pax-Construct</description>
  <properties>
    <org.osgi.service.http.port>8080</org.osgi.service.http.port>
<org.osgi.service.http.port.secure>8443</org.osgi.service.http.port.secure>
  </properties>
  <packaging>pom</packaging>
  <modules>
    <module>poms</module>
    <module>provision</module>
    <module>domain</module>
  </modules>
  <build>
    <plugins>
      <plugin>
        <artifactId>maven-compiler-plugin</artifactId>
        <configuration>
          <source>1.6</source>
          <target>1.6</target>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.ops4j</groupId>
        <artifactId>maven-pax-plugin</artifactId>
        <version>1.4</version>
        <configuration>
          <provision>
            <param>--platform=equinox</param>
          </provision>
        </configuration>
        <executions>
          <execution>
            <id>ide-support</id>
            <goals>
              <goal>eclipse</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

After the main project is created pax-runner will be able to start an osgi container from this folder and automatically deploy any homemade or imported bundles automatically. Before we do that, let’s give it something to deploy. Run pax-create-bundle from the main project directory, keeping the group id the same as the one you used when creating the project.

The pax-create-bundle command will create a familiar project structure for anyone who has used maven. From here you can explore the sample code, delete it when ready and then create your domain classes. My simple example just has two domain classes an Entry and a Raffle. If you are creating domain objects that are not anemic, you’ll probably have a good chance of needing some third party libraries. In my code I planned on using the commons-lang libraries for their equals and hashcode builders. When presented with the need for an external jar in an OSGi environment, you have to ask yourself if this should be embedded into your own bundle as a dependency or installed as a separate bundle within the container. Both have pros and cons, but the decision boils down to whether you feel this jar will be used by other bundles within the system. The commons-lang jars are only useful right now in our domain bundle so we’ll embed. Just update your pom.xml file with the dependency and maven will take care of bringing the external jar into a repository and ensuring that it’s compiled into your bundle. Now is also a good time to add a test dependency on TestNG or JUnit and write some unit tests around the logic in your domain classes.

One other thing to note before deploying this bundle. Pax Construct will make use of BND to generate the manifest file. BND scans the source files, generates an import list, and provides sensible defaults for the manifest so that you don’t have to roll it by hand each time. Since we are just using our domain bundle as a standalone bundle to be consumed by others and it doesn’t have a service or activator, you can simply nuke the generated line in the osgi.bnd file. If all has gone according to plan so far, the last thing is to fire it up. From the main project directory you can run mvn install to compile the domain and then run pax-provision to start up the container and deploy your bundle. Hopefully you get a slew of nifty ascii art indicating that pax-runner has successfully started up and you are greeted with an OSGi prompt. Issuing a short status command if you are using equinox should result in 2 happily active bundles:

So far we’ve created a project, learned how to use pax-construct, embedded a third party jar, and created a bundle that is relatively useless by itself. In the next post, we’ll see how to make use of that bundle from another.

Code samples are available on github.

Step 0 - Simple OSGi

2010-01-11T14:33:00.008-05:00

OSGi is a longstanding framework for building modular applications. I won't go into the details as I'm assuming you are somewhat interested in the framework to have found my blog. Two of the best resources I've found are:

Craig Walls' blog
Kirk Knoerschild's blog

Craig and Kirk are helping many (including myself) to grok the benefits of OSGi and dispel the myths around it's adoption. Specifically, Craig's post refuting the challenges with OSGi covers almost all of the issues usually brought up.

Now that that's settled, let's get a simple example up and running.

I'm going to be using Equinox for my examples. Primarily because it supports all of the features we'll need and also because it's bundled with Eclipse so you can easily find it and get it started.

Download equinox from here. You can just get by with the framework jar. You can start the eclipse container with

java -jar org.eclipse.osgi_3.5.1.R35x_v20090827.jar -console

Issuing a short status command with ss shows

Since we haven't built a bundle yet, the only thing available is the osgi core bundle. A bundle is simply no more than a jar file with a quirky META-INF/MANIFEST.MF file. Building the equivalent of Hello World can be done with this:

    1 package com.pillartech;
    2 
    3 import org.osgi.framework.BundleActivator;
    4 import org.osgi.framework.BundleContext;
    5 
    6 public class SuperSimple implements BundleActivator {
    7 
    8     public void start(BundleContext ctx) throws Exception {
    9         System.out.println("Starting Up!");
   10     }
   11     
   12     public void stop(BundleContext ctx) throws Exception {
   13         System.out.println("Shutting Down!");
   14     }
   15 }

This is just a java file though until we add the magic to a MANIFEST.MF file and jar the thing up. Here is a simple manifest to get started:

    1 Bundle-ManifestVersion: 2
    2 Bundle-SymbolicName: com.pillartech.SuperSimple
    3 Bundle-Name: SuperSimple
    4 Bundle-Version: 1.0.0
    5 Bundle-Activator: com.pillartech.SuperSimple
    6 Import-Package: org.osgi.framework

As long as you include the symbolic name of the bundle it should work, but the rest of it gives you a feel of the meaning of the manifest. It provides all of the information used by other bundles to consume it, and also a list of the other bundles that it imports.

Compiling this class and jar-ing the class and manifest together allows you to deploy it and start it

From there you can see descriptive information about this bundle and it's manifest with the bundle and headers commands

Stopping the bundle will produce an equally productive message and return it to the INSTALLED state. Finally issuing an exit command will shutdown the container.

So with this example we've done the simplest thing we can to get a bundle installed, started, described, and stopped in equinox. These commands are the core to developing and manipulating bundles in a runtime with OSGi. We'll leverage them further in the subsequent examples to build our solution.

Enterprise OSGi in 10 steps

2010-01-11T14:20:00.010-05:00

I'm presenting a Modular Java pre-compiler session at Codemash and in preparing I have learned a good deal about OSGi. My mode of learning was to take small granular steps towards a front to back solution to a typical project problem, but with the benefit of using OSGi. If you want to follow my footsteps, the best thing you can do is come to Codemash and attend the pre-compiler. If you aren't fortunate enough to get there, I'm going to post 10 blog posts to build up your knowledge of OSGi and the accompanying tools. This post will serve as a table of contents.

Step 0 - Simple Bundle
Step 1 - Creating your Domain
Step 2 - Importing from another Bundle
Step 3 - Service Oriented OSGi
Step 4 - Spring DM
Step 4.5 - Spring DM Extender Logging
Step 5 - Persistence
Step 6 - Integration Testing
Step 7 - Web Development
Step 8 - Skinning a UI
Step 9 - Deploy Time Configuration

My Mac Tools

2009-11-13T21:18:00.002-05:00

I recently started over from scratch on a shiny new 15" macbook pro. What I didn't realize is how many tools I've accumulated on the old machine. While downloading 4 separate tools at once, I thought it would be a good time to track my favorites for posterity's sake (and to help my friend Tim who is in a similar scenario).

Adium - IM client of choice
Quicksilver - I have trouble interfacing with my mac without it
TweetDeck - My favorite twitter client
Things - My method for Getting Things Done
Isolator - Tool for improving focus
TextMate - Great lightweight editor
SpringSource Tool Suite - Great heavyweight editor
Tomcat - Java webserver
MySQL - Powerful open source database
Sequel - Very nice client for SQL
Skype - You know
JumpCut - Better copying tool
SuperDuper - Backups that work
Cyberduck - FTP, webDAV, Cloud Client
Dropbox - Ass kicking Cloud storage
iTerm - My preferred terminal client
Evernote - Quick note taking
VMWare Fusion - Better than Parallels IMHO

And that's about it. I'll try to update as I remember others, but these are the tools that I really missed in the first two days of using my new mac.

5 Reasons I Work at Pillar Technology

2009-11-11T20:44:00.007-05:00

People
I've been aware of Pillar Technology for some time and I thought that they had some good people here and there, but I've been astounded by the amount of talent that I've had the pleasure of working with in my short time here. Up and down the organization I've met elite software developers, agile coaches, team leaders, and many others who excel at what they do. One of our .NET leaders informed me that his rate for recommending new hires is at about 8%. 92% of the candidates he interviews are not at the level needed to work for Pillar. If, like me, you want to elevate your career, go work with a team of people that you can learn from.

Process
I've contended for some time that Agile and Lean software development techniques will be more common than Waterfall within the next 5 years for some very simple reasons. Agile techniques result in working systems, with fewer defects, delivered faster, and with a lower total cost of ownership than the alternatives. We are already starting to see a backlash against some of the short term, cheap construction solutions that have proven time and again to be less efficient and more expensive than previously thought. As more and more companies begin adopting Agile as a cure to their failing projects, look for consulting companies like Pillar that are leaders in this space to grow exponentially.

Collaboration
In the first week of working at Pillar, I've had the pleasure of offering my ideas on where we could take this company to our CEO, COO, multiple VPs, my peers, my team, my wife, my kids, and everyone else who will listen.I was a part of no less than 4 meetings in the first week focused on taking our company to the next level. Ideas at Pillar are solicited, not criticized. True openness and collaboration in an organization is hard to come by, but it is crucial for long term, strategic growth.

Focus
The companies that have grown in this economy have shared a few traits. One of them is to know what you are good at, and you focus on taking that to market better than anyone else. Pillar is extremely good at applying elite software development teams using Agile methodologies, to solve problems. Everything that we do here in the next 10 years will be in line with that focus. Companies that attempt to react to every opportunity presented to them have historically diluted their talent and weakened their position in the market place.

No Boundaries
Applying the core capability referenced above to as broad a population as you can has been a pattern of success for organizations for well over a century. Nike didn't start out by thinking they could sell tennis shoes to a few high schools in Oregon. Microsoft was focused on delivering computers to every home in the world at a time when the market was really only medium or large businesses. Pillar is at a point where the company is transforming from a regional focus to a national focus and I have every confidence we'll be successful. Sit tight, it may not be that long until we're international.

Thank You QSI

2009-10-30T10:20:00.004-04:00

About 6 years ago I began my first consulting assignment with the idea that I would bounce around from client to client, build my network, and begin to determine what type of company I wanted to work for. A strange thing happened however, I began to realize that my consulting company, Quick Solutions, was exactly the type of company that I wanted to work for. They valued people above all else and built a staff of passionate leaders in technology that I was proud to be a part of. They realized that software development is most successfully done with TEAMS and not individuals so they fostered a team culture that made me excited to go to work every day. They understood that our company should be built with a mentality of ownership and provided benefits that cultivated that mindset across the organization. For the past six years I have been extremely privileged to work at QSI and it is with a heavy heart that I announce my departure from this great organization.

All things change and eventually come to an end, but words are not capable of articulating how hard this decision has been. I hope all of my great friends at QSI know that they made this decision gut wrenching and I am only able to make it knowing that I will keep in touch and continue to work, drink beer, talk shop, or golf with them in the future. The number of people at QSI who have helped change and crystalize my views on software development are too many to list here, but I sincerely thank all of you for the career altering opportunity to work with you. We will always be bound by our shared hatred of Lotus Notes.

TK

OSGi Talks

2009-10-12T20:15:00.002-04:00

I'll be presenting at the Central Ohio Java User's Group tomorrow, 10/13 on Modular Java with OSGi. This talk will be a discussion of why OSGi is needed with enough demo code and meat to leave the attendees ready to start exploring OSGi on their own.

I've also been fortunate enough to be selected to present a 1/2 day pre-compiler at Codemash on OSGi. This talk will quickly cover the gaps in Java that necessitate OSGi in addition to exploring, via hands on construction, the patterns and tools for successful OSGi adoption. We'll basically spend 3+ hours building an enterprise Java system with OSGi from scratch! I'm pumped and honored to be hosting the pre-compiler so please sign up if you want to get a 1/2 day submersion into building better Java applications that are more flexible, maintainable and testable than previously thought.

Interviewing Tip #3 - Focus on Fundamentals

2009-08-22T21:33:00.003-04:00

Apologies for the significant delay, but I wanted to pick up the interviewing tips to get at least two more topics off my chest before turning to other posts.

Jon Kruger is a coworker of mine and I'd clone him and staff 5 of him on every project if I could. Lately, I've been striving to understand what the qualities are in Jon and some other elite developers I've worked with that increase the velocity of the teams they work on. I'll expand more on the positive and negative characteristics of elite developers in later posts, but in regards to interviewing, one of the big things that I now focus on are fundamentals.

I was fortunate to have some good mentors early in my career who pushed me to truly understand the pros and cons of every possible solution. That drive lead me into study of object oriented programming, evolutionary design, and foundational principles of development like SOLID. Uncle Bob, Bertrand Meyer, Martin Fowler, and others instilled within me, if nothing else, enough questions to keep pushing to look at solutions from all angles. I'm far from being the authoritative source on which solution is "best", but I will always continue to ask questions to try to find pros and cons of different approaches.

I've at times let people slide in an interview without much pressure on their foundational knowledge, but no more. I could care less at this point if you know whether SQLException is a Checked or Runtime exception in Java. You better be able to talk through the tradeoffs of having it either way, though. I used to dismiss Bruce Eckel's suggestion of asking potential candidates "What's the object model of a chicken?". I have used it a number of times though and it's a great question to start probing how someone may decompose a business domain into OO concepts. If you get scheduled for an interview with me and you can't argue why you would or wouldn't favor composition over inheritance, then we're not ready to work together. I'm not naive enough to think that all interviewers are like me, but if you can talk at a foundational level on design tradeoffs, I can't imagine an interviewer not wanting to talk further with you or hire you the very same day.

Jon's post here is a great call to arms for software devs. If your interviewer focuses on the APIs of Silverlight or MVC, and not on the benefits or drawbacks of either solution, you need to question whether you are interviewing with the right company. Similarly, if you can't articulate a number of outstanding refactoring items from your last project, or openly debate the benefits and drawbacks of some design decision you made on that project, then you probably need to brush up more on the fundamentals of programming and design before you look at making a career change.

Interviewing Tip #2 - Don't be a Commodity

2009-05-04T20:41:00.005-04:00

Never forget that the interview process is no different than any other sales process. You are selling your goods to the interviewer and trying to convince them that you are the best candidate for the job. If you read the last tip then you know that you are also a consumer in this process, looking for the best career opportunity. The selling goes both ways, but for now, we'll focus on what the interviewer is looking for in a candidate.

In these times, it's a given that there are a number of viable candidates. It is inevitable that many of the competition will adequately answer the technically focused questions that are thrown at you. The fact that you know Spring, NHibernate, Cucumber, or MVC does not separate you from the competition. Ok, well if you know Cucumber it probably does, but what are the odds that you'll get questioned about it? My point is that Java developers who know Spring and Hibernate are commodities. Candidates need to articulate the other, softer skills that they bring to the table because they stand out.

Separate yourself from the pack by showing your passion for software development. The fact that you are learning Erlang in your own time may not be applicable to my immediate project need, but it surely shows that you are honing your skills in your own time. Furthermore it shows that you are trying to get better at what you do and that you have some ability to learn independently. Employees like that are assets in any organization.

Covering the technologies used on your past projects doesn't have nearly as much impact as communicating how you stepped into the QA role on the project because there was a resource gap. Or how you were able to reduce the time spent in system testing by setting up a CI server to eliminate defects. Or how you worked over the weekend to fix an issue with the production system that had plagued the company for years. These instances tell a story that your technical responses never do.

Leadership, passion, independence, and drive are the factors that sway decision makers. Don't let yourself get continually attacked with technical questions. Answer them accurately and succinctly, but steer the interview towards the softer skills and show where you've gone beyond your role to ensure success on your projects. That is where you'll separate yourself from the commodity and win the job!

Interviewing Tip #1 - Become the Interviewer

2009-04-30T10:23:00.002-04:00

This economy blows. Let's be clear about that. Nationwide unemployment rose to nearly 9% in April and IT is not immune. I've had the first hand experience of letting people go who I would love to work with again and I cannot describe how painful that experience is. We've had a number of clients cut projects short, shelve them completely, or force a reduction in rates regardless of their contract. IT and consulting jobs specifically have become extremely competitive as a result.

This post (and some subsequent ones) will be targeted to anyone who is currently out of work or in a consulting role, interviewing for one of the precious few jobs out there. Please don't kid yourself, you will be one of many potential candidates who are vying for jobs and a lot of candidates are out of work and willing to come in at a lower salary/bill rate. Separating yourself from the masses is even more critical in this environment.

I have the unique position of sitting on both sides of the interview desk. I interview about 50 people per year and also compete as a candidate for some consulting engagements. I'm going to try to let people know mistakes that I, my consultants, or my candidates have made and areas where we've excelled.

Tip #1 - Become the Interviewer
As an interviewer, if I don't feel like I want to work with you, I probably won't hire you. More importantly, if I don't feel that you want to work with us, I definitely won't hire you. I've worked with people who have no passion to come in on a day to day basis and help me solve problems, and they are a huge anchor to our team's productivity. They create tension and animosity among teammates, they slow the overall velocity of our team down, and generally do more harm than good.

If you want to dispel that feeling in your interviewer, then you need to let them know that you are eager to work in the position in front of you. I end every interview by asking a candidate if I can answer any questions that they have. If the candidate has not asked any questions to that point and still has nothing to ask me, I'm pretty much assured that they have no real interest in building a career at our organization. They are looking to fill a monetary gap in their life and I have no interest in helping them out.

Conversely, if the candidate responds with questions about the team, the environment, the projects, the challenges, the organizational structure, the benefits, the strategic direction, the color of the walls, the quality of the seats, the use of Lotus Notes, etc... then I know that they have a genuine interest in working with QSI and they are doing their homework. Further, they obviously are looking at making an educated decision on their career and not just getting a job. The interview should always be bi-directional, don't just get peppered by the interviewer.

Also, please recognize that the astute interviewer will draw some conclusions about you based on the questions you are asking. Candidates need to focus on asking the questions that are most important and critical in determining whether they want to work on this project. If you are only asking me about money, vacation time, health benefits, and the like I am pretty confident that your deciding factor is strictly based on compensation. I personally get much more excited when I'm interviewing someone and they are asking about the team makeup, type of project, challenges, technologies in use, and opportunity for career growth. Just please, don't ask me about our use of Lotus Notes, I don't want to scare you away.

2009 Goals

2009-01-21T20:25:00.002-05:00

My good friend Jeff tagged me to blog my goals for the year. I'm glad he gave me a little inspiration for this as my personal goals are something that's been floating around in my head for some time. With the exception of the first two they are in no order. Here they are, etched in the annals of the internet.

1.) Delight my wife
2.) Be a better father to my 2 girls
3.) Exceed the expectations of every customer I come in contact with
4.) Blog every other week
5.) Speak every quarter at a user group or conference
6.) Contribute to an open source project
7.) Hire 5 people smarter than myself (low barrier, I know)
8.) Exercise 3 times a week
9.) Read 12 books
10.) Brew 4 times

Burning Media from Mac OSX

2008-12-14T20:28:00.003-05:00

Repeat after me, "Sissies use GUIs. Sissies use GUIs. Sissies use GUIs."

Okay, now that we have that covered here is the command I continue to forget when trying to burn an iso onto a CD from my macbook pro.

hdiutil burn ~/Downloads/ubuntu-8.10-desktop-i386.iso

Beats fumbling around with Finder any day.

Howdy Parnter!

2008-12-09T08:23:00.002-05:00

I'm pleased to announce that Quick Solutions, Inc. is now a certified Systems Integration Partner with SpringSource.

The Spring framework is the standard for Java development. SpringSource has built upon that momentum with a keen vision in developing products and services that benefit enterprise customers using Spring. Whether it is the OSGi enabled dm Server, an instrumented version of the Spring offering with a common monitoring application, or a support model that bests Oracle, IBM, or JBoss, the future of Java development is a very clearly dominated by Spring and SpringSource.

Quick Solutions is driven by providing software based solutions to our customer's problems. We strive to do this in a manner that passes the most value on to our clients. Partnering with SpringSource will allow us to clearly articulate and demonstrate the value that their products generate within Enterprise Java development. We look forward to working closely with Rod and the gang in 2009 and beyond.

Stay tuned for some more information on making best use of the products in the SpringSource umbrella.

Thank You SchemaSpy

2008-10-14T10:55:00.008-04:00

I love open source contributors. These people are the geeks who bypass reality shows, online gaming, and time with their families so that they can bring you something that will make your life better. They get nothing in return but a "atta boy/girl" and a pat on the back.

I am taking it upon myself to give thanks to a few of these open source contributors over the coming months. I am a typically just a taker when it comes to open source, but I would like to at least let the givers know, that I appreciate their hard work and commitment.

The first shout out goes to the kind contributors of SchemaSpy. SchemaSpy allows you to generate a web based ERD of an entire database, that is easily navigated by clicking on images throughout the web application. It's a simple jar file that can be downloaded and executed on any platform that has java installed. It supports a wide variety of databases and produces a ton of information in an easy to digest medium. The first thing you'll be presented is the table listing:

This view gives you all of the tables in the system, the number of rows in each table, and a number of other tables directly related via foreign key to this table.

When you click on one of the table names you are presented with the more specific table view:

The top level of information is like a describe of the table, but checking some of the boxes will allow you to see navigable parent and child rows for each of the columns.If you click on the images in the mini-ERD at the bottom of the page, it will navigate to the appropriate tables. Also note that you can expand the mini-ERD to 2 degrees of relationships.

These features in and of themselves are enough reason to automatically document your database with this tool, but another really applicable portion of the tool is the ability to identify anomalies within the database. Clicking on that tab provides you with this view:

Here you'll see a number of bad database smells very quickly. Unleash a DBA worth their weight on this page and you'll have some table alterations done very quickly.

I honestly see no reason to not use SchemaSpy on any project that we have. It is extremely easy to use, can be integrated into a build, and provides better reference documentation on the database than the vast majority of projects I've worked on. Well done SchemaSpy crew, and a sincere THANKS!

Book Review: The Productive Programmer

2008-10-11T16:00:00.000-04:00

When I heard that Neal Ford was releasing a book on programmer productivity, I anxiously went to Amazon to pre-order it. Problem was that it was almost two years ago and Neal's busy schedule has kept him from finishing it for quite some time. Now that it is finally out and I've read through it (twice actually), I would highly recommend it to anyone doing software development on a daily basis.

I've seen Neal speak at a number of conferences like Codemash, NoFluffJustStuff, and eRubycon, and I've always walked away with something that will improve me as a software developer. The reason is fairly simple, Neal is at his core a developer, and more than many other speakers, he understands the problems and challenges that face developers.

The The Productive Programmer is really two smaller books slammed together.

The first part is focused on tools and techniques that help you work. I actually re-read the book a couple of times because I did a horrible job of keeping track of the utilities and power toys that I wanted to download my first time through. The second time I went through it I started putting post-its on the pages where I wanted to remember a command or download a utility that made sense. The book was littered with post its by the end of it. Neal provides a number of utilities and techniques for both Mac and Windows platforms that will allow you to work more efficiently in your day to day environment. *nix environments are notably absent here, but the command line advice provided for Macs, are obviously equally useful across any *nix distro.

The second part of the book is largely focused on how you build software. A large majority of Neal's talks at conferences carry this common theme. Neal is adept at identifying and explaining principles and patterns of development that yield the best results. Metaprogramming, TDD, Fluent Interfaces, Polyglot programming and many other topics are covered in an easy to digest manner that harkens back to the bible of software development: The Pragmatic Programmer. Neal is able to provide concrete examples throughout the book, based on his vast experience in consulting through the years.

The productive programmer is easy to read, full of valuable information, and applicable to junior developers and seasoned veterans alike. Again, I would highly recommend it to anyone developing software on a daily basis.

Looking for Smart People

2008-10-06T22:13:00.003-04:00

I tend to mention my company, Quick Solutions, very little in order to keep some sense of journalistic integrity with this blog. Unfortunately, I'm not a journalist so I'm going to skip the pretense for at least one post.

I started working at Quick Solutions 5 years ago this month for the sole reason that I was largely unchallenged in my daily job. My thinking at the time was consulting would expose me to a variety of businesses, technologies, and projects. I was fortunate enough to prove my commitment to the company and demonstrate a sufficient breadth of knowledge that I was offered an opportunity to work within the solutions group. At the time I thought that the only difference was the type of work (project versus role based consulting).

I quickly realized however that there was something else about the group that was different. This group was choc-ful-o-geeks. And I mean that in the most sincere way possible. Conversations within the group revolved around continuous integration offerings, Java and .NET trade offs, best practices for SOA, and dynamic language use instead of the typical bitching about a lack of requirements and rehashing of the previous night's reality show. The group was filled not necessarily with Mensa members, but instead with very bright software engineers who cared about their craft. Egos are checked at the door and you could be sure that every person in the door shared a passion and ability to learn that tends to permeate good software developers.

Our company has grown throughout the years and even in a downturn of the economy, we are in dire need of people who meet the above description. If you like working with smart, passionate people please drop me a line. We are always looking to bring people in who show a passion for technology, an ability to learn, and a nack for solving problems. Additionally, if your specialty is Java development, I can give you your choice of 3 different, exciting projects, right now.

If you think you meet these criteria and are interested in working with more people of your ilk, email me at tkaufman at quicksolutions.com.

Clustered Scheduling with Spring and Quartz

2008-09-19T15:52:00.003-04:00

I initially cut my teeth as a Java programmer writing some batch JDBC programs to update various sets of data. We were deployed in a Unix environment so we traditionally wrapped all of the JDBC programs in a bash shell script and kicked that thing off via a Cron entry. This environment was successful for the most part but there were issues. If our batch machine was taken offline due to upgrades, failed disk, or coffee spill then our batch process just did not run. If we wanted it to run after the fact we had to create another 1 time cron entry to kick the thing off. This typically took about 2 hours because the first hour and a half was spent with me trying to decrypt the cron syntax (minutes first or seconds? need a question mark here but star there? comma or dash between my minutes?).

Fast forward to 2008 and I have just completed a week long adventure with a coworker finally getting Spring and Quartz up and running to kick off some batch programs in a clustered environment. Failover is automatic, ad hoc runs are possible, and I still wind up spending an hour and a half each time I have to add a cron entry. I thought I would post some code snippets with explanation so that you kind reader, could maybe trim this process down to about a day.

1.) Download Spring 2.5.5 and only use the Quartz 1.6.1 RC1 jar that is bundled within the lib directory of it. DO NOT USE A PREVIOUS VERSION OF QUARTZ.

2.) Execute your appropriate database script. They can be found in the quartz distro under the docs\dbTables subdirectory. Make sure that indexes are setup as outlined here.

3.) Wrap a batch processing service with some form of Quartz. Here's the wrapper we used:

public class GenericQuartzJob extends QuartzJobBean
{
    protected Logger logger = new Logger(getClass(), Constants.LOGGER_APP_NAME);

    private String batchProcessorName;
    
    public String getBatchProcessorName() {
        return batchProcessorName;
    }
    
    public void setBatchProcessorName(String name) {
        this.batchProcessorName = name;
    }
    
    protected void executeInternal(JobExecutionContext jobCtx) throws JobExecutionException
    {
        try {
            SchedulerContext schedCtx = jobCtx.getScheduler().getContext();
            ApplicationContext appCtx = 
                (ApplicationContext) schedCtx.get("applicationContext");
            IBatchProcessor proc = (IBatchProcessor) appCtx.getBean(batchProcessorName);
            proc.invoke();            
        }
        catch (Exception ex) {
            logger.error("Unable to complete execution of " + batchProcessorName, ex);
            throw new JobExecutionException("Unable to execute batch job: " + batchProcessorName, ex);
        }
    }
}

With that wrapper, you can execute any batch processor that is wired up in Spring, that implements the homegrown IBatchProcessor interface (which typically only has some variant of an execute or invoke method). You don't need to manage the dependencies of those batch processes as they themselves are just beans defined a Spring app context somewhere. Additionally, the base class jumps through the various contexts that you must navigate to get a properly wired bean from the Spring factory.

4.) Configure the Job Detail in your Spring app context with an xml snippet resembling this:

    <bean id="someJobDetail" class="org.springframework.scheduling.quartz.JobDetailBean">
        <property name="jobClass" value="com.some-company.batch.GenericQuartzJob" />
        <property name="jobDataAsMap">
            <map>
                <entry key="batchProcessorName" value="SomeJobBean" />
            </map>
        </property>
    </bean>

This definition creates an instance of GenericQuartzJob (which fulfills the contract required by JobDetailBean), and plugs in the bean name of a batch processor defined somewhere else in the app context with all of it's necessary dependencies.

5.) Configure the Trigger in your Spring app context. A simple cron based version would look like this:

    <bean id="someCronTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
        <property name="jobDetail" ref="someJobDetail" />
        <!-- Cron expression runs at 1am and 1pm -->
        <property name="cronExpression" value="0 0 1,13 * * ?"/>
    </bean>

And yes, it did take me an hour and a half to get that cron syntax working correctly. Some habits die hard.

6.) Configure the SchedulerFactoryBean

    <bean id="scheduler" class="org.springframework.scheduling.quartz.SchedulerFactoryBean" lazy-init="false">
     <property name="applicationContextSchedulerContextKey" value="applicationContext" />
        <property name="dataSource" ref="qtzTxDataSource"/>
        <property name="transactionManager" ref="transactionManager"/>
        <property name="overwriteExistingJobs" value="true"/>
        <property name="autoStartup" value="true" />
        <property name="triggers">
            <list>
                <ref bean="someCronTrigger" />
            </list>
        </property>
        <property name="quartzProperties">
            <props>
                <prop key="org.quartz.scheduler.instanceName">SomeBatchScheduler</prop>
                <prop key="org.quartz.scheduler.instanceId">AUTO</prop>
                <prop key="org.quartz.jobStore.misfireThreshold">60000</prop>
                <prop key="org.quartz.jobStore.class">org.quartz.impl.jdbcjobstore.JobStoreTX</prop>
                <prop key="org.quartz.jobStore.driverDelegateClass">org.quartz.impl.jdbcjobstore.oracle.weblogic.WebLogicOracleDelegate</prop>
                <prop key="org.quartz.jobStore.tablePrefix">qrtz_</prop>
                <prop key="org.quartz.jobStore.isClustered">true</prop>
                <prop key="org.quartz.threadPool.class">org.quartz.simpl.SimpleThreadPool</prop>
                <prop key="org.quartz.threadPool.threadCount">25</prop>
                <prop key="org.quartz.threadPool.threadPriority">5</prop>
            </props>
      </property>
    </bean>

That configuration while lengthy does a few very important things. Let's go through them. The applicationContextSchedulerContextKey property will ensure that the app context is available to all of those instances of the GenericQuartzJob wrappers. The dataSource and transaction manager are necessary in order to ensure that the database is updated in a safe manner (many server instances may be updating the database at once). Their bean definitions are pretty much typical for Spring, checkout the Spring reference docs if you need more info there. OverwriteExistingJobs will make sure that every time the scheduler is started it will use the list of triggers found internally to overwrite any existing ones that may have been changed in the database. AutoStartup makes sense, but I'm not entirely sure it's necessary.

The quartz properties can be maintained in a separate file, but I prefer they are inline with the scheduler definition. They are all pretty much self explanatory, and explained in greater detail, but nested deeply in the Quartz docs.

Hopefully this article will advance you to the point of configuring a clustered, database backed, scheduling system within hours instead of days. This configuration will get you a set of spring enabled batch processes that have dependencies wired as normal, with the added benefit that their schedule is persisted and fault tolerant across nodes in the cluster. This also leaves the door open for a couple of methods of doing ad hoc runs of the jobs. We'll cover that in a part 2 article shortly.

eRubycon 2008 - Severely Belated Wrap Up

2008-08-16T09:22:00.006-04:00

I had the good fortune of attending eRubycon a few weekends ago and I can honestly say that I've never seen more value in a conference. My previous rant on how NFJS is letting the level of their speakers slip was only more evident in the 3 days spent at eRubycon. Neal Ford, Stu Halloway, Joe O'Brien, Jim Weirich, Charles Nutter and many others contributed valuable sessions that left me wishing the conference went on for a few more days.

The great thing about eRubycon was that it very rarely focused on what is typically the gateway drug in adopting Ruby, the Ruby on Rails framework. Rails is a multitude of patterns and practices that are worthwhile regardless of your current language, but it sometimes overshadows the true core of "beautiful code", Ruby.

The conference left me wanting to learn more about Ruby and to generally spend more of my time at work adopting Ruby. The challenge I (and many others have) is to do this while still meeting client requirements. Brian Sam-Bodden gave a great visual representation of how a Java project can incrementally adopt Ruby as a build language, scripting language, and testing framework. I missed half his presentation as I was so focused on pulling in buildr and some other utilities into my current project.

I think the two biggest topics covered though were essence vs ceremony and testing your application versus testing your framework. Stu Halloway gave a great keynote on essence vs ceremony and while his Java examples were typically very dated, his points remained true. Java (and C# by extension) force you to type crap that is just clutter or ceremony. The less ceremony in our applications, the easier they are to test, maintain, and actually produce on time. I don't recall who initially mentioned the difference between testing your framework and testing your application, but it was a point that haunted me all weekend. It's a very gray line typically between the two and one that can drastically impact a project schedule. Unfortunately, there is no easy answer to this one, so be sure to be cognizant of it when writing your unit, functional, and integration tests.

In summary, eRubycon far exceeded my expectations for a conference under $300. Speakers, content, hallway discussions, and after hours scotch induced discussions were all way above what I had hoped. Keep it up Joe, I'll see you there next year.

NFJS - Day 3

2008-07-27T09:08:00.002-04:00

Spring Talks by Keith Donald
Spring has largely been my focus for the last few years so I was happy to see one of the Spring Source crew stepping up at NFJS. Keith Donald is the lead of almost all Spring web related technologies and he tends to give a very down to earth, code centric presentation so I devoted all of my final day to his talks.

Someone in the audience asked Keith if he missed the common XML file that shows exactly how all your beans are wired when you move to annotation driven POJOs and that question is an important one. I've recently become much more comfortable with developing Spring applications due to the Spring Facet in IntelliJ. Moving to annotations tends to make me question exactly which resource or component is plumbed in to fill a dependency. Keith mentioned that an emerging best practice is to use annotations for self describing application dependencies, XML for infrastructure concerns. He also gave a soft sell for Spring IDE to show a nice dependency graph regardless of annotations or XML.

Spring 2.5 is downright giddy with it's extensive use of annotations and it didn't really dawn on me until Keith's talk that it might be the better way to go in some scenarios. I've traditionally balked a little bit at the extensive use of annotations as it feels very invasive to the code at hand, especially when the annotations are Spring specific. More importantly, swapping out facilities for testing purposes is not trivial when you are doing component scanning and autowiring with annotations. I think Keith articulates an appropriate scenario for use though. Swapping out mock classes via XML is not a benefit that is needed as much for application dependencies as it is one that is needed for infrastructure (data access, email service, file reader, etc...). Go nuts with annotations at the app level, but use a good editor and keep the XML where it belongs for infrastructure pieces.

Keith spent a good amount of time in all of his talks using an IDE focused on Spring MVC controllers. The thing that stood out to me was how Spring web controllers are truly just POJOWAs now (I know that's not a term that will stick but a distinction needs to be made between POJOs and POJOs with annotations). This means that you will rarely see HttpServletRequest or ModelAndView type classes in these POJOs. What's the benefit here? You don't have to waste many of your test cycles mocking up the container objects. Wanna test your login method on the controller? Send in a User object and a String password and then validate whether you are getting a success or error redirect back. Very simple, and anything that we can do to lower the barrier of entry to good testing is well worth it.

Keith demonstrated another key change in coding Spring apps is the impact that Rails has had on the Spring community. The focus has shifted in Spring 2.5 to more sensible defaults and a true convention over configuration model. Controllers in Spring web apps now mimic Rails controllers in URL and request parameter resolution. Additionally, there are a set of special parameter types that are automagically injected. An arg of type Principal will be supplied by the container if the user is already logged in. Very cool.

One final note on the Spring web side. Why do we have another javascript framework? If you have improvements to make to DOJO, please just contribute them to DOJO. Don't create an abstraction layer over DOJO that does what you need. We have finally whittled the .js framework comparison down to a handful in my mind (prototype/scriptaculous, dojo, jquery, yui, and ext). Don't lead us back in the other direction.

Conference conclusion
NFJS again provided some immediate exposure to worthy technologies and processes. I was able to touch base with a lot of people in the Java community that I hadn't seen in some time and I feel like I have more tools in my belt than when the conference started. A special note to the organizers though. If the prices go up again next year while the number of conferences goes up as well, then you won't see me there for the first time in 4 years. It was evident to me this year that the conference is not the value that it was 2 and 3 years ago. If the trend continues, I'd rather spend my training dollars on codemash, erubycon, and books.

NFJS - Day 2

2008-07-26T09:12:00.007-04:00

Stu Halloway's Refactoring Javascript was a topic I did not expect to see at NFJS this year, but I walked away truly impressed. Stu presented a totally different mindset on developing, testing and refactoring javascript code. The gist of it is to use a mock browser that will allow you to run tests against your javascript code in an automated environment. Here is the mock browser but it is still in a beta state so expect to have to bend it to your will. Stu made extensive use of JSSpec for writing BDD specs against an existing javascript framework (livepipe). It was truly telling in the presentation that the first hour was spent trying to get a solid safety net of tests up and passing before we even began thinking about refactoring. Stuart accurately identified this as the norm when refactoring legacy code and it lines up directly with my experience as well. Refactoring can't be done w/o that safety net. When we finally did get around to refactoring, Stuart boiled it down to four areas:

Extract and Shorten
- If your method does more than one thing it does too much
- Keep method size to around 5 - 7 lines

Reduce Clutter
- Nix out of scope variables
- Keep the effort to duplicate a method greater than the effort to understand it

Choose Good Names
- Probably the most difficult of the refactorings
- Definitely the refactoring most in need of pair programming

Use Existing Libraries
- If you are typing document.getElementById then punch yourself first and download prototype.js second.

The talk was extremely hands on and valuable and made me wish I had a bucket of crappy javascript to go back to on Monday. Other things mentioned included another mock browser env: Crosscheck. Stu also mentioned a lack of code coverage tools for Javascript, but I found one during his talk. One last thing, he mentioned a very important refactoring anti pattern. Mirror methods (get and set on cookie) used in each others tests. If you are testing the get method on a cookie object don't explicitly use the mirror method (set). Instead, roll it by hand as a bug in code shared by these methods could offset each other in the test execution.

The second talk I went to was Stu's GIT talk. I've been looking into GIT for a little while and Stu gave a good gentle intro into it by basically walking through scenarios at a command prompt for 90 minutes. That format doesn't necessarily translate to a blog so here are some highlights:

- When using GIT, think less in terms of verbs
- GIT is not SVN++
- It's crazy fast. Branching and tagging for instance are instantaneous
- The entire source history is in every copy of the repo
- It doesn't store deltas, it stores the begin and end and calculates the deltas.
- You are forced to tell git what you are committing. It's not tracking it like SVN
- GIT encourages agility. You may not know what makes a package/release/hunk of functionality until sometime in the future. Additionally, you can stash your current task deliverables, go do something else, and come back to them later.
- Bisecting allows you to do a binary search through releases of code to identify the release that caused a bug. Kick ass.

Metaprogramming in Groovy by Brian Sam-Bodden was the third session I attended. BSB did a good job of going over the major metaprogramming techniques in Groovy. Additionally he had enough examples of Groovy code with TextMate that forced my hand to download it and see if it will cure what ails me with IntelliJ in dealing with Rails and Grails. Stream of conscience notes on metaprogramming with Groovy:
- Much simpler than using dynamic proxies in Java
- Similar to ruby it allows on the fly evaluation of code via evaluate
- Duck typing like ruby with respondsTo
- See yesterday's bits on methodMissing and invokeMethod
- obj."$methodToCall"() is valid Groovy code that will try to call the contents of methodToCall as a method on obj. In my opinion that code is utterly disgusting.

Also, the inevitable Ruby v Groovy topic came up as it did at lunch with me and a few others. There still seem to be zealots on both sides and I wish for the life of me they would just shut up.

My 2 cents are this. Use whatever makes most sense for your environment and team. At my consulting company, that is Ruby b/c I have a lot of .NET fellas that I would like to roll into and out of projects whether it is deployed on MRI, JVM, or DLR. The differences at my admittedly noob perspective are few and far between with the two languages. If it was a room full of Java developers, Groovy might make more sense because it is an easier transition for Java heads. I don't care what you choose for your environment, just quit bitching about the other side.

EJB3 Testing by Joseph Nusairat was my final session of the day. Joseph covered a wealth of testing frameworks in a very code centric presentation. The session was very interactive and covered some cool mocking techniques outside of the realm of EJB3, like dealing with mocking static methods. My attention span was slipping this late in the day, but here is a set of highlights:
- Easymock provides the typical set expectations, record and playback functionality. It is still probably the most proven, well documented, and easy to learn of the mock frameworks.
- JMockIt is a relatively new player with some cool support for AOP and Hibernate. Please notice the IT at the end of the name. JMock is a different framework. Unlike EasyMock and JMock, JMockIt is a very small and straightforward API that can be used to test methods w/o dynamic proxies or cglib.
- EJB3Unit has some very cool facilities. It provides an in memory database out of the box (HSQLDB), uses CSV files for populating the data and provides mocks of almost everything that you would need in an EJB 3 bean. If you have to use EJB3, this framework seems like a valid contender.
- Embedded JBoss is the last one covered and it seems like a good fit for mocking out an entire container when necessary and using JBoss.
- Mocking for unit testing, other tools still necessary for integration and functional tests.

I'm tired. See you tomorrow.

NFJS - Day 1

2008-07-25T12:30:00.006-04:00

PreConf Arrived at NFJS. 2 cents before the conference gets started. Having the conference in two places like Phoenix and Columbus on the same weekend is diluting the talent (and the costs have gone up). Phoenix wound up with a few of my favorite speakers on the planet: Neal Ford, Venkat Subramaniam, and Ted Neward. Please don't get me wrong, listening to Stu Halloway, Scott Davis, and Keith Donald among others still makes for an excellent conference. However, in previous years, having to choose between Justin Ghetland, Stu Halloway, Dave Thomas, or Ted Neward all in the same time slot was a nice problem to have. If NFJS is to keep it's mantra as a value driven conference with A+ speakers then it needs to keep it's shows to 1 city per weekend.

Scott Davis - Groovy the Blue/Red Pill
Scott Davis has a presentation style that is really conducive to learning new technologies. He speaks at a very measured pace with a number of pauses, yet still with a passion that gets users excited. His first two talks were Groovy, how it is similar to Java and then how it breaks the mold.

Groovy - The Blue Pill covered the language at a high level and how it is seamlessly integrated with Java. I spent a good portion of this talk weighing whether that is a good or bad thing. I as a 10 year Java veteran have developed a very Java centric mind and it continues to bite me in the arse. When anonymous inner classes become natural for you in a programming environment, you have issues. Ruby seems to allow me to make a clean break from this mold. Groovy is a step up, but not in the same sense. That being said, I think it's seamless integration with Java make it an easier sell to management types. Hopefully the JRuby guys can keep up. Stream of conscience thoughts on the talk:
- GStrings while oddly named are very elegant
- Mixing in Groovy based unit tests is a no brainer for ease of adoption and powerful mocking
- Triple quotes surrounding XML blocks makes you ignore quotes and newlines (Cool for XML)
- Groovy converts all checked exceptions to unchecked exceptions
- Powerful scripting: s.class.methods.each{println it}

Groovy - The Red Pill covered some of the more interesting pieces of meta-programming in Groovy. Here is where the obvious Ruby influence shows up in Groovy. The ExpandoMetaClass accommodates closures in a syntax that is less intuitive then blocks in Ruby to me. It's called ExpandoMetaClass though so bonus points there. The keyword delegate can be used to access the thing within EMC. Overloading operators is limited to a specific set and done in a manner that is less than intuitive (implement putAt(b,c) for a to get a[b]=c). Groovy also provides methodMissing and invokeMethod to get peeps closer to Ruby like syntax. My conclusion after this session was I'll use Ruby wherever I can, then Groovy, and finally Java.

Scott Davis did such a nice job presenting the first two Groovy talks, I stuck around for his Grails for Struts Losers talk. That wasn't the exact title, but I think I captured the gist of it. This was a gentle intro to Grails and managed to show it's benefits quite nicely over existing Java web frameworks (struts in this case). The framework is very clearly alot of the concepts of Rails that work well in a Java/Groovy centric environment. I spent the session trying to get Grails up and running in my environment and stumbled into an issue that clearly articulated Grails v Rails. I left parens off a method call and it blew up. WTF? Basically it was trying to find a property of that name instead of finding a method. Luckily, I had a Grails book co-author next to me to help troubleshoot. For those of you keeping track at home, Rails 1, Grails 0.

Tech Ridden Month

2008-07-21T09:00:00.002-04:00

I have 3 different blog posts in the making right now and I unfortunately don't see myself having enough time to complete them in the next month. A wide variety of activities are chewing up almost all of my free time, but most of it is tech related. Here's a look at my schedule:

7/22 - Speaking about Spring at the Cincinnati Java User's Group

7/25 - 7/27 - Attending NoFluffJustStuff in Columbus

7/31 - 8/4 - Vacationing with wife in Cabo San Lucas (WOOHOO!)

8/15 - 8/17 - Attending eRubycon in Columbus

I'm extremely excited about all of these opportunities but it will nonetheless continue my procrastination of blogging. Please sit tight and check back in mid August for more content on Spring, Ruby, IntelliJ, and IT Leadership.

Ruby Integration with Spring

2008-06-10T20:22:00.001-04:00

I recently gave a presentation at COJUG on ten facilities in Spring that you may not know much about. I saved my favorite for the last. Dynamic Language integration. This facility has been around since Spring's 2.0 release, but has been mired in the relative obscurity of Chapter 24 of their docs.

While many IT managers might be hesitant to adopt a language like Groovy or Ruby for their production environment due to misconceptions about performance, security, or what have you. The management types should have very little concern with testing code making extensive use of dynamic languages. If Ruby or Groovy provides a lower barrier of entry for complex testing or mocking, why not make use of it?

Spring provides the typical DI goodness of abstracting away the underlying implementation and allowing the developer to switch from mock to production code without changing client code. In the case of Ruby backed implementations, JRuby provides the hooks to allow the .rb file to be recognized as an implementation of the Java interface.

Here's a simple interface implemented in Java with which I've created a Java and a JRuby implementation.

A JRuby based implementation of this only has a few tricks like including the library and calling a method to include the class

Finally, with the spring integration facilities, a few lines of xml locates the file and plumbs in the implementation. Client code accesses the Java interface, none the wiser to the Ruby implementation.

There are a couple of gotchas to keep in mind with JRuby/Groovy Spring integration:
- JRuby 1.1 is not supported until Spring Framework 3.0, use JRuby 1.0
- Model code after what I have above, not what is in the Spring reference docs. At last check the docs were somewhat confusing in the use of include versus include_class
- AOP can be used to wrap the dynamic beans just as if they were POJOs. The one big caveat here is to make sure they are implementing an interface. Class based proxies will not work

Spring's support for dynamic language beans is pretty powerful in it's simplicity. The uses of this technology are much farther reaching than just mock scenarios, though. JRuby or Groovy files can be polled and reloaded without compilation or server restarts. This can be immensely useful for dynamic portions of applications like lightweight rules, environment configuration, validation, or even web site navigation.

Hope this helps get you started with integrating dynamic languages into your Java code. Please leave a comment with your scenario if you are using or planning on using dynamic languages integrated with Spring.

Software Development Meme

2008-06-09T09:08:00.005-04:00

Mark Blankenship posted his own personal history of software development and charged a few of us with posting the same. Here's my background:

How old were you when you started programming?
I remember my first official "programming" class was my sophomore year in high school so roughly age 15. I was officially playing around with a Commodore 64 as early as age 8 though. While that was mostly gaming, the memory and file models puzzled and intrigued me back then. My cousin and I would waste hours playing Mission Impossible on the C64 though. That tended to get in the way of any real programming back in the day, establishing a pattern of gaming over real work that plagues me still.

How did you get started in programming?
My brother is 5 years older and he was already starting to see some serious need for programmers while he was off at college. I was but a young lad in high school, focused more intently on sports, girls, and parties at the time. When I took my first real course at his advice though, I was hooked. So, brotherly encouragement coupled with a strong background in math. My father is a science teacher as well so he was always giving me a slight kick in the rear to pursue any and all topics that stoked my interest.

What was your first language?
BASIC, baby! I remember sitting next to a guy in that first course who nearly had the mental capacity to bend spoons and levitate people. He scored a 35 on his ACTs and a 1520 on his SATs, no joke. He was scary smart. I believe he later became a nuclear physicist, but I'm not totally sure. Anyway, he wrote a 5 level Zork type game in that class in the first two days we were there. That opened my eyes to the power of programming. Thanks Bill!

What was the first real program you wrote?
The first open canvas program I wrote was in a computer graphics class my junior year in high school. I built a dart board type game that allowed two people to compete to see who could get the dart closest to the center. By compete I mean that you merely alternated pressing a button to throw darts and they went to random spots on the board but it was still pretty cool.

What languages have you used since you started programming?
Probably missing some but here goes:
BASIC
Pascal
Modula-2
C
C++
Java
Smalltalk
Lisp
Fortran
SQL
HTML/Javascript
Ruby
C#
ASP.NET

What was your first professional programming gig?
I hired on as a paid intern thanks again to my bro. I was told I would be working on a C/C++/SQL based project for a retail services company. Instead I was working on an S1032 database with Fortran code literally printing off reams of green bar code snippets. If that place didn't have a ping pong table on site that rolled out every day at 5pm I would've probably gone postal. That assignment nearly made me into a history major.

If you knew then what you know now, would you have started programming?
No question about it. If anything I would've been more driven to succeed in this profession and spent more of my free time learning and pushing my limits. I was largely a slacker through high school and college and it set me behind starting out in the real work force. While it did enable me to thoroughly dominate OSU's entire campus in NHL 93 on the sega genesis, it also nearly caused me to drop out, become disowned, and live life in a van down by the river. I was lucky to find my niche once again but I wonder what I would've accomplished if I would've truly applied myself.

If there is one thing you learned along the way that you would tell new developers, what would it be?
Get comfortable with change and figure out the best way for you to learn. Change comes at you at all times from all directions in this field. Requirements are never set in stone, business direction can sometimes sway with the wind, and technology will certainly never be static. While Java has been around for 14 years now, the language it is today is worlds apart from what Gosling and co pushed out to us in the form of crApplets in 1994. All of the people in this field that I want to surround myself are quick learners and very adaptive. If you can roll with the punches figure out things quickly, then you'll never cease being marketable.

What's the most fun you've ever had ... programming?
I'm not sure I've ever had a moment of euphoria while writing a piece of code, but there was one time when I solved a huge showstopper of a bug that was visible at every level of our company. We had been seeing our application lock up the server completely on a random basis for about 2 weeks. After code inspection day 10 I finally stumbled upon an infinite recursive subroutine that was nested 142 levels deep in app server that was impossible to debug. Finally getting it resolved wasn't fun as much as it was like someone moving a car that had been parked on your foot.

That being said, I think I've had fun at every programming job I've ever been in. I attribute that solely to the various people that I've worked with, though. I have yet to work at a client yet where I've woken up with an utter feeling of dread. Maybe I'm just lucky, but I can rattle off a number of people from every client I've been at who I could call right now to go grab a beer. I think the problem solving is always fun and gratifying but working with people you genuinely want to be around is priceless.

Next up:

I'm throwing the guantlet down for another group of peeps to answer these questions:

Matt Marquand

Kevin Munc

Steve Harman

Jon Kruger

Steve Horn

And then there were 3

2008-05-05T09:44:00.002-04:00

Oracle has now officially acquired BEA, leaving the app server marketplace for Java EE at 3 major vendors with IBM and JBoss completing the trinity. After having personal experience with both BEA WebLogic and Oracle's Application Server, I plea to Oracle, just use BEA's app server and don't mess with it. Orion was a good product and you turned it into a trainwreck. There is no reason that an application server install should take 4 days and 300 gb of memory. JSPs shouldn't be stored in the database. Oh, and your installer should not need to install JDK 1.2 to run. Leave it alone Oracle and focus on improving your DB.

A Polarized Community?

2008-05-01T10:34:00.004-04:00

The news that SpringSource announced yesterday regarding their application server is not surprising but possibly very concerning to many developers in the Java landscape. We see now a very clear divide between people's perception of what is standard for Java Enterprise Development. Is the standard the open source platform that is used in a greater number of projects or is the standard the framework released by the Java Community Process? In my opinion it is very clearly the former.

I'm in the fortunate position of seeing every Java requirement that comes in to my consulting company and guess what? The number of Spring requirements far outweighs the number of EJB, let alone JSF reqs. I still see more Struts requirements than JSF and EJB reqs combined. So now, is the news that not only is there a non-standard framework, but also a non-standard application server something to be concerned with? Hell no.

How many developers have actually taken advantage of the fact that your app can be ported to different app servers with "no code change"? I've been on one project in the last 10 years that did so and guess what? We still had to change a lot of XML to make sure that stuff didn't break. Portability is a farce at the app server level and it's largely unnecessary.

So if there is no need to port my SpringAppServer application to JBoss or WebLogic or god help me WebSphere, what are the issues with adopting SpringSource Application Platform? It surely will present some amount of a learning curve. It may not be adopted by many of my clients initially who continue to write checks to IBM and BEA for inferior products that lag behind the so-called standards.

On the flip side, it will provide the ability to roll out smaller, componentized versions of your applications. It will let you keep multiple versions of the same components running on the same server. It will let you deploy your app without restarting your server. It will cost about $15k per cpu less than BEA or IBM products. Oh, and it will not force you to use JDK 1.4 or prior in order to run on the system.

Is it really a polarized community if everyone is heading in one direction?

ACID Principles Applied to Testing

2008-02-28T09:11:00.005-05:00

Sometimes you don't realize that you have built your house on a fault line. Then the tectonic plates shift and you are left in a pile of rubble wondering, WTF just happened?

I recently found myself in that very position, as it applies to testing applications. We had been focused on writing unit, integration, and functional tests and automating them for a .NET application that was really nothing more than CRUD but with some significantly complex business rules thrown in. We made some mistakes along the way though that finally came to a head earlier this week when 3 out of 3 of our builds were broken, we were running a day late in releasing an application to UAT and we had made a significant number of refactoring changes that could have very well broken 50% of our functionality, but we couldn't tell for sure. Amidst the rubble, I took a moment to reflect on what got us to that point so that I could share with you, kind reader, the err of our ways.

Our biggest mistake in writing tests was not making sure that they were Isolated. I've always been kind of a persistence geek though, so I naturally started noticing how the ACID principles of database transactions apply so well to testing. For reference, the ACID properties are Atomic, Consistent, Isolated, and Durable, see wikipedia.

Atomic - Can't be broken into smaller parts.

Good unit and integration tests should be atomic. Not so much for functional tests as they are typically the assembly of multiple atomic parts into one whole. The unit and integration tests, however, should be focused on testing one feature of any target. Nothing more, nothing less. This makes the tests cleaner, it makes uncovering causes to failures easier, and it helps to differentiate functional tests from unit and integration tests.

Consistent - Repeated execution results in the same outcome.

Consistency, or the lack thereof, typically rears it's head in the integration and functional tests and in my experience, the cause is often broken dependencies. Database integration tests mysteriously start failing because another developer went in and updated some data that your test depends on. Making integration tests consistent requires some work. Typically it involves automatically prepping a database with all of the values and foreign keys populated so that your test follows it's directed path. This can be made much easier with a framework like DBUnit in Java which preps and removes data as needed.

Isolated - Concurrent operations do not have an affect on one another.

As mentioned earlier, this one was a biggie for us. We found that any person running tests while the build was also running them, caused both to fail. This was due to some poor choices in how we were trying to cleanup after ourselves (blindly searching by a non primary key field that picked up data from other tests). The easiest solution in many instances is to use a pattern provided by the bright fellas over at Spring, and start a transaction in your setup method and then roll that puppy back in your tear down method. This allows everything running in that test to see changes made by the code (assuming it uses the same connection as your tests), but it removes the sometimes complex logic of undoing your changes by hand. Additionally, it ensures that even if the same two tests are running at the same time, each will be on a separate connection and totally unaware of the other's changes. See AbstractTransactionalContextTests in Spring.

Durable - Once successful it is permanent.

It should really take a change in business logic or interface design to get a test breaking once it's in a successful state. This goes hand in hand with the previously mentioned anti-pattern, so don't let data or systems outside of your control muck with your tests. Make your functional tests fully dependent on mocks, stubs, or dummies instead of external systems or databases. Make sure your integration tests are completely responsible for setting up all of the data that they need before running. This will lower the probability of environmental issues throwing up a false negative on your CI server.

I've made a distinction throughout this article with Unit/Functional/Integration tests. Make sure you know the difference between them and be adamant about treating them differently.

Unit tests should never interact with a database, email server, web service, or other disparate system. Unit tests should be automated and run upon every code delivery. They should be extremely fast to keep ADD developers from staring blindly out the window or surfing the net while waiting for the tests to complete.

Integration tests should never be mocking the integrated system and in many cases should only be run on a daily or semi daily basis. Unless your database or external system is being modified multiple times throughout the day, just set these to run up when the lights go off at night. If someone is actively changing your entire persistence layer, they should have the common sense to run these manually before checking in.

Functional tests should almost solely rely on mocks, dummies, or stubs instead of external dependencies and since they'll be pretty zippy if you write your mocks correctly. If they are in the "runs in less than 5 minutes" category, then run them at every check in so that the staff is well aware of any issues as soon as a delivery is made.

Keep in mind the ACID principles when testing your code. They'll hopefully keep you away from the fault lines and in the happy state of green builds.

Failing to Fail Early

2008-02-06T21:47:00.000-05:00

A coworker and I stumbled across some weird behavior with NHibernate today that totally surprised me. I come from the Hibernate world which tends to be about a version to a version and a half ahead of NHibernate. So I'm used to going to look for excellent support for stored procedures and realizing I'm SOL.

Today though we found an issue that wasn't lack of support, but just a horrible design decision that differs between the two frameworks. In a nutshell, Session.Load was being called with an ID that didn't exist and we weren't receiving an exception. Huh? The difference between Get and Load in my mind was always, same behavior if the PK passed in exists in the database, if it doesn't Get returns null silently, and Load throws ObjectNotFoundException, violently alerting you that your PK is not what you think it is. In this case we were getting back a proxied object that would immediately throw ObjectNotFoundException whenever one of it's getter methods were called. Nice. I assumed that we were misusing the Interceptor functionality or something to cause this but a little googlage revealed this description of the ObjectNotFoundException class in the NHibernate docs:

"Thrown when ISession.Load() fails to select a row with the given primary key (identifier value). This exception might not be thrown when Load() is called, even if there was no row on the database, because Load() returns a proxy if possible..."

Come again? What use do I have for a proxied object that is just going to throw ObjectNotFoundException the moment I start accessing it? Even worse, what if I have a business method that checks a private attribute for null on this object. You shouldn't be able to do anything to this object because according to the database IT DOESN'T EXIST. When all else fails, go to the pragmatic programmers for advice and listen when they tell you to FAIL EARLY. Give me null or give me exceptions, don't fool me into thinking my row is there when it isn't.

Code Mash Recap

2008-01-24T22:17:00.000-05:00

Codemash came and went over a week ago but I've been hoarding my free time so the blog has gone by the way side. I must say that codemash is as unique a conference as you can find and the organizers do a superb job of providing valuable content and access to bright people. The speakers I saw were very good overall and made you question how the conference is not pummeled with only charging $200 or less.

The highlight for me had to be speaking with Jim Weirich on a couple of occasions about Ruby and his work at EdgeCase. He's the man behind Rake, XmlBuilder, FlexMock, and other pieces to the Ruby puzzle I'm sure I'm forgetting. He was extremely down to earth and very practical about using the appropriate language for the job. He's like DHH with his passion for Ruby and Rails, but without the arrogance.

Additionally, I was reminded why I really enjoy working with the people I do at QSI. We sent no less than 30 people there, had a couple of speaker/organizers, had the most rocking booth of all the vendors, and most importantly had great discussions about software over beer every night there. At least 5 people made comments to me about their desire to learn Ruby and/or Rails. We'll hopefully capitalize on this passion by forming a 12 step program to get us familiar with Ruby (first and foremost) and then Rails. I'll keep the blog updated through the journey to beautiful code...

CodeMash Coming Up

2008-01-02T09:11:00.000-05:00

I was fortunate enough last year to attend CodeMash in it's inaugural event. I can say without a doubt that it is the most unique conference I've had the pleasure of attending. I'm somewhat odd in that I code daily in .NET and Java and usually play with Ruby in the wee hours of the evenings. CodeMash is the only conference I know of where I can go to sessions for all of these topics and still learn about other topics like Python, Groovy, Android, etc...

The speakers aren't just slobs off the street either. Neal Ford, Jim Weirich, Bruce Eckel, and many others will present at CodeMash this year. Always cool to go to a session of someone whose book you have on your shelf. Being able to see speakers of this caliber for less than $200 does not happen often.

I'm always up for a beer after the sessions, so let me know if you'll be attending. I'll get Mark Blankenship to buy us a round.

Oh, and did I mention it's at a waterpark? That's right, 2 conferences within a month for me where I get to hammock up!

Practices of an Agile Developer

2007-12-19T14:03:00.000-05:00

On the flights down to FL, I was able to read a book that my team will be reviewing in the upcoming months, Practices of an Agile Developer by Venkat Subramaniam and Andy Hunt.

The authors chose a style that I think captures the bipolar disorder shared by most cognizant software developers. It presents topics both from the perspective of the devil who might encourage us to "just duplicate this code, don't bother keeping the system DRY" and the angel who would give us a gentle reminder that "duplication is just a shortcut, and you will have to maintain both sets of code later on". It's amazing how many times you find yourself listening to that devil on a daily basis.

The book is very easy to read even if you don't have a ton of time to devote to it. Chapters are usually 30 pages or under and each one is broken into a number of atomic topics. 'Fixed Prices are Broken Promises' and 'Architects Must Write Code' were two which immediately struck a chord with me.

Additionally, the book has a ton of good quotes that should be tattooed on the back of our hands or at least put on a whiteboard in plain view. My favorites are "Blame doesn't fix bugs", "Negativity kills innovation", and "Blindly picking a framework is like having kids to save taxes".

All in all, a very good read and immediately applicable to developers both agile and not.

The Spring Experience - Day 4

2007-12-15T14:39:00.001-05:00

Day 4 started much like Day 3 with another good presentation from Ben Alex regarding Spring Security. This session focused on the enhancements provided in Spring Security 2:

Annotation driven security that allows a developer to specify required roles directly above a method declaration

User management API for creating users and default implementations for JDBC and LDAP backed stores

Hierarchical roles. UPDATE_ALL implicitly grants you UPDATE_CUSTOMER, UPDATE_ORDER, etc...

Many other features like NTLM support, Portlet Security integration, and Automatic login page generaton

The second session I hit was RESTful web services in Spring by Arjen Poutsma. This session was a combination introduction to REST (Representational State Transfer) and highlight of facilities coming soon for developing clients and providers of these web services. It was great to see how similar the EndPoints Arjen displayed were to the Controller classes shown by Keith Donald earlier in the week in the RESTful web sites talk. Despite the number of disparate people working on these separate Spring based sub-projects, they are all adhering to a few common designs that make it very easy for a Spring developer to become productive in each of the environments.

OSGi was next up, presented by Adrian Colyer and Costin Leau. OSGi will provide Java application developers with a facility for deploying and managing separate versions of the same subsystems or libraries. The examples shown were somewhat contrived, but the concepts of OSGi are extremely powerful and immediately useful in at least two of the last three Java clients I've worked at. If you've ever found yourself trying to juggle version collision of components in your application, be sure to investigate OSGi.

Final talk of the conference for me was Spring Batch Internals by David Syer and Lucas Ward. They covered a plethora of the common issues facing batch processing systems and the solutions provided to these issues by Spring Batch. Batch processing seems to be the red headed step child of software development. Continually neglected while SOA and AJAX based applications win all of the adoration of the parents. Regardless, it is still a significant portion of the software projects out there and there are a wealth of common patterns that can be pulled up a level into a framework like Spring Batch. I'd go into more detail on it but I am seriously exhausted and my head hurts from the amount of information presented at this conference.

In a nutshell, the content at this conference was great and the majority of the speakers were very good. Only downside was that I didn't win an iPhone.

The Spring Experience - Day 3

2007-12-14T10:08:00.000-05:00

Started off day three with Spring Security directly from it's creator, Ben Alex. The session provided a good intro to Spring Security but I felt bad for Ben as he got peppered with 4011 questions during the talk. Even with all of the tangents, Ben did a good job of presenting the core benefits of Spring Security. An example of it's power is to have a configuration that intercepts a method, say getEbayItemListing() and checks to see before execution if the user is authorized to perform this action. This can be a complex check that even validates if the current requester is actually seller of the item (instance level authorization). Then upon method completion, the after advice will mask the minimum sale amount by nulling it out on the object if the requester is not the original seller. With this being performed at the method level, it doesn't matter if the request comes from a website, web service, or other Java program. Very powerful.

Rob Harrop gave a talk on concurrency in Java that was very in depth coverage of the Java 5 concurrency objects. There was a ton of info in that session that I'll not cover here other than to say if you are developing in a multi-threaded environment in Java 5, please review the docs in the java.util.concurrency package.

The third session I attended was from one of my favorite tech speakers Venkat Subramaniam. Venkat covered Spring's facilities for integrating dynamic language objects like Ruby and Groovy classes into your Java framework as plain beans. I had no idea that this facility existed and as with most things Spring, it is accomplished very painlessly with a little configuration. A great use case for this is on the fly loading of dynamic processing logic (like rules) without stopping your application or server. Want some small dynamic rules processing without dealing with a full blown rules engine like JRules or JBoss rules? Here's your answer (and you don't need to wait for your company to deploy Java6). The second half of his talk covered using the GroovyObjectCustomizer object from Spring to implement a DSL. This also seemed relatively simple, but it was surprising how much more complex it was to implement in straight Java versus Groovy. I still prefer Ruby though.

The final talk before 500 pasty, flabby geeks descended upon the Hollywood, FL beach was Spring Web Services by Arjen Poutsma. I didn't gather a ton of info out of this session but a couple of nuggets were useful. The speaker advocated the opposite approach of XFire. Basically that you should start with the contract and work back towards mapping to your domain which I think makes sense in all but some of the more trivial cases. Additionally it seems that once you get the gist of Spring and the annotations provided in 2.5, moving from Spring MVC and WebFlow to something like Spring WebServices is rather trivial.

Time to hammock up!

The Spring Experience - Day 2

2007-12-14T08:41:00.001-05:00

I focused on web framework related sessions today and came away with a wealth of knowledge and a long list of technologies to investigate further. Keith Donald gave a presentation on full stack web frameworks that was very interesting. He gave a very unbiased perspective on how frameworks like Grails, Rails, Django and others excel and lag with respect to about 10 criteria that comprise a "full stack" web framework (AJAX, REST, data binding, security, testability, etc...). He offered that Spring really doesn't have a full stack web framework, but instead acts as a foundation for them (Grails being built on top of it for instance). That is changing however as Spring MVC and WebFlow are adapting and integrating with some other technologies to really provide a complete offering.

I attended a few other sessions that focused on REST with Spring, AJAX offerings, JSF and WebFlow based apps, and Reasonable Server Faces. They all provided ton of information outside of my traditional area of expertise (middleware and persistence).

I think the most striking thing that I realized in these sessions though was how much of a huge step up in productivity there is in Spring 2.5. Annotations seem like a much better fit for metadata and being able to tell the container to only autowire a few certain dependencies right at their declaration point is immensely useful. Spring Security (formerly ACEGI) has gone from around a minimum of 150 lines of configuration to about 15 for the simplest cases by making use of these constructs. Spring 2.5 maintains backwards compatibility as well, so run, don't walk to the download if you haven't already.

The Spring Experience - Day 1

2007-12-13T10:25:00.000-05:00

Day one of TSE was just registration, dinner, and a keynote by Rod Johnson but it was a great evening for a couple reasons.

Rod's keynote focused on the changing of the guard and disruption for the Java EE landscape. Most notably, his contention was that the committee driven process that we have now is largely failing. EJB 3.0 being just the latest example. OpenSource is a much more dynamic and user driven process that nearly immediately produces software and frameworks that a lot of users need. The committees by contrast are steered by people who try to sell application servers and move at the pace of a Columbus driver in a 1/2 inch of snow (aka very s l o w l y). I think his point here was right on and I think it might signal a very polar divide between users and committee.

Additionally, the conference passed out Rod Johnson bobblehead dolls complete with the pose from this book cover:

Classic!

Christmas Comes Early for Spring Fanboy

2007-12-10T13:00:00.000-05:00

Wednesday I leave balmy Ohio (currently 38 degrees with showers) for Hollywood, FL (currently 82 degrees). Weather aside, I am extremely excited as I'm going there for the Spring Experience conference. As big a fanboy as I am with Spring, I'm somewhat daunted by the sheer volume of information that will be coming at me this week. There are 70 1.5 hour sessions crammed into 3 full days and I've got more than a passing interest in attending about 50 of them.

TSE is organized by Jay Zimmerman of NoFluffJustStuff fame, so I've got high expectations. I've been to NFJS conferences the last four years and every time I come back to work re-energized and armed with a bevy of new tools and techniques that are immediately applicable to my daily work. NFJS typically brings only noted authors and thought leaders in as speakers for these conferences and this one is no different with Rod Johnson, Juergen Hoeller, Rob Harrop, Ben Alex, Ramnivas Laddad, and a slew of other Spring contributors and experts speaking at the conference.

I'm going to have the laptop with me and battery willing, I'll be able to share my impressions throughout the conference.

Time to get packing. Now where did I put the old banana hammock...

The Long and Winding Road

2007-12-07T11:16:00.001-05:00

I've started back up on a new phase of a project and it's rekindling my hatred for extensive documentation. I'm beginning to feel that most developers are just not inherently good at writing documentation, myself especially. We're just not wired that way and I think people generally despise doing tasks that they are not good at.

In this instance maybe it's just more painful because there is no challenge involved. It's a straight migration of an existing system so there is none of the excitement involved in finding out what the business domain entails and trying to mate technologies to the problems facing the users of the system.

Luckily we're always running an Agile project at QSI, and this particular client is open to documenting only what is needed and not wasting money on anything more. Unfortunately this brings up the question, what are the bare essentials when it comes to documenting a system? In this instance specifically, what is the minimum amount of documentation needed to capture the state of an existing code base? How valuable is any documentation? It's always the first thing thrown out the door when timelines get compressed and it's typically the last thing updated after maintenance or enhancements are performed on a system.

The first of many Spring kicks arse posts

2007-11-21T09:51:00.000-05:00

In my opinion, nothing has brought life into Java development like the Spring Framework. Enterprise development in Java 4 years ago had me scratching my head. Why do I need to implement methods in this interface when I don't declare I implement it? Why do I need Entity Beans? Which of the 8 web frameworks should I be using? Why does it take so long to go from fixing a bug to actually testing that it worked? Spring has in some way, shape, or form answered all of these questions. Well except for the 8 web frameworks one.

Spring started with a couple of developers who were as frustrated as I was with the complexity inherent in Java enterprise development. Fortunately for our community, they were and are much smarter and driven than I am and they wound up with a set of tools that simplify almost every aspect of development in a Java enterprise. I think one of the keys of Spring's success has been focusing on simplifying advanced concepts and providing extremely useful tools out of the box. This example shows how Spring has simplified Aspect Oriented Programming which was a rather radical shift for Java developers in the early days of AspectJ. Additionally it provides something that 90% of all Java projects would want out of the box. Simple entered/exited logging and performance monitoring.

This code in it's current state will provide good trace logging for any bean named *DAO or *Service. Change or add the interceptor name to PerformanceTraceInterceptor and it will provide performance metrics using the JAMon performance monitoring utility. All of this provided out of the box with Spring. Stay tuned for more Spring fanboy-ism in the future.

Hibernate Hijinks

2007-11-20T11:09:00.000-05:00

I've been tasked with optimizing some slowly performing transactions in a system that has been built upon Hibernate. Yesterday I ran across an interesting situation where a single transaction was resulting in 6 separate queries to the database. This may not necessarily be bad, but when the transaction was performing as slowly as it was, we needed to take whatever means we could to speed it up. What was unique to me was that we eliminated all but 2 queries, and we eliminated each one in very different ways.

Use the second level cache for largely read only data. There was a table that had slipped through the cracks of our investigation that will rarely be updated. A couple of configuration entries later and it will be stored in a second level cache and refreshed on a weekly basis instead of causing an extra query hundreds or thousands of times per day.
Eagerly fetch associations that are needed every time the transaction is called. The Criteria API in Hibernate allows the user to specify the fetch mode for every association, effectively overriding the default behavior provided by the mapping file. Setting the fetch mode to Eager for this association resulted in an extra left outer join, but it eliminated an extra query per result row.
Don't haphazardly access collections that are not needed by all of your clients. The transaction in it's previous state would call off to a dependent collection and call the size() method on it to determine how many child records were linked to the parent. This caused every child record in that collection to be loaded by Hibernate, even though very few of the clients actually needed to know the size of this collection. Adding a property to the incoming request to determine if the transaction actually needed to fetch that count and then doing it by simply executing a separate projection based query resulted in much speedier performance for all clients.

This scenario illustrates the flexibility of Hibernate and also the ease with which you can misuse this powerful framework. Always output the SQL generated by Hibernate in development, and make it a regular part of your design, develop, test cycle to inspect that SQL to ensure that it is doing what you think it is.

Hello World

2007-11-20T10:53:00.000-05:00

I've finally been coerced into starting a blog. Thanks BHP!

Stay tuned for my thoughts on Beer, Software Development, Family, and Friends. Mostly beer though.