jCrete 2016

Many harvested content, videos and other materials are available at: https://github.com/JCrete/jcrete2016

My current go-to material on micro-Services

#kudos Graham Lea

CQRS Recipes (by @tjaskula)

Been trying to grasp my head around the DDD+CQRS+ES+CS+... concepts and finally found this step by step presentation:

CQRS recipes or how to cook your architecture from Tomasz Jaskuła

Enjoy! PS: kudos tjaskula

JCrete 2014

This post is, sort of, a reply to Geertjan’s post about JCrete (https://blogs.oracle.com/geertjan/entry/jcrete_2014).

Geertjan describes a group of trees and paths and a river and (purposely or not) lacks to mention he’s describing a forest. It’s great to have people with Geertjan’s reach to present and support this way of organising conferences, but I’d like to give my two cents on the subject.

Before continuing, let me say what Geertjan didn’t: "JCrete is a conference that uses Open Space Technology (http://en.wikipedia.org/wiki/Open_Space_Technology)”. A description like that is usually simplified as “FOO is an Open Space”. Hmm! We got rid of ’technology' and ‘conference’. 

I’m still surprised to see the amount of people still unfamiliar with OpenSpace Technolgy, but that’s probably because we’ve been using a lot in Barcelona (more on that later), therefore I'm the weird one. As Dmitry said during the wrap-up, this format should be used all around and I completely agree to that. 

But! I think wide adoption of Open Space Technology is still not possible, or that it should happen slowly.

Open Spaces don’t scale well. Even in JCrete things changed comparing 2012 edition and 2014. There were twice as many attendees and I’d say the number of people familiar with Open Space Technology is and how to *use it* remained the same (thus decreasing the %).

While preparing the main room on Sunday evening, some of the un-organizers and I went for a round gathering of chairs. That mutated the day after into a semi-circle around the screen the following days. Ok, we need to see code, it’s true. But then I noticed all rooms ended up in a similar layout. We are still too wired for the sit-and-listen conference format and unlearning this takes time.

In Agile Barcelona there’s been an open space every 6 months for the last few years. That led to some other communities (AngularJS, Java Hispano, Bcn Dev Con,…) to adopt it. The results are usually the same: people unfamiliar with the format, even when it’s explained (including 4 principles and law) still don’t grok it. 

Open Space Technology is easy to explain but hard to use.

I think one of the reasons why people get so surprised in JCrete is because it’s an Open Space, not a Conference. What I’m trying to say here is that when a conference was publicised as a using Open Space Technology, attendees should only hear the ‘Open Space’ part. Actually, it might be a good thing to just use the two terms separately: Conference vs Open Space. These are not exclusive terms though. It’s possible to have the two mixed up: plain-old-cfp-talks-in-slots during the morning and open spaces during the afternoon (see for instance ALE 2014 program http://ale2014.alenetwork.eu/ale14-program/ ).

But I agree with Geertjan on that (1) morning open space followed by (2) at-the-beach, spontaneous sessions in the afternoon followed by (3) wonderful share-all food is plain genius. 

PS: I was really surprised to see talks popup out of nowhere and happen on the so-called 'room 5' (see first picture of  Geertjan’s post) as that is actually the ultimate ‘Open Space’.

Baggage Collector Kata

I recently worked with a team trying to get them to grok some habits around testing. One of the most challenging tasks was to make them learn to unlearn.

They know a lot about their business and so they know all exceptions to all the rules. This leads to overcomplexifying any solution. That leads to paralysis by analysis in some cases too. Since the problem we were using to practice testing was getting in the way again and again (due to corner cases) I decided we would ignore the problem and do a kata instead.

Since I also wanted them to do a kata similar enough to what they were trying to solve I just made one up on the spot and made it so that some problems in the kata were close enough to the actual problem. 

DISCLAIMER: I know this intro is extremely vague. Just cut to the chase.

Baggage Collector Kata

Imagine you work at a baggage handling service of a regional airport in 1987. You spend your day taking bags that fall through a slide into a pile of baggage, from the pile to (a) the Hangars, (b) the baggage claim Hall or (c) Lost and Found.

Write the code so that a baggage is sent to the appropriate location:

 - a baggage has a label with a FROM and TO

 - a baggage whose FROM matches the current airport goes to the HANGARS

 - a baggage whose TO matches the current airport goes to the HALL

(Hidden rule)

 - all others go to LOST and FOUND

NOTE: The hidden rule was left out on purpose. I expected the team to notice there's cases not covered by the original set of rules and to (1) go ask the business expert (me) about what to do and (2) get a test for those cases.

Moving on to the future

While they were coding, I decided time was a variable to consider so I made it so each year would actually happen in 10 minutes. Meaning I added extra rules every 10 minutes.

Time goes by and the airport is being successful. Management is preparing some changes in order to make the airport an international hub.

Your team is merged with the Hangar's team. You now must:

 - send baggages to appropriate HANGAR depending on the TO indicated on the label. That is, you no longer send baggages to HANGAR but you send them to HANGAR-8 or HANGAR-4, etc...

Also, you may start receiving bags that are on transit!

 - a bag's label may now have several hops on the itinerary. Send bags to HALL if bag is on the last hop or send them to the appropriate hangar if current airport is on the itinerary. 

 -if the current airport is not on the itinerary, just move the bag to LOST and FOUND.

The Golden Years

The plan seems to be working and your airport is now an international airport! Kudos!

Here's some more considerations:

 - if a bag FROM your airport will go abroad, you must send it to X-Ray and then to the appropriate Hangar

 - a bag arriving (our airport is final destination) from a country abroad (original airport) then the bag must go to CUSTOMS and then to the HALL.

NOTE: At this point you can let your imagination flow and add all kinds of rules. For example:

 - merge your team with lost-and-found teams so that if you receive a baggage you can re-route (even if you are not on the itinerary), just do it.

 - make the hangar-to-flight mapping change during time. If a flight has a very early checkin, make it's baggage to the hangar even thought the plane is still not there or even if there's another plane there.

 - remove all bags belonging to JACINTO RAMIREZ (a known drug dealer on Interpol's Most Wanted List)

Head fake

I used the kata as a way to get the team to get used to a set of simple rules changing very quickly. You can only go refactor crazy if your basic functionality is granted.

The original problem was simple and easy enough to reason about so as to dig deep into the possible corner cases it presented. We got into the exception handling quick enough. The message I was trying to pass on was: a simple problem is easy to reason but will surely have unspecified corner cases get your business expert in hand, find those cases quickly. Before you start coding your cathedral, step back and think of a good set of possible inputs and the expected outputs. If you find cases you can't answer to, get back to the business expert.

Also, your business will change soon enough and probably on a direction different than what you expected. Don't try to anticipate the future and keep you design simple and uncoupled so that it accepts change easily.

My first take on reducing Garbage

I have been trying to grok several concepts regarding JIT, memory footprint, language abuse, mechanical sympathy, non-GC algorithms, etc... A problem I found is that there's so much to learn I was finding it hard to focus on a single idea and any attempt to do it faced a great difficulty to obtain clean info.


Few months ago I met Peter Lawrey of the http://vanillajava.blogspot.com  and  https://github.com/OpenHFT. At the moment I was not aware of who he was and so I missed the chance to ask him for some directions. Luckily enough, when I reach out to him earlier this year and presented my dilemma he was very kind and agreed to help. His first suggestion was to select one subject and then he offered to help me take small steps. The choice was no-GC algorythms and this is my first post on the subject.

Simple Garbage generation

First challenge is was presented was simple enough: "How can you iterate an ArrayList (without generating garbage)?". My first reaction was shock. It never occurred to me that a simple ArrayList iteration was creating garbage (which it sometimes is... not always...). 

Demonstrating there’s Garbage

From the challenge I knew there's garbage on:

    public static void main(String…args) {

        List l = new ArrayList();

        for (String s : l) {}

    }

This can be verified with several methods but first thing that came to mind was running this main with params:

  -verbose:gc -Xms4m -Xmx4m

which display the output:

...
[GC (Allocation Failure) 2436K->388K(5632K), 0.0003570 secs]
[GC (Allocation Failure) 2436K->388K(5632K), 0.0004620 secs]
[GC (Allocation Failure) 2436K->388K(5632K), 0.0004700 secs]
[GC (Allocation Failure) 2436K->388K(5632K), 0.0004410 secs]
[GC (Allocation Failure) 2436K->388K(5632K), 0.0006780 secs]
[GC (Allocation Failure) 2436K->388K(5632K), 0.0004480 secs]
[GC (Allocation Failure) 2436K->388K(5632K), 0.0004540 secs]
[GC (Allocation Failure) 2436K->388K(5632K), 0.0004570 secs]
…

which proves there's GC activity.

After a code review of the hotspot I found out the problem is caused in the foreach(). That is invoking ArrayList$iterator to iterate over the the list. In the (Hotspot) JVM that ArrayList$iterator returns a new instance of an inner class named ArrayList$Itr.

After checking out with Peter he confirmed I was on the right track but that I had to prove with a direct measure that my suspicion was correct.

Observing the memory consumption

At Peter’s direction I run the code and then checked the output of jmap (http://docs.oracle.com/javase/6/docs/technotes/tools/share/jmap.html). After playing around with parameters I fired up a ‘watch’ on my command:

   $watch “jmap -histo `jps|grep Main|awk -F\ '{print $1}’`"

With that I’d be constantly running jps and then filtering and asking to obtain y App’s PID. With my app’s PID I’d only need to jmap -histo.

 num     #instances         #bytes  class name
----------------------------------------------

   1:         65486        2095552  java.util.ArrayList$Itr
   2:          6619         853136  < methodKlass >
   3:          6619         764496  < constMethodKlass >
   4:           456         538304  < constantPoolKlass >
   5:           421         342368  < constantPoolCacheKlass >
   6:           456         308984  < instanceKlassKlass >
   7:          1373         135048  [C
   8:           111         117992  [I
   9:           702         116976  [B
...


What you don’t see in this output is that every 2 seconds (watch) the number of instances of java.util.ArrayList$Itr would change and increase until it’d disappear (GC wiping out old instances).

Go! Fire up your IDE of choice and try what we’ve done so far. I’ll wait.

Finding an alternative without Garbage

Now that I finally knew how to get some more insight re my memory usage and GC activity it was time to start searching for an alternative which didn’t generate Garbage. that was pretty easy:
        List l = new ArrayList<>();
        int lsize = l.size();
        for (long i = 0; i < GAZILLION; i++) {
            for (int j = 0; j < lsize; j++) {
            }
        }

Believe it or not. This plain old for-loop is better in terms of garbage generation.

Actually, if you don't believe you can go fire up your IDE and check. ;-)

Scala code in Android app

I've been meeting Dani at several meetups in Barcelona lately.

We're both interested on scala but he also investigates the options to use it in android apps.

Last Saturday I remebered reading something about scala being used in android apps and tried to restore my notes to share them with Dani. Here's what I could recover:

ALERT: The following is a druid recipee based on maven.

git clone git@github.com:jayway/maven-android-plugin.git 

git clone \git@github.com:jayway/maven-android-plugin-samples.git 

cd maven-android-plugin 

mvn clean install 

cd ../maven-android-plugin-samples/ 

cd scala/ 

sed -i '' 's/2.8.0/2.10.3/' pom.xml

mvn clean install 

adb install -r  target/scala-1.0.0-SNAPSHOT.apk

Note: the previous uses a Mac OSX flavour 'sed' that requires an extra pair of simple quotes.