Continuous Delivery Experience

ISC StormCast for Tuesday, June 18th 2013 http://isc.sans.edu/podcastdetail.html?id=3374, (Tue, Jun 18th)

Johannes Ullrich — Tue, 18 Jun 2013 16:00:21 CDT

...(more)...

EMET 4.0 is now available for download, (Tue, Jun 18th)

Johannes Ullrich — Tue, 18 Jun 2013 13:00:19 CDT

Somewhere I know TJ O'Connor is a very happy analyst. EMET 4 ...(more)...

Chess Pieces or Domain Expertise? Your Choice

Johanna Rothman — Tue, 18 Jun 2013 11:00:32 CDT

Many years ago, I started a job as a contract manager, and it became clear I had a big problem. I had developers who knew one area of the code well. I had testers who knew not much of any area of the code well, even though they had worked for the organization for many years. Why? They had been shuffled from one project to another almost every month for years.

The writers had more domain expertise than anyone, because they had learned the product from end to end. What was I going to do? This project needed to finish in eight weeks, I needed to hire my replacement, and the people were shell-shocked from one manager after another. I was the fourth Director of Software Development in six months.

I decided to make a project portfolio, and rank the projects to know which projects were first, second, and third. I knew about this first project, but what was second and third? I was sure we had more crises, waiting.

Next, I asked people what they wanted to work on. I was sure that people could select their areas of responsibility, and it would work out. This is before we had agile teams. I was using a staged delivery lifecycle, so we had cross-functional teams and we worked by feature.

Next, I asked the teams to develop their deliverables in two-week chunks: what deliverables, as in features, could they deliver in the next two weeks, working as a cross-functional team?

The teams started to work together. They started to work across the code, not just in one area of the code. It wasn’t perfect, but it was working. The testers expanded their knowledge, because they were able to focus on the entire product. The developers learned the product end-to-end. The writers were happier, because people answered their questions.

About three weeks into this, a crisis happened with the second-ranked project. A senior manager wanted to yank a bunch of people off the top-ranked project and put them onto the second ranked project. He told me I had to give him people.

“No. I have no one for you.”

“But you work for me.”

“Yes, I work for the organization. But this project, the one that everyone is on, is more important than your project. So, I have no one for you, and I won’t for another two weeks. Unless I hear from the CEO that your project is more important than this one. In that case, we will stop working on this one, and everyone will work on your project.

We only have eight people. We can’t multitask. We can only work on one project at a time. Moving people off and onto projects as if they are chess pieces doesn’t make sense. This top-ranked project has two more weeks to go, and then it’s done. When it’s done, I can assign all eight people to your project, no problem. But assigning them now? Craziness.”

Now, should I have called him crazy? That was darn close to a career limiting conversation. Don’t do that! But the rest of the conversation? Useful.

And that’s the topic of this month’s Management Myth #18: I Can Move People Like Chess Pieces. When you move people like chess pieces, you deny them the opportunity to learn domain expertise. You don’t manage the project portfolio, and you decrease the capacity of your organization. It’s all around bad.

Did I want to be nice to the senior manager? Of course I did. But, good management is not about being nice to everyone all the time. Much of management is about saying no when you have to. And, I really needed to say no.

With agile approaches, I would have had more options: I could have used iterations and sequenced the projects differently—maybe. But that would not have allowed the organization to release the top-ranked project in the next few weeks. It would have made the people feel horrible, as if I’d yanked the promised dessert away from them. Yes, the project was that close to completion.

While you can move people as if they are chess pieces, it’s rarely a good idea. You want to leave people with project teams, to let them create a solid team—that’s the forming, storming, norming part. And, you want to keep people on a product long enough that they develop significant domain expertise in the product. When they become bored and ask to move, then they will tell you in a one-on-one that it’s time to move.

In the meantime, don’t move people as if they are chess pieces. People need to develop mastery over their work. They need the autonomy to select the areas of their work that they will develop domain expertise in. They need to feel as if they have a sense of purpose about the work. Gee, it sounds as if I’m parroting Dan Pink, in Drive. I guess I am. When you commit to project teams, and leave people where they are, so they can learn the product and learn how to work with their teams, and let the work flow through the teams, you create an environment that allows for autonomy, mastery, and purpose. And, that’s part of what a great manager does.

Groovy on instantserver.io thanks to GVM

Guillaume LaForge — Tue, 18 Jun 2013 08:00:39 CDT

Deploying Applications to GlassFish Using curl

Jason Lee — Mon, 17 Jun 2013 16:00:15 CDT

Over the past few months, I’ve been posting tips on how to use the REST interface in GlassFish v3 and later to perform various functions. My last post used Scala. In this much briefer and far less ambitious post, I thought I’d share how to deploy an app using curl (from the shell of your choice). If you’re familiar with the REST endpoint, there’s really not just a whole lot new here: pass::[more]

curl -s -S \
          -H 'Accept: application/json' -X POST \
          -H 'X-Requested-By: dummy' \
          -F id=@/path/to/application.war \
          -F force=true http://localhost:4848/management/domain/applications/application

Remember that the actual archive contents are passed as the value to the parameter id, so we tell curl to send the contents of the file using the prefix @. The context root and application will be deduced by the system (or can be specified by passing contextroot and name parameters). The force parameter tells GlassFish to force the deployment even if an application is already deployed under that name (which is effectively a redeployment). As an added bonus, to undeploy, you can issue this:

curl -s -S \
          -H 'Accept: application/json' \
          -H 'X-Requested-By: dummy' \
          -X DELETE http://localhost:4848/management/domain/applications/application/$APPNAME

It’s as simple as that. If you’re using a shell script, you could always just use asadmin directly, of course. Using that approach, asadmin must be on the PATH, or you have to specify the full path in your script, so your choice comes down to preference, I think. Either way, now you know how to do both. :)

This week in #Scala (17/06/2013)

Jan Machacek — Mon, 17 Jun 2013 13:00:31 CDT

Welcome to another edition of #ThisWeekInScala. Another Scala Days conference has been and gone. Unfortunately there’s no videos yet, but we’re told they’re on the way. Hopefully I’ll have them for you next week. In the meantime, here’s plenty to … Read more →

Type-Safe Web with Kotlin

Andrey Breslav — Mon, 17 Jun 2013 11:00:35 CDT

We told you about Kara Web Framework a while ago. It is written in Kotlin and relies on type-safe builders. It doesn’t have to be the only web framework for Kotlin, but the general principles seem good, so I wrote an article about these principles: Type-Safe Web with Kotlin, it gives an overview of the framework design, and then gives many details about builders, extensions and so on.

Examples used in this article are available at https://github.com/abreslav/kara-files/blob/master/src/tutorial.kt and https://github.com/abreslav/kara-files/blob/master/src/small_builder.kt

Slides from my talk given at JPoint in Saint Petersburg are also available online.
The talk was given in Russian, video accessible is here.

Oracle Java pre-announcement: Upcoming JRE patch will plug 37 remotely exploitable holes. See http://www.oracle.com/technetwork/topics/security/javacpujun2013-1899847.html, (Mon, Jun 17th)

Johannes Ullrich — Mon, 17 Jun 2013 08:00:38 CDT

...(more)...

A scan is a scan is a scan, (Sun, Jun 16th)

Johannes Ullrich — Sun, 16 Jun 2013 11:00:37 CDT

Install Percona on Debian/Ubuntu

First step, get the software:

gpg --keyserver  hkp://keys.gnupg.net --recv-keys 1C4CBDCDCD2EFD2A
gpg -a --export CD2EFD2A | sudo apt-key add -

Next, add the following entries to /etc/apt/sources.list, replacing “VERSION” with the name of your distribution:

deb http://repo.percona.com/apt VERSION main
deb-src http://repo.percona.com/apt VERSION main

Once this is done, you are ready to install, but since it’s probably a good idea to back up the database first, you should probably run a few more commands, for safety’s sake:

cp -Rf /var/lib/mysql /var/lib/mysql-old
cp /etc/my.cnf /etc/my.cnf-old

Now, we’re ready to run install the Percona packages, which will take care of automatically stopping, removing, and replacing any existing MySQL packages:

apt-get update
apt-get install percona-server-server-5.5 percona-server-client-5.5

Once this completes, you’ll want to verify that everything works as expected, and that your existing sites are running normally. If all goes well, you should notice a marked performance improvement in your database layer.

Install Percona on Red Hat, Fedora, and CentOS

First step, set up the software sources and verify that the repository configuration was successful:

rpm -Uhv http://www.percona.com/downloads/percona-release/percona-release-0.0-1.x86_64.rpm
yum list | grep percona


// You should see the output similar to the following:
percona-release.x86_64                     0.0-1                       installed
...
Percona-Server-client-51.x86_64            5.1.47-rel11.1.51.rhel5     percona
Percona-Server-devel-51.x86_64             5.1.47-rel11.1.51.rhel5     percona
Percona-Server-server-51.x86_64            5.1.47-rel11.1.51.rhel5     percona
Percona-Server-shared-51.x86_64            5.1.47-rel11.1.51.rhel5     percona
Percona-Server-test-51.x86_64              5.1.47-rel11.1.51.rhel5     percona
...
xtrabackup.x86_64                          1.2-22.rhel5                percona

Once this is complete, you can now proceed to run the install, which will take care of automatically stopping, removing, and replacing any existing MySQL packages:

yum install Percona-Server-client-55 Percona-Server-server-55

Additional Resources

For additional resources, check out the Percona documentation:

About the author:

Lincoln Baxter, III is a Senior Software Engineer at Red Hat, working on JBoss open-source projects; most notably as project lead for JBoss Forge. This blog represents his personal thoughts and perspectives, not necessarily those of his employer.

He is a founder of OCPsoft, the author of PrettyFaces and Rewrite, the leading URL-rewriting extensions for Servlet, Java EE, and Java web frameworks; he is also the author of PrettyTime, social-style date and timestamp formatting for Java. When he is not swimming, running, or playing Ultimate Frisbee, Lincoln is focused on promoting open-source software and making web-applications more accessible for small businesses, individuals.

Grails for the Wicked Smaht

Brian Kotek — Fri, 14 Jun 2013 16:00:30 CDT

Grails for the Wicked Smaht

Something I like about Groovy/Grails is that I seem to get 90% of my work done with 10% of the language/framework. Here’s a guide to the 10%, intended for folks who already know what they’re doing with Web technologies.

Pretty much every entry in here has multiple pages (or even chapters) of documentation on the Groovy or Grails homepage, and is probably covered in-depth in a book. This is not that documentation or book.

Some things may seem restrictive, like “a controller method assumes a like-named view.” That’s just a convention, and I haven’t hit a Grails convention yet that won’t allow you to override it, whether it’s view names or complicated model-to-database table mappings.

Groovy

Language

It compiles to .class, so you can use it from Java or use anything Java from Groovy.

Everything’s an Object

Declare an int, you get an Integer. This means boxing. Drop to Java for fast math.

Strings

Strings in single quotes are literals. String in double quotes are “GStrings” (chuckle), and lazily evaluate anything inside of ${}.

(Hash)Maps and (Array)Lists are simple, replacing Arrays and such.

Declare them a lot like JSON, but with square brackets.

Closures may be new to you.

They’re anonymous function that can be assigned to variables, passed, etc. They have access to the scope around them.

def outerVariable = "foo" def closure = { closureArgument -> println "$closureArgument $outerVariable" } (1..10).each{ i -> closure.call( i ) //prints "1 foo", "2 foo", etc. }

Loop with Closures

“For” and such exist, but closures own loops. Each, EachWithIndex, Collect…there’s a lot of things you can do with them.

Truthiness

Null is false. Empty collections are false. Empty strings are false. Empty iterators/vectors or those at their end are false. Common sense, really.

Classes and Beans

Getters/Setters are Implied

Here’s a Person bean. Java code would see getFirstname() and such.

Constructors take maps

Could not embed GitHub Gist 5751490: API Rate Limit Exceeded for 178.63.146.0

Access via properties or dynamically.

All valid:

You can write your own getter/setter to override.

class Person { String firstname String lastname Integer age void setFirstname( String firstname ) { println "Setting firstname to ${firstname}" this.firstname = firstname } }

Grails

Grails is MVC with Spring and Hibernate (or MongoDB or whatnot) already hooked up and simplified. You can pretty much do anything possible in raw Java+Spring+Hibernate, but Grails simplifies the way 95% of folks did 95% of their tasks.

Installing

Download Grails and set a GRAILS_HOME environment variable to the unzipped directory. Add GRAILS_HOME/bin to your path. Done.

Creating an application

Navigate to the desired parent directory and do “grails create-application MyApplication”.

Application directory structure

It’s pretty straightforward. Directories are named what they contain. Grails-app contains your application, and grails-app/controller contains controllers. This pattern persists throughout.

Running your application

Go to your application’s directory and type “grails”. This’ll start the “interactive shell.” Now, type run-app and it’ll run and tell you the URL, like http://localhost:8080/MyApplication.

Model

Saveable Domain Models

The “Person” we already wrote is all we need. Grails will infer an autoincrementing “id” property and optimistic locking in a “version” property, adding them on the fly.

Data access, finders, and such

Anything that’s a Grails domain model can save, read, list, and generally CRUD. You can use dynamically-created methods, HQL, or criteria.

def company = new Company( "Acme" ) def person = new Person( firstname : "Bob", lastname : "Bobbyson", age : 80 ) company.addToPeople( person ) company.save() person.findByFirstName( "Bob" ) // finds all "Bob"s person.findAll( "from Person where Company = :company", [ company: company ] ) company.delete()

Relating Entities

Many-To-One

Add a property and say the child “belongsTo” the parent to enable cascading.

Could not embed GitHub Gist 3911192: API Rate Limit Exceeded for 178.63.146.0

One-To-Many

Add a static “hasMany” and declare a property if you want a collection type other than Set:

Could not embed GitHub Gist 3911192: API Rate Limit Exceeded for 178.63.146.0

Add to “one to manys” on the “parent” side with generated methods

def company = new Company() def person = new Person() company.addToPeople( person ) company.removeFromPeople( person ) company.save() // cascades

Validation

Add constraints to a model and it’ll runtime validate (using Spring validation) and manage database constraints for you. You can write custom validations. “nullable: false” is implied for EVERYTHING, and must be set to true to override.

Could not embed GitHub Gist 3911330: API Rate Limit Exceeded for 178.63.146.0

Controller

A Grails controller defines methods that equate to URLs.

Controller Methods

This’ll define /person/show, allowing /person/show?personId=2. URL/form values go into a ‘params’ map. A map returned by the method contains values available in the view.

Could not embed GitHub Gist 3911330: API Rate Limit Exceeded for 178.63.146.0

Views

A method assumes a like-named view. If one doesn’t exist, and you don’t explicitly do a render() or redirect() in your controller’s method, you get a 404. Therefore, here’s /view/person/show.gsp:

Grails tags

Grails defines a bunch of tags, like,and so on. I’d read up on what’s available and make use them.

JSON/Ajax/XML

Use “as JSON” or “as XML” to render something in place of a .gsp view:

This works on lists, maps, whatever.

Services

Your controllers shouldn’t really do anything except marshal form/URL input onto models and control application flow (where a user goes and when). Heavy lifting should be deferred to services. Every method in a service is wrapped in a transaction. This can be turned off.

Going Further

I’ve described the 2% of Groovy and Grails I use constantly, and ignored 98% of what they can do. I’ve done full-bore integration of x509 authenticated JEE applications using JPA annotations into Grails, Spring-Security work, Web Services, all that jazz. All of JEE is there, and it can all be configured however you need. HTML5 and JavaScript frameworks are easy to integrate. A huge library of community-driven plugins exists to take care of common stuff, like e-mail integration or security.

Groovy and Grails just reduce the amount of BS involved, letting you focus on what’s actually important to your application and its users rather than 5,000 XML files that contain tweaked boilerplate pasted from documentation.

When Hotel Alarms Sound, (Fri, Jun 14th)

Johannes Ullrich — Fri, 14 Jun 2013 13:02:05 CDT

I often wondered what an 'average' reaction would be to a fire alarm sounding in a hot ...(more)...

It’s Déjà vu All Over Again

Alan Shalloway — Fri, 14 Jun 2013 11:05:39 CDT

Several years ago I tried to discuss the need for Lean when Scrum was being used on projects with more than one team. Ken Schwaber didn’t want to hear this and eventually threw me off the Scrum Development Yahoo discussions group. I admit, I was talking outside the domain of what he wanted – the pretense was that Scrum wasn’t Lean and the Scrum Development group wasn’t the place to talk about Lean.

Times have changes – a little. We now hear how Scrum is based on Lean but I have heard little acknowledgement that Lean-thinking, used explicitly, is an essential ingredient for Scrum teams. At the Scrum gathering I heard of a successful Scrum implementation of a company that had 150-200 folks in it. I can validate that myself. Because I had been in there myself, before they were successful with Scrum across the enterprise. True, it didn’t take much to get them from successful team Scrum to successful enterprise Scrum, but it did take Lean-thinking, value stream mapping and an understanding of Lean-flow to get them there. While Scrum is a great team process I still contend it is rarely successful at scale because the framework itself does not provide the insights needed to extend it. If that’s the case, part of the Scrum discussion should be about what is needed to include in the framework.

For the last few years I have been promoting Kanban, both team-Kanban and the Kanban Method, a transition management method across the entire value stream. Team-Kanban is best used within a full-value stream view as well, but it doesn’t require the Kanban-Method’s approach. It is unfortunate that the term Kanban has been so overloaded (see De-Mystifying Kanban for a description of all the types of Kanban).

Net Objectives is not attached to any one method. I believe we are the only company that promotes Scrum, XP, Team-Kanban, Kanban Method, SAFe, and Lean, along with technical practices. Our approach is to see where people are, what their culture is, how much discipline they need, the extent they can form teams, amongst other things. If you missed our webinar, Beginning an Agile Transformation, you might want to watch it for a full list of what to look for.

Our experience enables us to cull the best approach from all of these methods to suit our clients’ needs. Sorry, I’m not trying to make this a sales pitch, anyone can take this approach. Unfortunately, relatively few consultants do. I’ll admit I'm writing this blog for practitioners and my advice is don’t decide between methods, learn from each of them. In other words, it’s not Scrum Vs Kanban, it’s how do we take the best approach for the needs at hand. This requires understanding the strengths and limitations, particularly where they best apply, of both.

Since the Lean-Kanban community has been forefront in bringing new methods to the Agile space, I had thought it would behave a bit different from my experience in the Scrum community. Unfortunately, apparently not. Because Net Objectives uses both Scrum and the Kanban Method, we’ve learned a lot from both. While the Kanban Method facilitates the creation of effective value streams when true teams can’t be created we’ve seen that teams are truly valuable (see Know the Power of Teams).

I started discussing the value of teams on the Kanban Development board and essentially ran into the same attitude as I had years ago. First it was – “teams are orthogonal to the method.” Then it was “let’s not discuss this.” Then it was “you (meaning me) are not behaving professionally.” This time I left before getting moderated or thrown off. But it’s a bit distressing to see the same attitude – “our method is correct, let’s not discuss things that aren’t defined in it.”

Tragic.

Practitioners – please don’t fall into this trap. There are many consultants out there not attached to any one method. Because of this, they often don’t have an accreditation from either the Scrum Alliance or the Lean-Kanban University. Don’t take that as a negative – it might be a plus.

Al Shalloway
CEO, Net Objectives

Author:

alshall

ISC StormCast for Friday, June 14th 2013 http://isc.sans.edu/podcastdetail.html?id=3368, (Fri, Jun 14th)

Johannes Ullrich — Fri, 14 Jun 2013 08:04:11 CDT

...(more)...

Thoughts On A Job Done

Alex Russell — Thu, 13 Jun 2013 16:04:15 CDT

Most of the time, when a bit of software you work on floats out of your life and into the collective past, there’s a sense of mourning. But that’s not how I feel about Chrome Frame.

The anodyne official blog post noting the retirement six months hence isn’t the end of something good, it’s the acknowledgment that we’re where we wanted to be. Maybe not all of us, but enough to credibly say that the tide has turned. The trend lines are more than hopeful, and in 6 months any lingering controversy over this looks like it’ll be moot. Windows XP is dying, IE 6 & 7 are echoes of their former menace, and IE 8 is going the same way. At the front of the pack, which is now most of the world’s users, new browser releases are reliably being delivered to users without friction. The evergreen bit of the web is expanding, and the whole platform is now improving as a result. This is the world we hoped to enable when Chrome Frame was first taking form.

I joined Google in December ’08 expressly because it’s the sort of company that could do something like Chrome Frame and not screw it up by making it an attention hogging nuisance of a toolbar or a trojan-horse for some other, more “on brand” product. GCF has never been that, not because that instinct is somehow missing from people that make it through the hiring process; no, GCF always was a loss-maker and a poor brand ambassador because management accepted that what is good for the web is good for Google and was given the freedom to act in that long-term interest.

Truth be told, we weren’t the right folks for the job. MSFT has the freaking source code for IE. There was always grim joking on the team that they could have put GCF together in a weekend, whereas it took us more than a year and change to make it truly stable. Honestly, if I thought MSFT was the sort of place that would have done something like GCF, I probably would have applied there instead. But in ’08, the odds of that looked slim.

Having run the idea for something like Chrome Frame past one of the core IE team engineers at MIX that year, the response I got was “oh, you’re some kind of a dreamer…a visionary”. I automatically associate the word “visionary” with “time-wasting wanker”, so that was 0 for 2 on the positive adjective front. And discussions with others were roughly on par. Worse, when the IE team did want to enable a cleaner break with legacy via the X-UA-Compatible flag, the web standards community flipped out in a bout of mind-blowing shortsightedness…and MSFT capitulated. Score 1 for standards, -1 for progress.

For me, personally, this has never been about browsers and vendors and all the politics wrapped up in those words: it has been about making the web platform better. To do that means reckoning with the problem from the web-developer perspective: any single vendor only ships a part of the platform that webdevs perceive.

Web developers don’t view a single browser version that’s on hudreds of millions of devices as their platform they way they would view an OS like iOS or Android (which might have much smaller penetration) a “platforms”. The promise of the web has always been universal access to content, and web developers don’t view any leading browser as their platform, they view the full set of browsers that make up the majority of use as their platform. That virtue that both makes the web the survivable, accessible, universal platform that can’t be replaced as well as the frustrating, slow, uneven development experience that so many complain about.

The only way to ensure that web developers see the platform improving is to make sure that the trailing edge is moving forward as fast as the leading edge. Just as the oldest cars and power plants are exponentially worse polluters than the most modern ones, browsers encode the same dynamic; if you want to do the most for the world, get the clunkers off the road and put scrubbers on those powerplants. Getting clunkers off the road is what upgrade campaigns are, and GCF has been a scrubber.

All power plants, no matter how well scrubbed, must eventually be retired. The trend is now clear: the job coming to a close. Most of the world’s desktop users are now on evergreen browsers, and with the final death of WindowsXP in sight, the rest are on the way out. Webdevs no longer face a single continuous slope of pain. We can consider legacy browsers as the sort of thing we should be building fallback experiences for, not first-class participants in our development process. The goal of making content universally accessible doesn’t require serving the exact same experience to everyone. That’s what has always made the web great, and now’s the time for non-evergreen browsers to take their place in the fallback bucket, no longer looming large as our biggest collective worry.

I’m proud to be a small part of the team that made Chrome Frame happen, and I’m grateful to Google for having given me the chance to do something truly good for the web.

First milestone of Spring Data release train Babbage arrived

Oliver Gierke — Thu, 13 Jun 2013 13:02:28 CDT

I am pleased to announce the first service milestone release for the Spring Data release train named Babbage. It includes the following modules: Spring Data Commons 1.6 M1 – Changelog Spring Data JPA 1.4 M1 – Changelog Spring Data MongoDB 1.3 M1 – Changelog Spring Data Neo4j 2.3 M1 – Changelog The first milestone includes Read more...

Stupid Little IPv6 Tricks, (Wed, Jun 12th)

Johannes Ullrich — Thu, 13 Jun 2013 11:00:26 CDT

With the IPv6 Summit on Friday, various IPv6 related topics are of course on my mind. So I fig ...(more)...

ISC StormCast for Thursday, June 13th 2013 http://isc.sans.edu/podcastdetail.html?id=3365, (Thu, Jun 13th)

Johannes Ullrich — Thu, 13 Jun 2013 08:00:41 CDT

...(more)...

Microsoft June 2013 Black Tuesday Overview, (Tue, Jun 11th)

Johannes Ullrich — Wed, 12 Jun 2013 16:16:13 CDT

Overview of the