I did work for years using the full XP/agile stack on a daily basis, after all that experience I can now finally judge when to use a methodology and when it's not the case. In the last years I had more than once the impression that all the various tools that are supposed to reduce the lead time where actually slowing me down. Now I am lucky enough to work in an environment that doesn't force me to use any particular methodology or technology. So, how is that possible that here we go live in weeks and not in months? What happened? No CI. Yeah, we don't use CI. How does it work? - Hire good developers, most of the times they will not commit&push crap - Keep your application simple, prefer having lots of small applications collaborating rather than having a monolithic huge application - Deploy live rather than on CI, that's where the system really runs, not on CI, that's where you will really understand if it's broken or not - Integrate often. The fact that the CI runs for every commit doesn't mean that you are integrating. Pushing live your changes is real integration - Failover on a previous version of the code/automatic rollback on the live system if the deploy breaks the application Test but not that much. Profanity? - Hire good developers, most of the times they will write decent code without a test - Hire good developers, most of the times they will test themselves the code they wrote - Keep your code simple, as simple as possible. If a method is 3/5 lines of code and your code base is small it will be simple enough to change/maintain/evolve - Write a test when you start feeling uncomfortable, most of the times a http://en.wikipedia.org/wiki/Read-eval-print_loop">REPL will be actually good enough to prove you are writing your code in the right way Ok, what about maintainability and refactoring? Let me tell you, refactoring is overrated. Maintainability is overrated. Write your application as prescribed above, use some dynamic language that will help you going faster and as soon as you will be fast enough you will realize that if you are not happy with your code you can actually trash it all out and write it again from scratch, it will take you less time. I've seen this working here, I am not talking bullshit. I've seen in the past projects that should have taken weeks taking months with huge amount of wasted time spent fixing the build, fixing the selenium tests, maintaining hundreds of tests, writing non sense mocks tests, writing non sense tests. You may have embraced change, now it's time to embrace courage, be a fearless dev.

I've been involved in quite some agile consulting gigs in my work life. There's one lesson I want to share: if you want to change an organization you have to change their company culture first. If you are trying to employ a consultancy firm try to get ready your people to start with, avoid wasting money making sure that your employees are ready to embrace the change. Make sure that you communicated well and straight that things are gonna change now. If you are a consultant, make sure that you will have enough slack on the project to work on the company culture, plan, discuss with the client when and how that work will be performed. Otherwise change won't work: people will work in micro waterfall iterations, they will use continuous feedbacks to drive others people carriers, they will still micro manage their teams, they will eventually fail and all their effort to "change" and your effort to make them "a better" place to work for will vanish without evident benefits.

Lately I've been working quite a lot with environments and release issues. On my last project we had eight environments: development, continuous integration, test, performance, pre-live and live plus a couple of demo boxes. This over-engineered  situation caused few issues but at the same time made me think and write this blog post :-) What we did to improve deployment Well, automation is the key, even if we are in a windows environment we wrote remote deployment scripts to do remote deploy in the machines using psexec and robocopy. We template our config files and we have one config per machine. So far so good, old school good practices. What still doesn't work Your software is done, finished, all the test are passing, unit and integration on the development boxes, acceptance on the CI box, the QA is happy in the testing environment and the product owners are happy on both of the demo boxes, the performance test team is happy on the performance box, the security team is happy with their tests executed on the pre live box. But still your sofware may not work on live. Or most likely in any of those environments. Configuration is vital Your application without configuration doesn't neither start or with a wrong configuration won't log or work properly. There's test coverage on all the aspects of your application but not on environment configuration. You may misspell a name of a server or simply forget that something is different from environment to environment. Your application won't work as expected. The configuration of the environment itself may cause serious problems. The most different your live boxes are from your test/integration/performance ones the higher is the risk that your software won't perform as expected on the target servers. So, my humble recommendations Move fast Moving fast is essential in any aspect of software development, when talking about releasing software is essential that you should be able to continuously deploy and redeploy in a repeatable stable way your application as fast as possible. Use remote scripts, even if you are not working with the cool kids in the ruby world do something similar to what capistrano does. Have a look on the heroku website for a good example of moving fast... Reduce as much as possible the number of environments It's clear that in the above example we have too many environments, if you have full control ask yourself why do you need them and just take them off. If an environment doesn't produce any value it's just a waste of time and resources. Environments can be recycled, most of the applications won't need two demo environments and a performance environments available for the whole project life-cycle. How would I cut down the number of environments on my previous example? Development, continuous integration, test & performance, that's it, four, it's a 50% cut down! A good rule is that your environments should be the same number of your story wall lanes:

1. Story in dev > development environment
2. Story dev complete > CI environment
3. Story in test > test environment
4. Story ready to sign off > performance environment

The performance environment can be used as stable build environment for showcases, demonstration of the software and indeed performance tests. But also for one last, more important thing, which is the whole point of this blog post: Promote environments not artifacts Imagine that you're happy with you software, it has been signed off by the product owner and it's performing well. Probably you made some changes to the performance environment, maybe you started with a cluster of two machine and ended up with a cluster of four. Now you want to go live, but the live environment that you have is different, you may have some problems. My idea is to promote the whole environment. You are probably using cloud computing or virtualization, just move this stuff to the live environment, clone it and you are done. Promote the environment with the artifacts, all together. Try to define your environments as executable scripts (in the *nix world won't be that hard) and version control these scripts with your environment configuration, the creation of the whole environment should be scriptable, repeatable and should evolve with your code. Mark told me that Chris Read has been talking about this stuff for years, you may want to check out his blog or his presentations, he's anyway way much better than me explaining and making these things real.

Somehow it happened, the wall was full of cards, literally there was no more space for adding anything anymore. I didn't count them but probably we had more cards this week than in the last 2/3 iterations summed up together. I went mad, I started grouping them. Then I realized why I hated so much all those cards sticked on the wall: it seemed impossible to make it. Putting tasks on the story wall is a very dangerous activity, basically you loose in the blink of an eye all your knowledge on the team work load and on the team todo and done. A card is like a tile, the lanes on the wall have fixed size, it's a constrain, it's a choice, you cannot change the size of the smallest story while the project is ongoing, you will loose track of what's going on. Extending this then I thought about our bigger story cards, we are currently using three sizes, 1 story point (S), 2 story points (M), 4 story points (L). In theory you may try having three different sizes of paper cards, probably in that way you will better understand the load on the wall, or even better try to split those huge cards in smaller ones, as much as possible (always good). Putting tasks on the story wall using the same card format/color as stories is seriously dangerous, use a dedicated swim lane, group them or write a card that contains similar tasks all together.

Agile is hard to embrace, agile is hard to teach. "The problem" (and the strength) of agile development is that it never defines strict rules to follow always and in any occasion. I can imagine myself being a consultant preaching RUP (IBM I am not impressed with you guys preaching agile), I'll give to the client the book, I'll tell them to follow my rules, end of the story. Now, let's forget the manifesto, let's forget all the blog posts, the white papers, the conferences. What the world agile means? I've searched on the Mac Dictionary application It says:

agile |?aj?l| adjective able to move quickly and easily : Ruth was as agile as a monkey | figurative his vague manner concealed an agile mind.

That's all you need, be agile like a monkey. Embrace iterations of two weeks? Fine, after some time if you feel like you can go faster without, take them off. Embrace a retrospective per iteration? Fine, after some time if you feel like they are not making you go faster, take them off. Embrace the iteration planning meeting? Fine, after some time if you feel like they are not making you go faster, take them off. And once you took off all the practices, start from scratch, try to understand if re-introducing them will make you go faster, be agile like a monkey in the jungle, don't follow pre defined paths.

I recently had a conversation with a project manager, who was asking me (tech lead on the project) if I was on top of everything and if everything was under control. We are due to go live in a couple of weeks and we finished the development almost a week ago, we run some performance tests and fixed a couple of issues, we deployed the system on the live servers already and everything is fine. Well I smiled at the project manager, failure happens, what if an heart-quake will destroy both of the data centers were the application will be deployed? Damn! We won't be live that day! Will the world end? Nope. Failure happens, day by day, iteration by iteration, release by release and even if you've done everything right...

Sometimes you hit the bar, sometimes the bar hits you!

* Be ready to fail and keep in mind that failure is there ready to hit you, the only thing you can do instead of worrying is mitigating the risks and prioritize your actions against it.

In Che: Part One, Ernesto Che Guevara states:

In War and Peace Tolstoy remarks that military science assumes that bigger armies with more men wield greater force. On the other hand, only vaguely, do they recognize that during military combat the final strength of an army is also its true physical capacity multiplied by one unknown variable. One unknown variable. This variable is none other that the spirit of the troops measured as their greater or lesser desire to fight and confront danger. Men with the desire to fight who also understand why they are fighting regardless of who they are fighting whether under military geniuses or those of normal intelligence fighting with clubs or machine guns that fire 30 rounds a minute these men will put themselves in the best positions and so they will win.

Immediately after hearing this I've started a comparison with agile projects. It's obvious in these days, with agile becoming mainstream that bigger armies (expensive tools? costly hardware? costly solutions?) and bigger teams don't imply success in software development. What is then the unknown variable in an agile team? I would compare the physical capacity with the knowledge base of the team, with the general collective intellect. The desire of fighting is nothing else than desire of success and play with risks and issues in a project. But then, it's in the understanding of the why they are fighting the most interesting part. I've seen teams over-performing in few different situations: some were strongly committed to the success of a startup, feeling part of it (my experience at Fluidtime), some other were committed to reach excellence (a common behavior at ThoughtWorks). The why can be different from team member to team member, some people need to feel that they are part of a whole, some others need recognition and differentiation. Some others are carrier driven, some others have a strong desire to learn, improve. Finding the right motivation in your team members will make your team run faster and smoothy, the unknown variable will boost your speed and quality.

I really loved a couple of phrases from the Ted's blog post "Agile is treating the symptoms, not the disease": the title and the last sentence: "We are in desperate need of simplicity in this industry.". Complexity Writing software it is still way to complex in these days. I can recall few attempts to make it easier, they all failed: Java Enterprise & EJBs, Web Services, Enterprise Service Bus. Rather than making things simple they forced you to write more code, to use lots of xml, tons of libraries, implementing unnecessary interfaces. How many days do you waste every time you start a project, just in infrastructure & build files? REST seems so simple, but implementing REST in the right way seems always so damn complicated, so everybody writes RESTish services, a good progress compared to old style web services but still... What about web applications? I like the simplicity that can be achieved with Sinatra, Scala , WebLocks and Seaside. But still, the impression is that nobody is trying hard enough to make all these things even simpler. The Twitter example I've started using Twitter only few months ago (even if I've the account since 2006...), it started to make sense when I've linked it to my Facebook account. Facebook is like software in these days: way too complex. Too much stuff, too much noise, too difficult to get what you really want. Then I've started to use TwitterFeed as well, I've my flickr, google reader and tripit profiles feeding the twitter account. So we have three content providers, one content consumer and forwarder and one content receiver, it's a fairly complex architecture and it never failed so far, it's quite stable, it just works. I didn't had to write a line of code, I've pressed just few links, it all works using RSS in a RESTish way. Simplicity How do you imagine software development in ten years? I do imagine it like in the twitter example, an aggregation and collaboration of few simple services working together. The role of the Agile movement Agile is treating the symptoms, it's true, I've seen many times agilists (including me) preaching for the old good principles (don't reinvent the weel, keep it simple, etc) but how many times do we have to rewrite the same software? For how long will we have to rewrite the same objects, the same business logic? Why are we still writing web applications like ten years ago? Why are we still using Spring, Hibernate, Databases, Layered Architectures? I think it's time for laziness, simplicity and more real reuse, somebody told me that it's an utopia, but even only trying will make the IT world a better place to be. Agile can cure the disease The Agile developer community, being so open minded and always embracing chance has a great chance to change the way software is written. There's not so much new from a process point of view in these days (at least, that's a complaint that I've heard from many people, especially coming back from big Agile gigs), but there's so much work to do on the technical side of things. The NoSQL movement is one of the signs that something new is just born and growing, don't miss the opportunity.

This year I've decided to propose an "experimental" session: a wall will be available during the conference and the attendees will be asked to put their questions/puzzles about agile, with a special focus on real life projects. The wall will be handled like a retrospective one, clustering common subjects and people will be asked to vote, then I'll choose the most interesting and I'll talk about it . This will give me a chance to understand how and where agile software development is going in Italy and it will be quite a challenge: I won't bring my Mac, I won't prepare anything, pure improvisation! Surely I'll blog about it right after, and hopefully I'll find some time & energies answering to some of the unanswered questions on the blog.

One of the biggest strength of Agile software development is the capability of mixing up many different tools and techniques. Agile is not a check- box list of practices that you have to embrace in your project. With this experience report I want to share the experience of a continuous, repeatable successful project: the agile cocktail we created worked perfectly for us. This is not necessarily saying it will be perfect for you. However, on any future project I will always begin with this mix, inspecting and adapting it afterwards, only changing if necessary.