Roadmap review and retrospective – 2016: DbOps definition

2016: DbOps definition

Source Control

  • Mercurial (TortoiseHg) vs Git (SourceTree)
  • Flyway
  • Redgate (Source Control, DLM Automation Suite)
  • ReadyRoll

Continuous Integration

  • Jenkins
  • TFS Build vNext (2015)
  • Database unit/integration tests
    • tSQLt
    • DbFit

Continuous Delivery

  • Octopus Deploy
  • TFS Release Management (2015)

Infrastruture as Code

  • Chef
  • Puppet
  • Powershell DSC
  • Azure Resource Manager

During 2016 the main focus around DevOps was the its definition. My DevOps definition was built around databases operations and development (hence the title “DbOps”).

The plan was to cover the 3 stages of the deployment pipeline: source control, continuous integration and continuous deliver. And include infrastructure.

  • Source Control
    • Continued to use Flyway with Git, abandoning Mercurial.
    • Unfortunately, I did not had lot of time and opportunities to use Redgate Source Control and ReadyRoll.
  • Continuous Integration
    • Went full gas with TFS (and VSTS) for building process.
    • Tried tSQLt and DbFit. tSQLt suits best my use cases because allows to write unit tests using T-SQL and for that reason it’s more powerful than DbBit.
  • Continuous Delivery
    • Made the first approach to TFS/VSTS Release.
    • Still preferring Octopus Deploy when compared with above one.
  • Infrastructure as Code
    • ARM templates (and PowerShell) became the only way to create and manage Azure resources.
    • Made the first approach to Chef and Puppet. They are both very powerful and similar, but in my point of view Chef can be more friendly when learning.

At professional level, after more the 3 and half years I left Celfinet and joined Basecone. It was  the right time for me to leave and start a new adventure.

Also, 2016 was my year as rookie speaker. It was great to share my talks, to learn with attendees and other speakers, to feel the community spirit.

In October 2016 it happened the first DevOps Porto community event/meetup. This community, created by me and my friend Miguel Alho, have the purpose to share, discuss and promote the DevOps culture, values and practices. For me was a big personal achievement.

Scrum Gathering Portugal 2016

Scrum Gathering Portugal was my last event of  2016. My good friend Miguel Alho (@MytyMyky) and I delivered the workshop “DevOps: delivering changes for applications and databases”. The purpose was to explore the key technical (and cultural) aspects of creating a continuous delivery pipeline, from code inception to getting the conde into production.

We created so much and so good material that we hope we can repeat this experience/workshop in the future.

SQLSaturday #546 Porto 2016

This was my second SQLSaturday Porto as member of the organization team and my first time as speaker.

After the stressful initial preparation: conference rooms, speakers room, check-in area, networking/coffee break area and all the other logistic details, I presented my session about source control for database.

The feeling and the feedback from attendees and speakers about  the event organization was globally positive. The same thing can be said about my talk.

You can find the slides here:

SQLPort 22 September 2016 @ Aveiro – Database source control: Migrations vs State

This was the first SQLPort event held in Aveiro. I was invited by Niko Neugebauer to deliver a session and I chose to explore the topic migrations-based approach vs state-based approach for database source control.

The purpose of my talk was to let audience know about the two existing approaches to start with source control at database level. After presenting the advantages and disadvantages of each approach I promoted a discussion with the participants so they could understand how, where and when they should apply each one of the approaches.

I would like to thanking Niko for inviting me and Ricardo Peres for the support in making this event possible.

Hope to repeat the visit to the beautiful city of Aveiro for more events.

Agile, Scrum and DevOps – The workshop

The workshop “Agile, Scrum and DevOps – Let’s Play with LEGO” is part of  TugaIT 2017 conference schedule and will happen at May 18th. TugaIT 2017 is one the greatest Portuguese IT conference and will happen between May 18th and 20th – 3 days with sessions and workshops to provide the highest possible quality training and knowledge to the community, on a variety of topics.

My friend and agile coach Nuno Rafael Gomes (@nrgomes) challenged me to take his original workshop about Lean, Agile and LEGO Serious Play (more info here) and introduce the DevOps perspective to the workshop. From our numerous discussions about Scrum and Agile he knew that I love to deconstruct Scrum, so it was easy to me to accept the challenge.

Develop software is not enough anymore, you need to deliver it. This is the trademark to my contribution to this workshop where my purpose is to introduce the DevOps values, practices and challenges. In other words, my mission is to demonstrate how DevOps can improve the way you deliver software. I like to call it the “Beyond Scrum” approach, where DevOps arises as the answer to the challenges that Scrum brought to the software development and was not able to solve.

The best part of the workshop is that we will do this playing a game using LEGO. We can garante lot of learning and fun.

Our objectives are:

  • Learn through practice what the agile values really mean
  • Deeply understand the roles, events and artifacts of Scrum
  • Understand the power of Systems Thinking, Feedback Amplification and Continuous Learning
  • Establish the relationship between Scrum and DevOps
  • Understand the 3 ways of DevOps

Topics that we will cover:

  • All 4 agile values
  • All Scrum values, roles, events and artifacts
  • The DevOps definition
  • The 3 ways of DevOps
  • Value Stream Mapping
  • Theory of Constrains
  • Continuous Improvement (kaizen)

Required materials:

  • You just need to bring fun, good mood, and desire to learn!

This workshop is intended to everyone, independently of the knowledge level about Agile, Scrum and DevOps. If you don’t know what is Scrum,, or you never heard about DevOps or Agile, or you already know what’s each one of the topics, join us. We garante that everyone will learn something new!

You can register here: Agile, Scrum and Devops – Let’s play LEGO

Hope to see you May 18th in Lisbon with us.

Roadmap review and retrospective – 2015: Hello WordPress

 2015: Hello WordPress

Introduction to WordPress

In the end of 2015 I decided to create this blog with the purpose of documenting and sharing the acquired knowledge during my professional and personal journal. Writing what I learn is a good habit and helps me to structure my knowledge. Plus, is good to share with more people and even better get some feedback and extend that knowledge.

The name “The privilege of making the wrong choice” was inspired by a music band called Zen from Porto city in the 90s. It represents the privilege of having time and space to experiment, to make mistakes, to learn from failure and success in the working place. Fortunately in Celfinet, my company at that time, I had that privilege.

My first post was about my experience as organization team member of the event SQLSaturday Porto 2015. After that I recovered one old post about a Arduino project and wrote a post about the first experience as speaker in Porto.Data community.

I was able to make my WordPress initiation: create posts and pages, add content including media, manage plugins, appearance and all the different aspects related to a WordPress website.

DevOps Porto as TugaIT 2017 participating community

TugaIT is one the the greatest IT conferences that happens in Portugal. The TugaIT 2017 edition will happen between 18 and 20 of May in Lisbon (Microsoft HQ).

This 3 day conference includes workshops and talks and the following tracks: Azure Infrastructure, Agile & DevOps, Integration, Microsoft Data Platform, Office 365, Open Source Data Platform, PowerShell, Professional Development, Programming, SharePoint.

As a member of DevOps Porto I’m proud to represent one of the participating communities of TugaIT 2017. As a member of this community we hope to contribute to a high quality conference.

If you would like to be a speaker I would like to invite you to submit your talk (or talks): Call For Speakers.

Hope to see you all at the conference, as a speaker or as attendee.

Flyway command-line easy setup

One of the reasons for adopting Flyway command-line was the easy setup process (no need to install). First, I will show how to setup Flyway for a single database, i.e., as if have only one database in your server, and than the setup for multiple databases in a server.

Single database

You can download Flyway command-line here.

After download and extract you have the following folders/files structure:


In this case it’s only necessary to work the “flyway.conf” file (you can find it in conf folder):

  1. Set url to target server/database
  2. Set user and password to target server/database

Alternatively you don´t need to set user and password on the configuration file, it can be provided as argument.

Multiple databases

Because numbers matter! If you have two or more databases in your server, a option is to apply the single database setup for every database. This approach will lead to an unnecessary files multiplication. As alternative we can create a centralized folder dedicated to the Flyway application.

  1. Extract Flyway for a folder and change the name of the folder for flyway;
  2. Create a folder each one of your databases (MyDatabase1, MyDatabase2, …);
  3. Create the folder flyway_conf and copy the file flyway.conf inside;
    1. For flyway.conf file set url, user and password for the target database;
    2. Uncomment the locations configuration and set his value with “filesystem:.” (flyway.locations=filesystem:.). This means that Flyway will scan recursively for migrations the folder that contains the flyway.cmd (MyDatabase1 for example);

  1. Create the file flyway.cmd with the following code inside:

  1. After this you can execute the flyway command from each database.

And that’s it! You are ready for manage the changes of multiple databases.

Multiple servers

If you have multiple servers like: dev, test, staging, … the solution is to create a flyway.cmd and flyway.conf for each server/environment.

  1. Inside flyway/flyway_conf folder create a flyway_servername.conf file for each server (flyway_dev.conf, flyway_test.conf) and set the url, user and password for the target server and database;
  2. Create a flyway_servername.cmd file for each server (flyway_dev.cmd, flyway_test.cmd). Muke sure that you are using the correct configuration file;
    1. Inside of each cmd file should be the following code:

You just have to execute the flyway command for the intended server/environment.

User and Password

Because we want to source control all migrations and Flyway related files, saving the user name and the password in plain text in the configuration file it’s not such a good idea. So, where’s a solution:

  1. Remove your user name and password from your configuration file;
  2. Create the folder “C:\Program Files (x86)\flyway” and inside this folder create the file flyway_ep_dev.cmd
  3. The file flyway_ep_dev.cmd should contain the following code:

  1. Add the folder “C:\Program Files (x86)\flyway” to you PATH.

This way you just have removed your credentials from you source control. You should run the new cmd file from your target database.

#c9d9 Continuous Discussions – Open Source and DevOps

slide1During my vacations in August I participated on a Continuous Discussions (#c9d9) episode about Open Source and DevOps.

Continuous Discussions is a community initiative powered by Electric Cloud, and consists of a series of community panels about Agile, Continuous Delivery and Devops.

This formate, a discussion panel that debates different perspectives about a specific topic, surprised me with the interaction and fun that was to make part of this event. So, well done #c9d9, I really enjoyed.

Following, some insights from my contribution to the panel:

Open source – free as in beer or free as in puppy?

You always have a cost, independently if it’s open source ou closed source. The cost depends on the size of your team, the complexity of your tasks and the frequency of change. The good thing about open source is that you can contribute to the change and take it in your direction. That’s what I like the most in open source.

Where do you use open source?

Open source tools are used in some points of the development pipeline, like for example delivering changes to databases. If you are a startup company there is a high probability to use a lot of open source tools, and with the evolution and the complexity of your team/organization you will probably start to migrate to commercial tools. The rule is “try before you buy it”. You can combine open source and commercial tools like Jenkins, Team City, TFS Build, Octopus Deploy, etc. Even Microsoft is becoming more open.

Where would you not use an open source tool?

In big and complex systems. An important question is, what’s the common factor when you migrate to open source tool,  ot to an closed source tool, in both directions? In my opinion it’s the size aligned with complexity. So, use the right tool for the right job, open source or commercial tools.

Quality concerns?

First, if the open source tool role it’s to support your development pipeline, you have more flexibility to manage the exposure to errors. But, if the open source tool makes part of your product you have to take responsability for that integration, you have to assure that quality is there. Quality must be present everytime and everywhere, however if we are talking at develpoment pipeline level I have more flexibility, but if we are talking at costumer level I have to be more careful.

Security concerns?

Open source tools security concerns make part of the general security concerns. I try to make the development environment as closed as possible.

Legal concerns?

Both, open source and closed source has law concerns. Sometimes the decision is made at a higher level and you can not do anything, other times you can influence or even make the decision, at that time is better the read the license (you probably thinking “who does that?”).

You can see full episode here!