Oct 26 2016

The IBM Interoperability Challenge

I was walking early this morning to the keynote area to get ready for our demo.

Interop Challenge

We were all fidgety backstage and ready to get started. They gave us plenty of coffee, food and round tables to spark conversations. It was a fun breakfast.



Later 16 of us went on stage at the keynotes of the OpenStack Developer Summit and run a workload on our respective clouds all at the same time. All the workloads run smoothly from stage. One ansible playbook did run smoothly on 16 different clouds. Amazing. One of them was the Linaro Developer Cloud, running on ARM64 servers. This shows how much all the work we have been doing at Linaro with the different upstream projects has paid off and how smoothly OpenStack runs on ARM64. Exciting times ahead!

After the fact we got a token of appreciation and recognition to all the hard work that went into making the challenge a reality. Thank you!

The prize

The prize

This demo was all about demonstrating that one single workload can run smoothly on different clouds as long as they adhere to certain rules, the interoperability requirements. These requirements are being worked on by the Interop Working Group (was: DefCore Committee) and I would like to encourage everyone to get involved!

And now back to sessions and conversations. Very productive week!

Oct 05 2016

OpenStack on ARMv8, the journey

Deploying OpenStack on AArch64 has been an adventure. At Linaro, we have been working on that for a while. Last week, at Linaro Connect in Las Vegas, we shared the journey of making OpenStack work on ARMv8.

OpenStack is high level and written in python, so one would think it is easy to run it any architecture and get the same results. This is partially true, we haven't had many issues with OpenStack as a whole, but with the way it interacts with the hypervisor, the way devices are configured, the way libvirt/qemu behave on AArch64 (and OpenStack's expectations) compared to the way they behave on x86. 

If you are interested on the topic, here is the presentation that we gave last week in LAS16:

Sep 22 2016

Ready to set up 3rd Party CI for AArch64

I am so excited after attending the OpenStack Newton Midcycle for Infra/QA this week! 

I have spoken to the OpenStack Infra team and they gave me advice on what we need to do to set up a 3rd Party CI for AArch64. We have a Moonshot with a few cartridges ready to be put to good use testing OpenStack:

Moonshot for 3rd Party CI

Moonshot for 3rd Party CI


I also had the chance to discuss with some of the tempest folks about the Interop Test Spec and we had a first discussion about how to lower the barrier of entry to run tempest by autogenerating a tempest.conf that can be used to run the interop tests. All of those are TODO items on my list for the coming weeks… cannot wait! smiley

But first things first, I am on my way to Las Vegas next week for the Linaro Connect. We will be discussing, amongst other things, AArch64 and the OpenStack work we are doing. 

Sep 20 2016

Screen freeze

I have been running Arch Linux happily for a while now. A few days ago, however, I started to suffer from what I considered to be random crashes. The UI would freeze, mouse not moving, not responding to keyboard either and the only solution would be to reboot. This was ok when it happened once every few days, however today it started to happen every 5 minutes. I would reboot the laptop and the screen would freeze as soon as I had all my browsers open again. 

I noticed a strange message on dmesg:

[drm:intel_pipe_update_start [i915]] *ERROR* Potential atomic update failure on pipe A

It would appear once, then twice, then three or four time and the machine would freeze.

After a bit of research and a full update of my Arch Linux that didn't fix the issue, I tried the following:

% cat /etc/modprobe.d/i915.conf  
options i915 enable_psr=0

Then rebooted the laptop and I have had half an hour of undisturbed typing now. 


Aug 10 2016

Arch Linux newbie

A couple of weeks ago I decided to try Arch Linux, not only because I like to try new things every now and then but also because the idea to use new versions of my favourite software and the opportunity to try rolling releases was too tempting to pass. I have also fallen in love with the Arch Linux wiki and extensive documentation.

My laptop has encrypted partitions and it also has several OSs installed. I went through the Installation Guide and learnt a lot about how Linux boots and how the configuration works… but at some point I got distracted and missed a key piece of information, forgot to install the xf86-video-intel drivers for my laptop. The Vesa drivers sort of worked, but I had a problem when connecting an external monitor, it would often switch off for 2 seconds and then come back again. And I would see this error in my dmesg:

[drm:intel_cpu_fifo_underrun_irq_handler [i915]] *ERROR* CPU pipe A FIFO underrun

After installing the drivers all is good. I am quite happy with the user experience and with all I learnt from the installation!

Aug 04 2016

DefCore Test Spec

Last cycle we started talking about having a test spec to describe what a good interoperability test is and how to make tempest tests better for interoperability purposes. During the time it took to write the spec and get it approved I realised why OpenStack developers were confused about the requirements: it took us almost three months of meaningful discussions to agree on what the recommendations for good interoperability tests for OpenStack are.

Today we have finally reached a consensus and the test spec made it into the DefCore repository.

The DefCore test spec can be found here.

Thank you everyone for the reviews! This document is work in progress but it clarifies a set of rules that were implicit and controversial before, which in my opinion is a big step in the right direction.  

Jul 27 2016

AArch64 and OpenStack in Barcelona

It has been an awesome two months since I started at Linaro. I have had the opportunity to experience AArch64 servers and to spin a few DevStack clouds on them, plus do some good testing. We have a few patches landing soon that will make the AArch64 experience smoother.

I have also had the opportunity to admin a real cloud based on AArch64. Fun!

My team is preparing to do a few talks at ODS Barcelona and we are looking for support and votes on them! So if you are interested on what we are doing and how to get involved, make sure you give us your vote:

  1. Go to this page.
  2. Search for: "AARCH64"
  3. Vote!
  4. Then vote for all the other talks you like also laugh

See you all in Barcelona!



Jun 10 2016

Joining Linaro!

I joined Linaro this week. I have taken on the role of Tech Lead of Software Defined Infrastructure, in other words, making cloud related software run smoothly on ARM. We will be testing OpenStack and Ceph amongst others, on ARM servers making sure they run as expected. 

More details will follow, for now I have a lot of things to learn and a huge TODO to get through. I am very excited with the new role and the challenges ahead! 

May 20 2016

Contributing to DefCore

I started attending the IRC meetings of DefCore during fall last year. I found the topic of cloud interoperability fascinating and tried to learn as much as possible to be able to start contributing, needless to say I was useless for a while until I understood the basic concepts and what the DefCore Committee was trying to achieve and how. Figuring out what interoperability means in a project as complex as OpenStack.

There is a very interesting article written by Mark Voelker that I think to be an excellent explanation of what we do.

My first commit happened on April 20th, 2016. I managed (with a lot of help from everyone in the team) to score Keystone's capabilities. I had to look at the Keystone APIs, understand them, look at tempest tests using those APIs and understand them and also talk to the Keystone PTL to make sure I wasn't assumming anything wrongly. I learnt a lot in the process and I would encourage any quality engineer working with OpenStack out there to take on the challenge of scoring a project in the coming releases.

Now I am working on a new commit, the DefCore Test Specification. There is a lot of fruitful discussion around the topic and I am learning a lot trying to summarize and make sense of everybody's opinions/experiences whilsts trying to do what I think is right also. Any comment/review welcome, please contribute your thoughts!

Oct 29 2015

OpenStack Summit 2015 Tokyo – Day 3

Today was the last day of the OpenStack's Summit Main Conference. Tomorrow there are still developer's sessions ongoing for hacking code and continuing pending conversations, however the bulk of people goes home after today. I did some networking today due to the amount of down time I had, I even got a few minutes to send a postcard from Japan home. 

In the morning I attended two QA working sessions: one of them about resource configuration and the other one about tempest-lib service clients. It was quite interesting to get a heads up of where things are going with tempest and get help from the tempest PTL with a little problem I was having running tempest. It felt like a very productive morning. 

Later I took a bit of time to socialize with one of the ladies that we met at the Women of OpenStack event on Monday. It was lovely to get to talk to her and learn about Japanese work culture. I am looking forward for her to visit the UK and be able to meet again. 

Afterwards I attended a session about debugging the virtualisation layer, by someone at RedHat. I found the talk interesting and I personally learnt quite a bit about how to more efficiently diagnose issues and figure out exactly where things are going wrong. The speaker also shared his blog: http://kashyapc.com/. I have decided to add his blog to the list of blogs I follow.

Towards end the day I attended a double RefStack session about the future of RefStack and the long term strategy. The RefStack team and the DefCore committee are interested in seeing full test suite results (not just the tests used for DefCore certification) for data analysis purposes. As they stand the tempest tests are sometimes unreliable or they may be testing a capability that nobody is interested in implementing, this is all valuable input into the capability review process. There will also be the ability to have "companies" not just "users" on the RefStack website, so that people can upload "official" runs to their respective companies and still provide other runs / keep them for history reasons. When the data is associated with a user, it belongs to them. Once the data is moved to their company, then it stops being theirs and starts to be owned by the company. 

It's been an awesome day. And after dinner I saw with a hint of disappointment the tear down of the main event:

Tear Down

Tear Down

I am heading home soon with a lot of ideas and energy to make them happen… #WeAreOpenStack