Tutorials at MSST Spring 2014

The first day of the MSST Spring 2014 conference had Ceph front and center. It was a beautiful late spring day on the Santa Clara University campus in California. Jean-Charles Lopez from Inktank led us off with a detailed background in Ceph’s inner workings. The inventor of Ceph, Sage Weil, was on hand to provide an even deeper dive into the internals. Can I summarize by saying that the Object Storage Device is connected to the journal, librados (direct access through libraries), radosgw (REST gateway), libcephfs (POSIX), and librados (block)? I don’t do the hours of presentation justice with one sentence. The whole of the Inktank Ceph presentation can be found here. I can say that between the access flexibility and their CRUSH data placement algorithm, Ceph is an impressive storage solution. We ranged from Ceph on paper to Ceph in action. The session wrapped up with a full tutorial using a pre-built VM to install Ceph v0.67 Dumpling on. It went very smoothly. I learned new things about Ceph and how to run a training session. Thanks Inktank!

We transitioned into the Seagate Kinetic project. Ricky Huynh and Chiaming Yang lead the presentation. Kinetic pushes the boundary of traditional disks. Seagate has shruged off the commodity label and created something new out of their standard disk offering. They have replaced the onboard SATA daughter card with a CPU, an ethernet connector, and an key-value API. This fits nicely with Ceph, so the bottom layer of disk services can be replaced with a Kinetic key-value disk. No more pesky file structures. While this solution doesn’t likely fit all use cases for distributed storage, it is a welcome option in the nascent field of new tricks for spinning platters. Sage Weil pulled double duty by presenting the technical benefits of Ceph on Kinetic. No Kinetic presentations posted yet. I will update this post when they are available.

This was a good day. See you tomorrow.


OpenStack Juno Summit update

These are not all the etherpad links nor all the topics discussed. I am posting my interests for your reference. Feel free to link and comment.

We have been working on updating the definition of the Openstack trademark requirements since last summer. Defining the capabilities of OpenStack clusters leads to interopability. The Refstack project is the culmination of the board, the technical committee, and the tempest group’s work. The work is progressing nicely.

CI / Infrastructure
There were a lot of good sessions. Including ideas on replacements for Jenkins and Gerrit. Here are some of the sessions.
Negative testing 
improve gate uptime and feedback 
juno cycle improvements 
elastic recheck, avoid race conditions 
turbo-hipster as the replacement for jenkins 
improving third party CI pipeline integration 
vinz as the replacement for gerrit 

We started discussing what the operators needed with a mini summit a few months ago. We followed up on the actionable items with operator summit sessions with the Project Technical Leads. Intel also helped with enterprise talks focusing on interopability between OpenStack and infrastructure service providers. Find the rest of the operations sessions below.
neutron ops 
scale ops 
nova ops
monitoring ops
ops governance, user committee
ops arch sharing

Launchpad has outlived its usefulness and storyboard is the replacement is under development. It is looking very promising. Find it’s design session here. I am thinking of adding one of my projects to it as well.

Policy management is a hot topic as we can not build SLAs without it. Each of the OpenStack projects have some policy controls. Congress intends to tie them together. The 3 hour design session notes are here. I have have posted some about the project in past as well. Here are some of the other project policy sessions.
neutron policy 
nova policy or tetris blueprint
nova concept scheduling server groups 

Nova-network parity is critical work. Mark McClain is leading the charge. This work must be achieved in this cycle or returning Neutron to inucubation is on the table for the Technical Committee..
What use is feature parity without a migration path. Obondarev is working on that.
We need to share the review load. The idea of assigning 2 reviewers to a patch for critical, high priority blueprints was discussed. Perhaps a rotation on call. Tracking critical high, critical bugs to resolution was also considered.

Other various, important design sessions listed below.
Additional testing coverage

Lbaas Juno design

LBaas SSL, L7 focus  
IPv6 comcast, intel work so far
neutron core refactoring
state management
distributed virtual router, DVR update

We want continuously improve the OpenStack user groups. The ambassador group got together and discussed ideas for kick starting new groups and quality improvements. I will be posting on these ideas soon.

teaches openstack 12 jul 2012

OpenStack Community Developer Training, part 2 of 3

This post is a follow up to a previous post here.

Teaching the OpenStack architecture and services is a good place to begin. That means no installing software until  the student is ready to understand what they are installing. The best place to start is going through the basics in the Associates Training Guide and the intermediate topics in the Operators Training Guide first. These Guides are  mostly combined from the existing OpenStack documentation.  The Training project aims to reuse the existing materials as much as possible.

The Developer Guide deep dives on the OpenStack CI pipeline.  We start with the local tools Pycharm, Git, Sourcetree, Maven, and Git-review. Some review of Gerrit etiquette is in order and why resubmission of your code is part of the learning process. We then dive into the CI pipeline tools GitHub, Gerrit, Jenkins, Gearman, Nodepool, Jeepy, Logstash, and Zuul. The student will have five days of classroom and lab time.

The next three days are devoted to learning the OpenStack APIs in depth. We settled on using  Django and Horizon to interact with a variety of OpenStack APIs. We aim to teach enough of the OpenStack API details in three days that the student will have a good grasp of how the APIs work. This should be the basis of further API development during the Architect Training. The Developer Training Guide is under development itself. The Developer Training Guide design is posted here.

There is a glaring, purposeful omission in that learning Python is not part of the Training Guides. Python is the language that OpenStack is written in. Understanding Python is required to become an OpenStack developer. There are some great ways to learn Python out there. The main Developer Guide prerequisite is to learn Python. We recommend Python Koans and Learn Python the Hard Way.

The next post will detail how we plan on implementing OpenStack Developer Training at a location near you.


MSST massive storage conference

Just a quick reminder that early registration for MSST 2014 is available only through this Friday. You can register at:


The great program is generating a lot of interest.
Ceph training with Seagate Kinetic on Monday.
Attendees can sign up at the conference to give short talks in the Lightning sessions on Tuesday and Wednesday.
Research and other great material.

See you in Santa Clara!


Why I Support Mark McClain for OpenStack Neutron PTL

I am biased. I admit it. Mark McClain is a friend and a fellow Yahoo. I think he is doing a good job as the current OpenStack Neutron PTL and he deserves another 6 month term. That being said, I have many friends in this business and they are all good people. There are no bad guys in this picture, only differing views. I do have a few points to make as why Mark is still the best choice for the OpenStack Neutron PTL Continue reading

Thoughts and narration on the business of open source