Previous articles on Open Source First have been more strategy than recipe. You need a clear, easy to understand plan for making the case for an upstream project to your manager. To help you with your boss, I have rewritten the How to use Public Projects to Build Products article into a list of ten steps. These steps are comprehensive, covering strategy to implementation. A motivated Developer, Development Manager, or Open Source Director should lead these organizational changes over the many months it will take to implement them. I wish you success on your journey to better, stronger organization.
10 Steps to Supporting Upstream Projects
Strategy: When you set out your strategy and objectives for the year, highlight the open source projects you will be working with. This is important for recognizing the risks, dependencies, and commitments you are making working with the external engineering team that is the open source project.
Partners: Treat a public open source project like another engineering organization. If you are going to work with another engineering team, you expect to have a clear understanding of their responsibilities and timelines. You need to have the exact same expectations when working with a public open source project.
Individual Contributors: Often, a developer will step forward with the desire to work on an open source project that is not part of the organization “plan.” This is exactly what the organization needs. Self motivated engineers are developing leadership skills . Allow the developer to allocate some of her/his time. Set responsibilities and timelines. This is just as much to protect the developer as is it is the company. Encourage open source contributions as technical social good efforts.
Fully Support What You Start: Middle management needs make upper management aware of what the upstream open source work purpose is and its importance to supporting the overall development strategy. Never commit to an open source project that your organization is not willing to fully support.
Reviews: Annual developer performance reviews and headcount updates must include the upstream open source projects. This means that the Development Managers are ready to defend their open source developers with the facts of their work. Developers that excel at working with downstream and upstream development projects are the people who you want to recognize and promote. These developers are very often your best people and likely team leaders.
Products: Make sure your Product Managers understand how the public open source work contributes to delivering the product. In your private Project Management tool, establish an epic for each upstream open source project. Create stories for each upstream feature that is being worked on. Link each upstream feature to a product epic or story that a Product Manager is responsible for. The upstream developer needs to work with the Product Manager at least month to month, so as the upstream work progresses, there is a tight understanding of how that work will go into the Product Manager’s product. A developer advocate like an Open Source Director or Development Manager can be an alternative to work with the Product Manager.
Projects: Make sure the Scrum Master works with the upstream developer just like the rest of the downstream developers. The upstream stories need to be in the backlog along with the rest of the development work. It is likely the upstream schedule of delivering features is different from your downstream product. Adjust your stories for your backlog to match your downstream scrum schedule. Meaning, if your downstream team is on a two-week sprint, then make sure your upstream stories can be delivered in that two-week sprint. Treat all of your developers equally for equal results.
Test, Build, Test, Repeat: As the upstream work starts to take shape, get the code into a testable branch on your software pipeline. Make sure you have quality unit and build tests to verify the upstream work, so it can be more easily merged into your code base trunk when the time comes. Get members of your team to comment and help with the CI effort. As interest in what you are working on gets more help and visibility, you probably will get some public commits from your downstream development team.
Report: Track all your development work and report on progress. Highlight the upstream and downstream work as separate, but equal. Update leadership on the upstream project progress as much as the private development. Approach this as updating on the progress of a valued engineering partner.
Schedule: Keep up to date with the public projects release schedule and strategy. Internally, publish both your private release schedule alongside the public projects release schedule. Alignment of schedules is critical for success.
I have broken down below a simple six part outline of what a technology platform organization is and how to run it. The platform organization is the bottom of the technology stack. The top of the stack will differ from the platform, in that, it will have external customers and more focus on product design and the user interface. However, this outline can still largely apply. This plan is built up over 20 years of working in an around technology organizations. You will find my ideas on Open Source First is the rallying cry of how to run a better platform organization. All the steps that follow have the Open Source First behavior mixed in. Coming from my Open Source First post, transparency is the key thread throughout this outline.
1. What Are You?
A platform organization can be defined as all the underlying functions required to build and / or support customer facing software applications. That includes the data center, server, network, load balancer, firewall, storage, datastore, service bus, service registry, data search, data analysis, data visualization, application server services, monitoring, and alerting (did I miss anything?) in all their various forms and sizes. All that follows is how to run a platform organization with transparency, not what technology to utilize.
2. Know What You Support
Each platform organization supports various services. I loosely call services as something that is listening for requests from a client that the customer is using. There are a variety of service support levels. A Network Operations Center (NOC) is a form of a service. The NOC is listening and waiting for requests. A Service Bus that is implemented at each data center and supported by a centralized team is a service. Each implementation of the Service Bus is listening for and relaying messages.
The software developer creates a software product. A product can utilize services as outlined above. A service can be defined as a product as well. A software product is versioned with specific features and bugs. A product has customers with expectations. A service that is also a product will have similar customer expectations. The critical attribute of a product is the customer. A product has a customer. For example, when you acquire a product and your expectation is that it will continue to function as is for a year. When the product fails after a month, you expect to have it replaced or be compensated. A product has transparently communicated iterations of increasing functionality or also called versions. Each product version has advertised, expected functionality. Product bugs happen, but are expected to be corrected in a reasonable period of time.
So the point here is from a platform leadership perspective, you always have customers. Therefore you manage products for your customers. You have services as part of your product(s) functionality. If you were to only focus on the services, then you miss the customer aspect of your responsibilities. You have products that you stand behind and guarantee of level of service delivery to your customers.
Now Define What Products You Support
Make their versions, service levels, end of life, outstanding customer requests, support, documentation, current features, and features to be implemented transparent to everyone. Being transparent means starting with everything being public information.
3. Know Your Customers
So often, we rush through designing and implementing a great product that has little to no customer interest. You must understand and personally know your customers. Schedule recurring meetings with your customers and key product managers. Hold those meetings on time no matter what. Delivering bad news in person is just as important as good news. Remember that the platform organization exists to serve its customers. And that your platform customers are the people that make the profit in the P&L.
4. Define your Roles
Every organization needs well defined roles with responsibilities. I have used the RACI model to help describe the roles. I have outlined below the most critical roles for a platform organization with a few of their most important responsibilities.
Image 1: Platform Organization Hierarchy
In my definition of the platform organization, everyone reports up through the Platform SVP. There are four Development VP positions for Services, PaaS, IaaS, and Hardware, one Product VP position, and one Operations VP position. Six VPs report to the SVP. The Network Operations Center with the first and second level operations engineers report up through the Operations VP. The Development VP positions oversee their slice of the platform technology and are accountable for the third level operations engineering. The Product VP oversees the Product Managers, Scrum Masters, and Release Managers. The Product Managers report directly into the Product VP, while the Scrum Masters and Release Managers for each technology slice can be dotted line to the Product VP. This allows the Scrum Masters and the Release Masters to be closer to the developers that they support. As long as there is accountability, it should not matter who reports direct to whom.
Each and every person in the organization is expected to be involved in development and operations practices through CI / CD. That means everyone practices DevOps and the resulting agile behaviors.
Platform SVP role
Responsible for the annual strategy derived from the CTO strategy
Accountable for leading the organization
Accountable for hiring practices of the organization
Responsible for the organization budget
Accountable for the platform products
Accountable for on-boarding new customers through platform operations
Recognize the the buck stops here with the SVP. The SVP delegates responsibility to the chain of command.
Development VP role
Responsible for the annual strategy for their slice of the technology organization
Accountable for quarterly objectives
Responsible for leading their slice of the organization
Accountable for hiring in their slice of the organization
Accountable for their part of the organization budget
Responsible for their technology slice platform products
Accountable for layer three operations engineering support
Development Manager role
Very similar to the Development VP role, except they are responsible for the capabilities of the VP in their slice of the organization
Product VP role
Accountable for quarterly objectives
Responsible for the Platform Product Managers and Scrum Masters
Accountable for regular meetings to determine the status of the release schedule
Accountable for planning and organizing the product roadmaps, releases, and reporting
Responsible for the Product Management retrospective and Product Management roadmap review scoring processes
Accountable for the platform products schedule
Responsible for coordinating product release schedules and product milestones across the entire organization
Consults with Security, Operations, Open Source, Legal, and other horizontal teams on requirements and assistance
Product Manager role
Very similar to the Product VP role, except they are responsible for the capabilities of the VP in their slice of the organization
Responsible for prioritizing the Product Roadmap(s) features while working directly with the Scrum Master and Development Manager
Responsible for product backlog scoring used during scrum retrospectives
Collaborates with other Product Managers on Product Roadmap risks and dependencies
Responsible for demonstrations, roadshows, and product showcasing
Accountable for engaging with customers through Customer Advocacy meetings
Responsible for marketing the product for customer adoption and on-boarding new customers
Responsible for accepting or rejecting product delivery from the development team for a release based on product reviews and quality
Accountable for quarterly objectives
Accountable for general operations support (layer one and two) including the Network Operations Center
Responsible for managing alerting and monitoring services
Responsible for managing the CI / CD infrastructure
Responsible for on-boarding new customers into the platform organization
Release Manager role
Responsible for their own personal quarterly objectives
Accountable for software pipeline quality
Responsible for tracking unit and build test application. Are the tests doing anything useful? Do the tests get set to noop during the push for release?
Responsible to work with the Development Manager(s) on how testing can be improved. Are the same tests being run for gate and build? Can some of the tests be run pre patch submission by the developer? Are there some project teams that are having problems with test implementation?
Responsible to work with developers and/or the Infrastructure Manager reviewing software pipeline logs for information and errors.
Responsible to work with the Infrastructure Manager on maintaining the software pipeline. Especial focus on keeping the software pipeline functional towards the end of a product release cycle when there will be heavier than usual load.
Responsible to work with the Product Manager and Development Manager on the product release cycle.
Scrum Master role
Responsible for their own personal quarterly objectives
Accountable for managing scrum or kanban boards for measuring progress against the roadmap
Accountable for holding scrum or project retrospectives
Responsible for working closely with the Product Manager and Development Manager on backlog prioritization
Responsible for working with the Product Manager on the product release
Accountable for managing appropriate engineers’ time management during sprints
Consulted by the Development Manager on feedback around engineers’ performance, productivity, and quality
Responsible for identifying and removing risks and dependencies in coordination with the Product Manager
Responsible for their own personal quarterly objectives
Accountable for completing work assigned by the Scrum Master
Responsible working within the DevOps software pipeline(s)
Accountable for personal technical capabilities
Responsible for mentoring junior engineers
Responsible for practicing transparency and collaboration
5. Communicate on a Well Defined Schedule
Communication milestones that your customers can come expect is critical to gaining trust. Most importantly these milestones provide transparency to the product development process for customers and collaborators. If delivering on these milestones becomes difficult, consider moving the Scrum Master and Release Manager roles under the Product VP for more accountability for the product management process.
Bi-weekly communication to the platform organization
Progress on features
More in-depth information from blog posts
Monthly communication to the company
New product updates
Few blog posts to highlight
Weekly to bi-weekly project retrospectives
Backlog progress scoring based on backlog reviews by the Product Manager
Project epics are created and updated by the Development Manager, Product Manager, and Scrum Master. Then the epics are kept up to date throughout the quarter
Monthly Product Roadmap reviews
Each Product Manager updates their published roadmap monthly
Each Product Manager publishes a Product Roadmap quality score for each product they are responsible for. The roadmap quality score is based on: are all the roadmap details available, is the roadmap published on-time, and the quality of the roadmap details.
Monthly updates on product, project status based on Product Roadmaps and Releases
Report published with last quarter product releases, scoring metrics, progress on features, status of risks, dependencies, and the status of customer requests
Updated, published annual product release schedule
Quarterly product roadmap reviews
Features, bugs, risks, and dependencies
References to epics for cross-project discussions
Quarterly Headcount and Finance review
Travel, equipment, events, sponsorships
Headcount adjustments by project, product, and roles
Fine tuning from the annual review
Quarterly Customer Advocacy meetings
Each functional group of products holds quarterly Customer Advocacy meetings
These customer meetings can happen as often as required, but with the Product Manager(s) attending, representing the product
6. Create a Culture Based on Transparency
Every organization needs a strategy. But very few organizations have a strategy that makes sense to the organization. That is because most people start with blue sky planning. That is a mistake. This is not a company you are running, rather a team that supports an existing company with a strategy and goals of their own.
After putting the platform organization together around products and customers, you have a solid baseline for what your strategy and goals need to be. Before you would have wasted your time. Now you can be clear and spend the minimum time planning.
Using that baseline, along with the CTO strategy mixed in, the Platform SVP maintains no more than 5 annual goals, created two months before the new year starts. Those annual goals are the strategy for the organization. The strategy must be clear and timely, so everyone can reference their part in delivering the strategy. The platform leadership, Platform VPs, will need use the annual goals to plan out their annual year products and headcount.
To publish and communicate the strategy, use the method of OKRs. My best OKR experience was when each employee started with a blank gdoc indexed to the organization structure. Transparency of everyone’s goals, starting with senior leadership, builds organizational trust and confidence. OKRs can be a key tool for organization change. Transparency of leadership’s goals is an important aspect of open source behavior derived from Open Source First. Once the Platform SVP publishes the annual goals as OKRs, everyone can read, write, discuss, and debate the annual strategy.
The organization then updates their OKRs quarterly. Senior leadership should take no longer than a week to create and publish their OKRs. Senior leadership and the rest of the organization publishes their OKRs for the next quarter 2/3’s of the way through the previous quarter. Take no more than a couple of weeks following the first round of OKR publishing, to debate and revise any major inconsistencies. That means timing wise, the whole organization will have their OKRs completed for the next quarter, weeks before that quarter starts.
Do not be tempted to create a hierarchy of OKRs from leadership on down. I have never seen it work well. If your leadership understands your products and customers, then their goals will be very similar to the rest of the organization. Senior leadership cannot understand all the details of the working parts of the organization. Additionally, if everyone waits for the leadership goals, before starting their own, it will cause delays of weeks to months. It is better to get 80% accuracy in your quarterly goals while publishing them on time. Think quarterly OKR train release.
Points to highlight:
Each person maintains 3-5 OKRs each quarter. OKRs should be their priorities only.
OKRs are not meant to be a project or product management system
Transparency of everyone’s goals to improve collaboration
Each manager holds their directs responsible for their OKRs. Use your directs OKRs as part of their leadership mentoring.
Each person rates the success of their OKRs. 60-80% OKR success rate is what you want. You can also call this stretch goals.
Let Transparency Take Hold
In conclusion, take this outline as just that, the broad strokes. This outline is focused on the process steps that can allow development teams to independently create their own way of running their teams within the platform organization. When you have a well structured organization with milestones, transparency, and good communication, it encourages merit based work from everyone. And that is a place, I want to work at.
I joined the Linux Foundation and many other open source professionals at the Open Source Leadership Summit OSLS in Squaw Valley this week. It was full of great and interesting people who make the open source community live and thrive. I can’t mention everyone, so I will just call out two of the exceptional people, Sarah Novotny and Jono Bacon. Both Sarah and Jono have a track record of consistent community leadership. Both of them spoke at the OSLS. I would like to re-emphasize a common open source community theme they hit on.
Clear Communication Leads to Strong Communities
The community supporting the projects needs to understand where they are headed. The practice of “extreme clarity” in all aspects of the community activities, builds strong communities. For example, as commercial products are relying on the projects for feature development and bug fixes, clear communication on the product strategy helps everyone considerably. Transparency and good communication between project and product teams leads to successful outcomes. This obviously makes sense in the private development of products. When combining upstream and downstream work, it is even more critical.
Failures are a common occurrence. When missing a development deadlines, how is it handled? The community must meet the challenge head on and discuss the problem transparently and publicly. Not communicating well or ignoring the problem leads to distrust of the release schedule and project strategy. Not every problem needs a full debate, but the process of discussion, resolution, and documentation, leads to broad acceptance of outcomes.
The community processes themselves must not be taken for granted. The people involved in the projects, governance, and products with evolve over time. By holding consistent, transparent reviews of the community processes, the communication will continue to be strong. It is often taken for granted how the community functions. It shouldn’t be. New contributors need to understand why and how the community mechanics work. They need to be able to debate their views on how to improve the process of contribution.
To conclude my point here, clear communication is a key attribute of successful communities. Consistent contribution, as Linus Torvalds spoke about at OSLS, is critical for successful projects. But without clear communication, consistent contribution cannot be maintained.
I want to focus on one specific aspect of project to product development from the Open Source First article. Coordinating upstream (publicly licensed) and downstream (private non-licensed) development work. In any organization that is developing and supporting software, there are some number of engineers that work on public projects. It is a common problem that the Development Managers and the Scrum Masters do a poor job of tracking that public work. Let’s assume that the Development Manager is aware and supports the public development work.
Support the Developer
What typically happens is that upper management is not aware of what the upstream work purpose is and its importance to supporting the overall development strategy. What likely started as support or even direction from the Development Manager to work on the public project evolves into a fog of light understanding.
Many months later, when performance reviews and head count updates come around, typically the upstream developer gets lets left out, as the public work is not part of what upper management understands. This leads to either dropping the upstream work, alienating one of your likely best developers, and/or the upstream work continues, but with zero Development Manager insight. In some cases all three things happen, in which case, you now have a increasingly separated, forked development effort, with a increasing annoyed developer, that is going to be putting in more and more time into the public projects. Not a good workplace situation for the developer or the manager.
Treat Public Projects Like an Company
It takes more work, but there is a proven solution. Treat the public projects like another engineering organization. If you are going to work with another engineering team, you expect to have a clear understanding of responsibilities and timelines. You need to have the exact same expections when working with a public project.
Right at the beginning, when the developer has plans to work upstream, set out responsibilities and timelines. This is just as much to protect the developer as is it is the company. Define what the public work deliverable will be and when it will be delivered. Make sure your Product Managers understand how the public work contributes to delivering the product. Make sure the Scrum Master works with the upstream developer just like the rest of the downstream developers. The upstream developer is still part of the team. As the upstream work starts to take shape, get the code into a testable branch on your software pipeline. Make sure you have quality unit and build tests to verify the upstream work, so it can be more easily merged into your code base trunk when the time comes.
Track all your development work and report on progress. Highlight the upstream and downstream work. Update leadership on the upstream progress. Keep up to date with the public projects release schedule and strategy. Publish both your private release schedule along side the public projects release schedule.
Make It Happen
This is just the broad strokes. You will need to take this and put in specifics for your implementation. Organization structure and tools vary greatly, but hold to the basic tenants outlined above.
I get nervous before every job interview. Most of us do whatever we can to avoid getting in the situation of having a job interview. That generally means avoiding career planning as well. The messy truth is that we put off uncomfortable activities. I have put together a simple step by step plan that can help you make working on your career a positive experience. We all need some help with learning and growing. Your career is no different.
We are developing your career understanding by creating a plan. This plan is for increasing your personal capabilities to make your career goals happen. To create a truly effective plan, you need to pick a mentor to work with. This mentor can be a wife, boyfriend, mother, co-worker, or boss. The mentor you choose needs to work with you on creating your plan. It is helpful, but not mandatory for your mentor to be in the same career path. You need your mentor to be slightly senior to you skill wise, and be available over the next 3 to 12 months to work with you. I have practiced different forms of these steps below over the years at many companies and situations. As a result, I have a list of ten mentors that I ask for advice and council. You just need one.
Let’s dig into who you are and what you want to develop into. Follow the steps below to get to know your future self.
Create a long term career goal
We need to figure out what do you want to be doing in three years. Your goal need to be attainable, but a stretch to accomplish. It can be a skill like software development, managing people, or leading organizational change. Or it can be a specific role like Engineering VP, Senior Project Manager, or OpenStack open source developer.
What skills do you excel at? What skills do you fall short? As you are reviewing your skills, you may find things that must be fixed immediately, others can be areas to be developed over time, and still others you can ignore. Are any of the areas you need to improve on can be classified as character flaws? For example, if you have a habit of lying to your co-workers, that is a problem that must be fixed quickly. Your character is the core of your working career. If you have some software development skills to improve and you are a software developer, then you need to improve this skill over time as your technical skills are important to your position. If you are not very good at following up on the details of a large plan, you can augment your team to fill the skill gap. You can ignore the flaw, because you can correct the gap with your team.
You should now have a long term career goal and short list of attributes labeled good, improve, and ignore.
We have a rough career strategy at this point and a mentor to lean on. You know what you want your future you to be. Congratulations, most people do not get this far.
Company long term goals
Before we jump into your career development steps, you need to understand your organization business objectives. Where is your organization going to be over the few years? What is the business strategy to get there? If you are not sure or do not know, then you need to figure it out.
Ask your mentor for help. Aligning your goals with your organization is critical to figuring out where and how you fit.
Your short term goals
Next, by brainstorming with your mentor, we create a few one year goals. These should be steps towards accomplishing your long term goal of what you want to be doing in three years. Utilize the goal attributes of specific measurable, attainable, relevant, and time-bound. See https://en.m.wikipedia.org/wiki/SMART_criteria
Your activities to complete goals
We will need to create a plan for accomplishing each goal. You will learn more from mistakes and failures while doing what you are learning rather than reading or watching. So I would lean heavily on activities that get you directly involved in doing something productive. Document what resources you need Include some thoughtful details on how you can be successful. Come up with alternatives if you have them. For example, with a 1 year goal learning multi-threaded python software development, aligns with company objective of 99.999% production uptime, then document (3) multi-month activities, each with a timeline, that will get you to a good proficiency level within a year. Taking a month long university python development extension course, working with python test driven development tutorials, and working on an open source python project feature would be good activities.
Now you have a thought out plan that you have co-developed with your mentor. It is time to socialize the plan with your manager. Make adjustments depending on your manager’s feedback. Be thoughtful on what your manager is telling you. Does your manager understand and support your career goals? It shouldn’t be too different that what you expected. If there are large differences between your goals and your manager’s expectations for you, then you need to double check with your mentor if you are being realistic with your goals. If you can not reconcile your goals with your organization, then you need to start thinking about a transfer. Talk it over with your mentor. Reach out to your Human Resources department to start looking for where your career job opportunities are. Take a look at linkedin jobs https://www.linkedin.com/jobs/view-all. It is likely that a job opening already exists for the career you’re aiming for. Before you make any job changes, talk to a few hiring managers. Ask them questions before you decide to make a change. You may find that the grass isn’t greener over there.
After you have fine tuned your career plan, it is time to make it happen. Pick the easiest goal off the plan and start working with your mentor and manager to accomplish your dreams.
Have regular meetings with your mentor and manager. Gauge your progress against what you expected. Make adjustments in your plans as necessary. Keep your eye on your long term career goal.
As you get about halfway through the year, review your career plans. Make adjustments. Start planning your next year of short term goals.
Your resume is very similar to a career plan. A resume is all about what you have done, but it tells the story of your career. If you understand your career goal, then you should be able to glance at your resume to know if you are on track to success. Having your resume up to date is a great confidence builder. Nothing says you are ready for the future like a ready resume.